Sayyiduna Abdullah bin Umar (may Allah be pleased with them both) narrates that when we did not find someone present in the 'Isha and Fajr prayers, we would think badly of him (that he had become a hypocrite).
Hadith Referenceصحيح ابن خزيمه / كتاب الإمامة فى الصلاة وما فيها من السنن مختصر من كتاب المسند / 1485