Sahih Ibn Khuzaymah

Sayyiduna Abu Hurairah (may Allah be pleased with him) narrates that the Messenger of Allah (peace and blessings be upon him) said: "Indeed, the heaviest prayers upon the hypocrites are the ‘Isha and Fajr prayers. And if they knew how much reward and virtue there is in them, they would surely attend them, even if they had to crawl on their knees. Indeed, I have intended to order the prayer to be established, then I would appoint a man to lead the people in prayer, and I would go with a group of people carrying bundles of firewood to burn down the houses of those who stay behind from the congregational prayer along with their houses." This is the narration of Ibn Numayr, and in the narration of Abu Mu‘awiyah, the words are: "I have intended?" And he said: "Then I would order a man to lead the people in prayer, then I would go with some people who have bundles of firewood with me to those who stay behind from the prayer, and I would burn their houses over them with fire."

Sayyiduna Abdullah bin Umar (may Allah be pleased with them both) narrates that when we did not find someone present in the 'Isha and Fajr prayers, we would think badly of him (that he had become a hypocrite).