It has been narrated on the authority of Anas bin Malik (RA) that the Messenger of Allah (ﷺ) would not come (back) to his family by night. He would come to them in the morning or in the evening.
Hadith ReferenceSahih Muslim / The Book on Government / Hadith: 1928