Al-Aswad bin Yazid (may Allah have mercy on him) narrates that he heard Abu Musa (may Allah be pleased with him) say: "My brother and I came from Yemen, and for some time we thought only that Abdullah bin Mas'ud was one of the household of the Prophet (peace and blessings be upon him), because of how (frequently) he went in to the Prophet (peace and blessings be upon him)."
Hadith Referenceالمعجم الكبير للطبراني / باب العين / 8465