It is narrated from Sayyiduna Ibn Umar (may Allah be pleased with them both) that the Messenger of Allah (peace and blessings be upon him) said: "O Banu Abd Manaf, let me not find that you have prevented anyone from performing tawaf of this House at any hour of the night or day."
Hadith Referenceالمعجم الكبير للطبراني / باب العين / 13454