Sayyiduna Abdullah bin Abbas (may Allah be pleased with them both) narrates: When the Prophet (peace and blessings be upon him) left Makkah, he said (addressing Makkah): "By Allah, I am indeed leaving you, yet I know that you are the most beloved of all Allah's lands to me and the most honoured in the sight of Allah Almighty. Had your people not forced me to leave, I would not have left." (Later the Prophet (peace and blessings be upon him) said:) "O Banu Abd Manaf, if you become the custodians of this affair after me, do not prevent anyone performing tawaf of the House of Allah at any hour of the night or day. Were it not for the fear that Quraysh would become arrogant, I would have told them what rank and standing they have with Allah Almighty." (Then he supplicated:) "O Allah, You made the first of them taste ruin, so make the last of them taste favour." Al-Tirmidhi recorded part of this narration.
Abu Ya'la recorded it, and its narrators are thiqat (trustworthy).
Hadith Referenceمجمع الزوائد ومنبع الفوائد / كتاب الحج / 5691