It is narrated from Sayyiduna Jubair bin Mut’im (may Allah be pleased with him): On the Day of Arafah, I went to Arafah in search of a lost camel and saw that the Prophet (ﷺ) was standing in Arafah with the people. I said (in my heart), “They have left Hums; what has happened to them that they have come out of the Haram?” That is, Hums refers to the Quraysh, who used to stand at Muzdalifah and would say, “We are Hums; we will not go outside the boundaries of the Haram.”
Hadith Referenceالسنن المأثورة / باب ما جاء في فدية الأذى / 462