It is narrated from Tawus (may Allah have mercy on him) that he said: In the days of ignorance, people would return from Arafat before the sun had set and from Muzdalifah after the sun had risen, and they would say: "O Thabir (the name of a mountain), shine so that we may return." So Allah the Exalted delayed (the departure from Arafat) until sunset and made (the departure from Muzdalifah) earlier (i.e., before sunrise).
Hadith Referenceمسند الإمام الشافعي / كتاب الحج / 997
Hadith Takhrijصحيح من غير هذا الطريق : اخرجه البخاري الحج، باب متى يدفع من جمع (1684)۔