It is narrated from Sayyiduna Ibn Abbas (may Allah be pleased with them both) that the Prophet (peace and blessings be upon him) ordered the women and children of Banu Hashim to depart early on the night of Muzdalifah.
Hadith Referenceمسند احمد / ومن مسند بني هاشم / 3159