Sayyiduna Abu Umamah (may Allah be pleased with him) narrated that the Prophet (peace and blessings be upon him) said: "Women stay with their husbands wherever their husbands are, except the women of the Ansar, who are not to be taken out of Madinah."
This Ja'far bin al-Zubayr is very da'if (weak).
Hadith Referenceالسنن الكبرى للبيهقي / كتاب الصداق / 14444