It is narrated from Ibn Abbas (may Allah be pleased with them both) that he said: "There is no impediment (ihsar) except the enemy." One of Ibn Tawus and Amr bin Dinar narrated these additional words: "Now even that (enemy's) impediment has ended."
Hadith Referenceمسند الإمام الشافعي / كتاب الحج / 942