Sayyidah Aishah (may Allah be pleased with her) said: I asked the Messenger of Allah (peace and blessings be upon him) whether the wall is part of the House of Allah. He (peace and blessings be upon him) said: "Yes!" I said: Why then did they not include it in the House of Allah? He (peace and blessings be upon him) said: "Your people ran short of funds." I said: Why is its door raised high? He (peace and blessings be upon him) said: "Your people did this so that they could let in whomever they wished and keep out whomever they wished. Were it not that your people are recently out of the Jahiliyyah, and were there not the fear of disapproval in their hearts, I had thought to include the wall in the House of Allah and to set its door level with the ground."
Hadith Referenceالسنن الكبرى للبيهقي / كتاب الحج / 9315