Sayyidah Aishah (may Allah be pleased with her) said that the Messenger of Allah (peace and blessings be upon him) said: "Were it not that your people's time of polytheism is so recent, I would have demolished the Ka'bah, set it level with the ground, made two doors for it, on the eastern and western sides, and included in it the six cubits up to the wall, for Quraysh cut it short when they built it."
Hadith Referenceالسنن الكبرى للبيهقي / كتاب الحج / 9317