Sayyidah Aishah (may Allah be pleased with her) said that the Messenger of Allah (peace and blessings be upon him) said: "Do you not see that when your people built the Ka'bah, they fell short of the foundations of Ibrahim (peace be upon him)?" She said: I said: "O Messenger of Allah (peace and blessings be upon him), then why do you not restore it to the foundations of Ibrahim (peace be upon him)?" The Messenger of Allah (peace and blessings be upon him) said: "Were it not that your people have only recently left disbelief, I would have done so." Abdullah bin Umar (may Allah be pleased with them both) said: If Aishah (may Allah be pleased with her) heard this from the Messenger of Allah (peace and blessings be upon him), then I think he (peace and blessings be upon him) left touching the two corners next to the Black Stone only because it was not completed on the foundations of Ibrahim (peace be upon him).
Hadith Referenceالسنن الكبرى للبيهقي / كتاب الحج / 9243