Ismail bin Abi Khalid says that I asked Hazrat Abdullah bin Abi Awfa (may Allah be pleased with him), "Did the Prophet Muhammad (peace and blessings be upon him) enter the House of Allah (the Ka'bah) during Umrah?" He replied, "No."
Hadith Referenceمسند احمد / تتمہ مُسْنَدِ الْكُوفِيِّينَ / 19125