Sa'd bin Ubaydah says that Ibn Umar (may Allah be pleased with him) heard a man swearing by the Ka'bah, so he said to him: Do not swear by the Ka'bah, for I heard the Prophet (peace and blessings be upon him) saying: "Whoever swears by other than Allah has committed disbelief (kufr) or associated partners (shirk)."
Sa'd bin Ubaydah did not hear this from Ibn Umar.
Hadith Referenceالسنن الكبرى للبيهقي / كتاب الأيمان / 19829