Abis bin Rabi’ah says: I saw Sayyiduna Umar bin Khattab (may Allah be pleased with him) turning towards the Black Stone and kissing it, then he said: Beware! By Allah, you are only a stone, you can neither benefit nor harm me. If I had not seen the Prophet (peace be upon him) kissing you, I would never have kissed you.
Related hadith on this topic