It is narrated from Sayyiduna Jabir (may Allah be pleased with him) that the beginning of Sayyiduna Umar's (may Allah be pleased with him) acceptance of Islam was as follows. Sayyiduna Umar (may Allah be pleased with him) related: "One night my sister went into labour, so I left the house and, on a bitterly cold night, hid among the coverings of the Ka'bah. Then the Prophet (peace and blessings be upon him) came and entered the Hijr (Hatim), wearing sandals. He (peace and blessings be upon him) prayed as much as Allah willed and then turned to leave. I heard from him (peace and blessings be upon him) speech the like of which I had never heard before, so I came out (from the coverings) and followed him (peace and blessings be upon him). He (peace and blessings be upon him) asked: 'Who is it?' I said: 'Umar.' He (peace and blessings be upon him) said: 'O Umar, will you not leave me alone, neither by night nor by day?' I feared that he (peace and blessings be upon him) might invoke against me, so I said: 'Ashhadu an la ilaha illallah wa annaka Rasulullah' (I bear witness that there is no deity worthy of worship except Allah and that you are indeed the Messenger of Allah (peace and blessings be upon him)). He (peace and blessings be upon him) said: 'O Umar, keep it (your Islam) hidden.' I said: 'No, by the One who sent you with the truth, I will proclaim it openly just as I used to proclaim polytheism openly.'"
Imam Abu Bakr bin Abi Shaybah recorded it through Yahya bin Ya'la al-Aslami, and he is a da'if (weak) narrator.