Sayyiduna Umar (may Allah be pleased with him) said to a man whose Islam was outward and who was known (for something else): I think you are only seeking refuge (in Islam). He said: In Islam there is what has given me refuge. He (may Allah be pleased with him) said: Yes! Islam gives refuge to whoever seeks refuge in it.
Hadith Referenceالسنن الكبرى للبيهقي / كتاب المرتد / 16851