Ata' bin Dinar (may Allah have mercy on him) says that Sayyiduna Umar (may Allah be pleased with him) said: Do not learn the language of the non-Arabs, and do not enter upon the polytheists in their places of worship on the day of their festival, for the displeasure of Allah descends upon them.
Hadith Referenceالسنن الكبرى للبيهقي / كتاب الجزية / 18861