Sayyiduna Abu Hurayrah (may Allah be pleased with him) says that the Prophet (peace and blessings be upon him) said: "When you silence your companion on Friday while the imam is delivering the sermon, you have engaged in idle talk."
Hadith Referenceالسنن الكبرى للبيهقي / كتاب الجمعة / 5827