It is narrated from Abu Huraira (may Allah be pleased with him) that the Messenger of Allah (peace and blessings be upon him) said: "When you say to your companion 'Be quiet' while the Imam is delivering the sermon on the day of Jumu'ah, then you have engaged in idle talk yourself."
Hadith Referenceمسند الإمام الشافعي / كتاب الجمعة / 428