Abdullah bin Busr (may Allah be pleased with him) says: I was sitting to one side on Friday when a man came stepping over the necks of the people. The Prophet (peace and blessings be upon him) said to him: "Sit down, for you have caused harm and you have come late." Abu al-Zahiriyah (may Allah have mercy on him) says: We used to talk among ourselves until the imam came out.
Hadith Referenceالسنن الكبرى للبيهقي / كتاب الجمعة / 5886