It is narrated from Sayyiduna Ibn Amr, may Allah be pleased with him, that the Noble Prophet, peace and blessings be upon him, said: "It is not lawful for any person to intrude and sit among people without their permission."
Hadith Referenceمسند احمد / مسند المكثرين من الصحابة / 6999