It is narrated from Sayyiduna Ibn Umar (may Allah be pleased with them both) that when there are four people, then there is no harm if two of them converse separately.
Hadith Referenceالادب المفرد / كتاب / 1172
Hadith Gradingالألبانی:صحيح
Hadith Takhrij«صحيح : أنظر الحديث ، رقم : 1170»
Explanation & Benefits
Shaykh Maulana Usman Muneeb
Benefits and Issues: The point is that when there are four or more people, there is no harm if two separate themselves and converse, because in this way there is no risk of creating suspicion.
Source: Fadlullah al-Ahad: Urdu Commentary on al-Adab al-Mufrad, Page: 1172