It is narrated from Sayyiduna Abdullah bin Mas'ud (may Allah be pleased with him) that the Prophet (peace and blessings be upon him) said: "When three people are together, two should not converse privately apart from the third, for that grieves him."
Hadith Referenceالمعجم الكبير للطبراني / باب العين / 10208