It is narrated from Sayyiduna Abu Mas'ud (may Allah be pleased with him) that a man had a slave who was a butcher. He said to his slave: "Prepare enough food for us for five or six men; perhaps I may invite the Messenger of Allah (peace and blessings be upon him) as the fifth." So he invited them, and a man followed them. The Prophet (peace and blessings be upon him) said: "You invited me as the fifth of five men, and this man has followed us. If you wish, permit him, and if you wish, leave him." He said: "Rather, I permit him."
Hadith Referenceالمعجم الكبير للطبراني / باب العين / 14601