It is narrated from Sayyiduna Abu Hurayrah (may Allah be pleased with him) that the Messenger of Allah (peace and blessings be upon him) said: "Everything has some rubbish (or refuse), and the rubbish of the mosque is 'No, by Allah' and 'Yes indeed, by Allah' (such pointless oaths and idle talk)."
I say: Rishdin (the narrator) is da'if (weak).
However, it has a shahid (supporting narration) from the hadith of Sayyiduna Abu Qirsafah (may Allah be pleased with him), which Imam al-Tabarani (may Allah have mercy on him) recorded in al-Mu'jam al-Kabir.
The word 'qumamah' (with a dammah on the qaf) linguistically means sweepings and rubbish.
Hadith Referenceإتحاف الخيرة المهرة بزوائد المسانيد العشرة / كتاب المساجد / 1003
Hadith Gradingمحدثین:إسناده ضعيف
Hadith Takhrij«قلت: رشدين ضعيف» «أخرجه المعجم الأوسط للطبراني برقم: 827، وأخرجه مجمع الزوائد ومنبع الفوائد برقم: 2041»