Sayyiduna Abu Hurayrah (may Allah be pleased with him) narrates that the Messenger of Allah (peace and blessings be upon him) said: "Fasting is not only from eating and drinking; rather, fasting is refraining from idle talk and obscenity. If someone abuses you or quarrels with you, say to him: 'I am fasting.'"
Hadith Referenceالسنن الكبرى للبيهقي / كتاب الصيام / 8312