Sayyiduna Ibn Abbas (may Allah be pleased with them both) said: "Talking is disliked on four occasions: on the day of Jumu'ah, on the day of Eid al-Fitr, on the day of Eid al-Adha, and in the prayer for rain (istisqa'), when the imam has ascended the pulpit and is delivering the sermon, until he comes down."
Hadith Referenceالمعجم الكبير للطبراني / باب العين / 11044