Sa'id bin Jubayr narrates from Sayyiduna Abdullah bin Abbas (may Allah be pleased with them both): "Whoever hears the adhan and does not respond to it (that is, does not come for the prayer) has no prayer, except with an excuse."
Hadith Referenceالسنن الكبرى للبيهقي / كتاب الجمعة / 5585