Imam al-Shafi'i (may Allah have mercy on him) said: "And these are people who held firm to Islam but refused (to pay) zakah."
And he cited as evidence for this the incident of Sayyiduna Abu Bakr and Sayyiduna Umar (may Allah be pleased with them both).
Hadith Referenceالسنن الكبرى للبيهقي / كتاب قتال أهل البغي / Q16730