Sayyiduna Mu'adh bin Jabal (may Allah be pleased with him) says that the Messenger of Allah (peace and blessings be upon him) sent me to Yemen and commanded me "to take from every adult one dinar or its equivalent in Ma'afiri cloth." Yahya bin Adam says: The people of Yemen were Arabs, and the jizyah was taken from them because they are People of the Scripture; and he said: Do you not see that he (peace and blessings be upon him) said: "A Jew shall not be lured away from his Judaism."
Hadith Referenceالسنن الكبرى للبيهقي / كتاب الجزية / 18643