Sayyida Asma bint Abi Bakr (may Allah be pleased with her) used to wear clothes dyed with a deep kashm color in ihram, but there was no saffron in them.
Imam Malik, may Allah have mercy on him, was asked: If there is fragrance on a garment, and then its scent disappears, is it permissible to wear it in the state of Ihram? He said: Yes, as long as the color of saffron or wars does not remain on it.
Hadith Referenceموطا امام مالك رواية يحييٰ / كتاب الحج / 714