Sayyiduna Zayd bin Khalid (may Allah be pleased with him) said that the Messenger of Allah (peace and blessings be upon him) said: "Were it not that it would be hard on my Ummah, I would have commanded them to use the siwak (tooth-stick) at every prayer." So Sayyiduna Zayd (may Allah be pleased with him) used to keep the siwak on his ear in the same place where a scribe keeps his pen, and whenever he stood up for prayer he would use the siwak.
Hadith Referenceالمعجم الكبير للطبراني / باب الزاي / 5205