Sayyiduna Abdullah bin Abbas (may Allah be pleased with them both) said: "I did not see the Messenger of Allah (peace and blessings be upon him) touch anything other than the two Yamani corners."
Hadith Referenceالسنن الكبرى للبيهقي / كتاب الحج / 9240