Sayyiduna Ibn Abbas (may Allah be pleased with them both) said: "The Messenger of Allah (peace and blessings be upon him), Abu Bakr and Umar (may Allah be pleased with them both) used to lower the deceased into the grave from the direction of the qiblah."
Hadith Referenceالمعجم الكبير للطبراني / باب العين / 11066