It is narrated from Sayyiduna Ibn Abbas (may Allah be pleased with them both) that the Messenger of Allah (peace and blessings be upon him) addressed the people while his back was towards the Ka'bah, between the Black Stone and the door.
Hadith Referenceالمعجم الكبير للطبراني / باب العين / 11191