Sayyiduna Sa'd bin Ubadah (may Allah be pleased with him) said: I came to the Prophet (peace and blessings be upon him) while he (peace and blessings be upon him) was in a house. I stood facing the door and began asking permission to enter, so he (peace and blessings be upon him) gestured to me to move away. Then I came and asked permission, and he (peace and blessings be upon him) said: "Asking permission is only so that the gaze does not fall (inside)."
Hadith Referenceالمعجم الكبير للطبراني / باب السين / 5367