It is narrated from Ata bin Yasar (may Allah have mercy on him) that the Messenger of Allah (peace and blessings be upon him) said: "When one of you has a doubt about the prayer and does not know how many rak'ahs he has prayed, three or four, let him pray one rak'ah and perform two prostrations while sitting before giving the salam. If the rak'ah he prayed was the fifth, it will be made even by these two prostrations, and if the rak'ah he prayed was the fourth, then these two prostrations will be a cause of humiliation and disgrace for Satan."
Hadith Referenceالسنن الكبرى للبيهقي / كتاب الصلاة / 3801