Sayyiduna Jabir (may Allah be pleased with him) narrates: Smiling does not invalidate the prayer, but laughing out loud does.
This is the preserved (mahfuz) version, as mawquf (a statement of the Companion). Thabit bin Muhammad al-Zahid narrated it as marfu' (from the Prophet), and that is an error (wahm) on his part.
Hadith Referenceالسنن الكبرى للبيهقي / كتاب الصلاة / 3359
It is narrated from Sayyiduna Jabir bin Abdullah (may Allah be pleased with them both) that he (peace and blessings be upon him) said: "Smiling does not invalidate the prayer, but laughing out loud breaks the prayer."
Another hadith has been narrated about smiling in prayer, the like of which is not used as proof.
Hadith Referenceالسنن الكبرى للبيهقي / كتاب الصلاة / 3360
Sayyiduna Jabir bin Abdullah (may Allah be pleased with them both) narrates: We were praying with the Messenger of Allah (peace and blessings be upon him) during an expedition, and he (peace and blessings be upon him) smiled during the prayer. When he (peace and blessings be upon him) completed his prayer, we said: O Messenger of Allah, we saw you smile. He (peace and blessings be upon him) said: "Mika'il (peace be upon him) passed by me, with traces of dust on his wings, returning from pursuing the people (the enemy). He laughed towards me, so I smiled at him."
We have already narrated in the Book of Purification the report of Sayyiduna Jabir (may Allah be pleased with him) from Abu Sufyan. It concerns the person who laughed in prayer: he should repeat the prayer but does not need to repeat the ablution.
We have also narrated from Sayyiduna Abu Musa al-Ash'ari (may Allah be pleased with him), in an incident reported from him: "Whoever among you laughed should repeat the prayer." A similar report is also narrated from Sayyiduna Ibn Mas'ud (may Allah be pleased with him).
Al-Wazi' bin Nafi' al-Uqayli al-Jazari has been criticised (by the scholars).
Hadith Referenceالسنن الكبرى للبيهقي / كتاب الصلاة / 3361