Imam Malik (may Allah have mercy on him) says that Sayyiduna Abdullah bin Amr and Sayyiduna Abu Hurayrah (may Allah be pleased with them both) were asked about a man who had entrusted his authority to his wife, but the wife returned it to the husband without using it. Both of them said: This is not a divorce.
Hadith Referenceالسنن الكبرى للبيهقي / كتاب الخلع والطلاق / 15043