Ata' narrates from Sayyiduna Abdullah bin Abbas (may Allah be pleased with them both) that he used to say: "The matter rests with the master, whether he (the slave) was given permission or not." Then he recited this verse: "Allah presents an example: a slave owned by another, who has no power over anything." [Surah al-Nahl: 75]
It has also been narrated in the hadith of Abu Ma'bad from Ibn Abbas what indicates this.
Hadith Referenceالسنن الكبرى للبيهقي / كتاب النكاح / 13857