1997. Commentary: There was a restriction for the Muhajirun (emigrants) of Madinah that they should not stay in Makkah for more than three days. For other Muslims, there is no such restriction of any kind—whether they remain or return.
Source: Sunan Abu Dawood – Commentary by Shaykh Umar Farooq Saeedi, Page: 1997