Do I need to have an actual office space to set up my company in International Free Zone Authority Dubai?

There is no requirement for an office space to set up a company with IFZA. IFZA will provide you with office rental agreement with all licenses.