The numerical solution of PDE-constrained optimization problems subject to the non-stationary Navier-Stokes equation is a challenging task. While space-time approaches often show favorable convergence properties they often suffer from storage problems. We here propose to approximate the solution to the optimization problem in a low-rank from, which is similar to the Model Order Reduction (MOR) approach. However, in contrast to classical MOR schemes we do not compress the full solution at the end of the algorithm but start our algorithm with low-rank data and maintain this form throughout the iteration. Theoretical results and numerical experiments indicate that this approach reduces the computational costs by two orders of magnitude.