Space-time finite element discretizations of time-optimal control problems governed by linear parabolic PDEs and subject to pointwise control constraints are considered. Optimal a priori error estimates are obtained for the control variable based on a second order sufficient optimality condition.