Based on the orbital invariant formulation of Moller-Plesset (MP) perturbation theory, analytical energy gradients have been formulated and implemented for local second order MP (LMP2) calculations. The geometry-dependent truncation terms of the LMP2 energy have to be taken into account. This leads to a set of coupled-perturbed localization (CPL) equations which must be solved together with the coupled-perturbed Hartree-Fock (CPHF) equations. In analogy to the conventional non-local theory, the repeated solution of these equations for each degree of freedom can be avoided by using the z-vector method of Handy and Schaefer. Explicit equations are presented for the Pipek-Mezey localization. Test calculations on smaller organic molecules demonstrate that the local approximations introduce only minor changes of computed equilibrium structures.