Phase compensated proportional resonant (PR) controllers with phase leading angles embedded into resonant controllers, are used to improve the system stability, when the grid-connected inverter is connected to a weak grid with large grid impedance. However, an inappropriate phase leading angle will cause obvious anti-peak in the magnitude response of phase compensated PR controller. Thus, the phase leading angle should be selected carefully. The conventional phase compensation (CPC) method for selecting the phase leading angle can maximize the system phase margin, but at the expense of large peaks in the magnitude response of sensitivity transfer function. Meanwhile, the bandwidths of resonant controllers will be reduced. In the paper, an improved phase compensation (IPC) method is proposed, which does not cause peaks in the magnitude response of sensitivity transfer function while providing sufficient phase margin. Experimental tests were carried out on a single phase grid-connected inverter. The effectiveness of the proposed method is proved in terms of the steady-state and transient performances, as well as the performance under the grid frequency deviation.