An efficient third-order propagator method to compute ionization potentials and electron affinities of atoms and molecules is presented. The development is based on the algebraic diagrammatic construction (ADC) representing a specific reformulation of the diagrammatic perturbation series of the electron propagator G(omega). In contrast with previous approximation schemes, relying on the Dyson equation and approximations for the self-energy part, the ADC procedure here is applied directly to the (N-/+1)-electron parts G(-)(omega) and G(+)(omega), respectively, of the electron propagator. This leads to decoupled secular equations for the ionization energies ((N-1)-electron part) and electron affinities ((N+1)-electron part), respectively. In comparison with the Dyson-type approach, there is a substantial reduction of the secular matrix dimension opposed by a small additional expense in computing some second- and third-order contributions to the secular matrix elements. The relationship of the non-Dyson ADC(3) method to coupled cluster methods is outlined.