In this paper we have studied the extent to which the accuracy of quantitative analysis by x-ray photoelectron spectroscopy is affected by neglect of the shake-up electrons. Three sets of alloys (CuAu, CoNi, and CuNi) were studied. It was found that the accuracy of quantification from first principles based on the intensity in a narrow energy range around the peak energy can lead to errors of up to 114%. The error is discussed in terms of the importance of shake-up effects. When reference spectra are applied, these effects are reduced and the Shirley method for background estimation then yields compositions for the considered alloys that differ by between 2% and 22% from those determined with an algorithm that is based on a physical description of the inelastically scattered electrons. However if, as found for CuAu, the shake-up in the reference sample is strongly different from that of the alloy, it may be less accurate to use reference spectra with the Shirley method.