Ion binding to acidic groups is a central mechanism for ion-specificity of macromolecules and surfaces. Depending on pH, acidic groups are either protonated or deprotonated and thus change not only charge but also chemical structure with crucial implications for their interaction with ions. In a two-step modeling approach, we first determine single-ion surface interaction potentials for a few selected halide and alkali ions at uncharged carboxyl (COOH) and charged carboxylate (COO-) surface groups from atomistic MD simulations with explicit water. Care is taken to subtract the bare Coulomb contribution due to the net charge of the carboxylate group and thereby to extract the nonelectrostatic ion-surface potential. Even at this stage, pronounced ion-specific effects are observed and the ion surface affinity strongly depends on whether the carboxyl group is protonated or not. In the second step, the ion surface interaction potentials are used in a Poisson-Boltzmann model to calculate the surface charge and the potential distribution in the solution depending on salt type, salt concentration, and solution pH in a self-consistent manner. Hofmeister phase diagrams are derived on the basis of the long-ranged forces between two carboxyl-functionalized surfaces. For cations we predict direct, reversed, and altered Hofmeister series as a function of the pH, qualitatively similar to recent experimental results for silica surfaces. The Hofmeister series reversal for cations is rationalized by a reversal of the single-cation affinity to the carboxyl group depending on its protonation state: the deprotonated carboxylate (COO-) surface group interacts most favorably with small cations such as Li+ and Na+, whereas the protonated carboxyl (COOH) surface group interacts most favorably with large cations such as Cs+ and thus acts similarly to a hydrophobic surface group. Our results provide a general mechanism for the pH-dependent reversal of the Hofmeister series due to the different specific ion binding to protonated and deprotonated surface groups.