Specific counterion effects represented by Hofmeister series are important for a variety of phenomena such as protein precipitations, surface tensions of electrolytes solutions, phase transitions of surfactants, etc. We applied heterodyne-detected vibrational sum-frequency generation spectroscopy to study the counterion effect on the interfacial water at charged interfaces and discussed the observed effect with relevance to the Hofmeister series. Experiments were carried out for model systems of positively charged cetyltrimethylammonium monolayer/electrolyte solution interface and negatively charged dodecylsulfate monolayer/electrolyte interface. At the positively charged interface, the intensity of the OH band of the interfacial water decreases in the order of the Hofmeister series, suggesting that the adsorbability of halide anions onto the interface determines the Hofmeister order as previously proposed by Zhang and Cremer (Curr. Opin. Chem. Biol. 2006, 10, 658-663). At the negatively charged interfaces, on the other hand, the OH band intensity does not depend significantly on the countercation, whereas variation in the hydrogen-bond strength of the interfacial water is well correlated with the Hofmeister order of the cation effect. These results provide new insights into the molecular level mechanisms of anionic and cationic Hofmeister effects.