Ultrafast polarization and wavelength selective IR pump-probe spectroscopy is used to measure the inertial and long time orientational dynamics of pi-hydrogen bonding complexes. Inertial orientational relaxation is sensitive to the angular potential associated with the hydrogen bond. The complexes studied are composed of phenol-OD (hydroxyl hydrogen replaced by deuterium) and various pi-base solvents with different electron donating or withdrawing substituents (chlorobenzene, bromobenzene, benzene, toluene, p-xylene, mesitylene, 1-pentyne). The different substituents provide experimental control of the hydrogen bond strength. The inertial orientational relaxation of the complexes, measured at the center frequency of each line, is independent of the hydrogen bond strength, demonstrating the insensitivity of the OD inertial dynamics, and therefore the H-bond angular potential, to the hydrogen bond strength. OD stretch absorption bands are inhomogeneously broadened through interactions with the solvent. The hydrogen bonding complexes all have similar wavelength dependent inertial orientational relaxation across their inhomogeneously broadened OD stretch absorption lines. The wavelength dependence of the inertial reorientation across each line arises because of a correlation between local solvent structure and the angular potential. These two results imply that local solvent structure acts as the controlling influence in determining the extent of inertial orientational relaxation, and therefore the angular potential, and that. variation in the pi-hydrogen bond strength is of secondary importance.