We investigate microscopic properties of ultrathin oxynitride gate dielectrics using a combination of first principles electronic structure methods and the attenuated total reflection (ATR) infrared spectroscopy. We use a theoretical structural model based on the Si-SiO2 interface. The quantum molecular dynamics simulations suggest that N accumulates at the interface. We have generated samples with the nitrogen concentrations from 1.69x10(14) to 6.78x10(14) cm(-2). The analysis of nitrogen containing cells indicates a significant structural improvement of the oxide layer and the strain reduction at the interface. We have performed a calculation of the vibrational density of states. The experimental infrared ATR data is in qualitative agreement with the calculation. The valence band offset is estimated with two different theoretical methods. Calculations reveal a close agreement between a reference energy level based method and a direct estimate based on the density of states analysis. For the highest nitrogen concentration considered we find a 0.3 eV increase of the valence band offset due to nitrogen at the interface. The leakage current is studied using the Landauer theory to model the conductance through the gate dielectric.