The anomalous behavior of the glass transition temperature (T-g) in low silica calcium aluminosilicate glasses has been related to the structural modifications observed by neutron and X-ray diffraction. The diffraction data indicate that Al and Si are in tetrahedral sites and that Ca atoms are in distorted octahedral sites. By subtracting the correlation functions for glasses at constant SiO2 or constant Al2O3 content, we have shown that Si and Al atoms are introduced in a different way within the glass structure. Si is present in various Q(n) sites, while Al resides in Q(3) and Q(4) sites for glasses with high CaO content and enters fully polymerized Q(4) sites with increasing SiO2 or Al2O3 content. The higher proportion of Al in Q(3) positions at high CaO content yields a depolymerization of the network. The lower connectivity will contribute to a decrease of the viscosity, which may be at the origin of the decrease of T-g for glasses at low silica content.