The defect structure of the oxide ion conducting solid electrolyte, Mg substituted Bi4V2O11-delta (BIMGVOX), was examined by high-resolution powder neutron diffraction. A detailed explanation of interpretation of the defect structure is presented. The general formula for the BIMGVOX solid solutions Bi2V1-xMg(x)O(5.5-3x/2) assumes complete oxidation of vanadium to V-V. Analysis of the neutron diffraction data reveals the defect structure and indicates that there is, in fact, partial reduction of vanadium to V-IV. The extent of reduction is dependent on thermal history, with high temperature quenched samples showing a greater degree of reduction than exponentially slow cooled samples. This is correlated with differences in electrical behaviour at low and high temperatures. Differences in ionic conductivity and activation energies between samples with different thermal histories are explained in terms of the balance between charge carrier concentration and the extent of defect trapping effects.