By applying a combination of characterisation tools, changes in structural and superconducting properties with nominal Mg non-stoichiometry in MgxB2 are found. The non-stoichiometry produces enhanced in-field critical current densities (J(c)'s) and upper critical field / irreversibility field (H-c2/H-irr(T)) values. Upper critical fields of similar to 21 T (4.2 K) were obtained in nominal Mg-deficient samples compared to similar to 17 T (4.2 K) for near-stoichiometric samples.