Gallium nitride (GaN) layers have been synthesized from mixtures of trimethylgallium, tertiarybutylhydrazine (TBHy), and hydrogen (H-2) at 958K using the metalorganic vapor-phase epitaxy technique. The role of H-2 in the reaction system is explored by investigating the film property changes at various H-2 concentrations. Epitaxial growth of GaN with carbon incorporation less than 3 mol% is accomplished at a low H-2 concentration of 20 mol%, while it turns gradually into polycrystalline growth with high amount of carbon incorporation when the H-2 concentration is increased up to 90 mol%. Density functional theory calculations are performed to examine the thermochemistry of the H-2 elimination reaction of gaseous TBHy on Ga-terminated GaN surface. The result suggests that at high H-2 concentrations, a C-N containing species derived from TBHy due to incomplete P-hydride elimination may be ascribable to the appearance of polycrystalline phase and high amount of carbon incorporation. (C) 2004 Elsevier B.V. All rights reserved.