We studied whisker formation on aluminum (Al) films on silicon substrates during heat treatments, and investigated the influence of the aluminum film properties and the conditions of the heat treatment on whisker nucleation and growth. The whiskers, which were confirmed to grow in single crystals due to thermal stress, tended to form on films contaminated with oxygen and nitrogen. This is because these impurities suppress the Al film grain growth during heat treatment, causing localization of the film stress relief which results in whisker formation. Oxidizing the atmosphere during heat treatment suppresses the nucleation and growth of whiskers. This indicates that breaking Al surface oxide and/or atom rearrangement through surface diffusion has a critical role in the growth process of whiskers.