We consider the role of nonlamellar-forming lipids in biological membranes by examining fluctuations, within the random phase approximation, of a model mixture of two lipids, one of which forms lamellar phases while the other forms inverted hexagonal phases. To determine the extent to which nonlamellar-forming lipids facilitate the formation of nonlamellar structures in lipid mixtures, we examine the fluctuation modes and various correlation functions in the lamellar phase of the mixture. To highlight the role fluctuations can play, we focus on the lamellar phase near its limit of stability. Our results indicate that in the initial stages of the transition, undulations appear in the lamellae occupied by the tails, and that the nonlamellar-forming lipid dominates these undulations. The lamellae occupied by the head groups pinch off to make the tubes of the hexagonal phase. Examination of different correlations and susceptibilities makes quantitative the dominant role of the nonlamellar-forming lipids.