The primary objective of this study was to improve montmorillonite clay-platelet separation in vinyl ester resin matrix by organically modifying the nanoclay platelet with a partially reactive onium salt. The reactive onium salt (omega-undecylenyl amine hydrochloride) was synthesized from commercial (omega-undecylenyl alcohol through a series of synthetic conversions. Nonreactive onium salt (undecyl amine hydrochloride) was made from commercial undecyl amine. These salts were characterized with H-1 and C-13 NMR and Fourier transform infrared techniques. The relative amounts of exfoliated, intercalated, and as-treated clay and the size of the clay particle aggregates depended significantly on the composition of clay and the processing conditions. When the clay was ion-exchanged with a mixture of reactive and nonreactive onium salts, a partially exfoliated vinyl ester resin polymer nanocomposite was formulated. The addition of a comonorner styrene and high-intensity ultrasonic mixing produced vinyl ester nanocomposite with the highest degree of clay-platelet exfoliation. (C) 2004 Wiley Periodicals, Inc.