The effect of the clay content and the method of its combination with amine-terminated butadiene-acrylonitrile (ATBN) on the structure and behavior of epoxy was studied. In the case of the simultaneous addition of both components, the increasing clay content had a very small effect on the size of the reaction-induced phase separation-formed particles at 5% rubber content due to predominant elimination of two major clay effects, i.e., the nucleation due to phase separation and the kinetics. As a result, both the time window between the onset of phase separation and vitrification and the viscosity at the cloud point did not change significantly. The minor change in the particle size/clay content dependences with different curing temperatures indicates that the balance between the two clay effects shifted. The corresponding study of the mechanical behavior indicated that the best balanced mechanical properties were obtained at certain clay/ATBN ratios, and thus, there was synergy between the components. Similar mechanical parameters were obtained for the application of both components in the form of ATBN/montmorillonite intercalate. (C) 2012 Wiley Periodicals, Inc. J Appl Polym Sci, 2012