In rubber-toughened mixtures, the rubber is intended to dissipate the stress so that the blend shows ductility and plastic deformation. But poor interfacial adhesions, high interfacial tension, differences in polarity and structure between rubber and thermoplastic phases are the main reasons for incompatibility of these systems. The present work deals with the study of nanocomposites based on poly(styrene-acrylonitrile) (SAN) and ethylene-propylene-diene monomer (EPDM) blends with and without an organically modified montmorillonite (O-Mmt) filler as a compatibilizer. These blends are prepared by melt compounding using an internal mixer (Brabender). The dispersion of O-Mmt layers and the morphology of the nanocomposites, were determined by X-ray diffraction (XRD) and scanning electron microscopy (SEM) analysis. The effects of EPDM amount and O-Mmt on the rheological, mechanical properties and thermal stability of the nanocomposites were also investigated. All molten blends containing nanoclay particles exhibit higher viscosity than those of neat SAN/EPDM binary blends. XRD patterns and SEM examination revealed that silicate layers were exfoliated and well dispersed in the blend and the presence of nanoclay has reduced the domain size of EPDM phase. Thermogravimetric analysis (TGA) results showed that the incorporation both of EPDM and O-Mmt increased the thermal stability of SAN material. In binary blends, addition of EPDM has reduced the stiffness and tensile strength; however it improved the impact toughness and ductility of SAN. But, on adding nanoclay, results display the reverse trend: the stiffness of SAN/EPDM/O-Mmt nanocomposites increases in comparison to neat SAN and SAN/EPDM blends.