Equilibrium solubility of m-nitroaniline and p-nitroaniline in supercritical carbon dioxide (SCCO2) is essential to design the process of SCCO2 extraction and to investigate the effect of each solute on the solubility in SCCO2 ternary system. However, the solubility data is not reported so far. We performed the solubility measurements at the temperatures of 308-328 K and in the pressure range of 11.0-21.0 MPa. The experimental results showed the solubility of m-nitroaniline and p-nitroaniline was enhanced in m-nitroaniline +p-nitroaniline +SCCO2 ternary system. The improvement factor (i), separation factor (mu) and separation efficiency (HE) in the ternary system were defined and calculated, and the best separation result could be obtained at 21.0 MPa and 328 K using SCCO2 extraction, where the separation efficiency was up to 90.9%. Based on the chemical association theory, a new model was developed to calculate the solubility of mixed solutes in SCCO2. The correlation result of the new model was tested by about 500 solubility data from 15 kinds of two solutes mixtures in SCCO2. The correlated result showed that the new model could achieve much better AARD (%) than those of frequently used Sovova and Sovova-T models. (C) 2014 The Institution of Chemical Engineers. Published by Elsevier B.V. All rights reserved.