Distribution of surfactants in an aqueous/nonaqueous two phase system is of the great importance especially in enhanced oil recovery (EOR) processes. In this paper, the application of genetic algorithm to estimate the binary interaction parameters of the nonrandom two liquid (NRTL) activity coefficient model for a brine/oil/ionic surfactant system has been investigated. The presence of ionic surfactant in the system has been taken into account by employing the modified Debye-Huckel model. Moreover, the binary interaction parameters used in NRTL activity coefficient model were found to be interdependent and related to each other by a set of linear equations known as closure equations. These equations were considered as constraints to the optimization calculations. On the other hand, when experimental data were not available, the LLE data were estimated primarily through the Scatchard-Hildebrand activity coefficient model. These data were used to evaluate the GC-NRTL model parameters and calculate equilibrium mole fractions. The estimated binary interaction parameters using GA (genetic algorithm) showed a proper fitness with experimental values and the application of closure equations exhibited lower root-mean-square deviations. In addition, employing the Scatchard-Hildebrand model predictions for the modified GC-NRTL model calculations resulted in an acceptable accuracy. Accordingly, the presented model in this work can be utilized as a powerful method to study liquid-liquid equilibrium systems including surfactants.