Hydrocracking is used in the petroleum industry to convert low-quality feedstocks into highly-valued transportation fuels. This process is the best source of low-sulfur and low-aromatics diesel fuel as well as high-smoke point jet fuel. Many approaches have been proposed for solving optimization of hydrocracking units in the last decades, but they usually neglect the reaction in hydrotreater where hydrocarbon cracking often occurs, thus leading to suboptimal solutions in industrial problems. Unlike existing literature, this paper considers the hydrocarbon cracking reactions in hydrotreater and hydrocracker simultaneously. The models are based on energy balance, mass balance and a discrete lumped model approaches for kinetic modeling. Before optimization, the properties of feedstock are predicted with ASPEN PLUS by using laboratory data from the refinery, and then the model parameters are estimated with genetic algorithm (GA) based on industrial data and validated by comparing the simulating results with industrial data. To improve the yield of the lighter products, the operation conditions are optimized by GA and Sequential Quadratic Programming (SQP). The yields of the diesel or kerosene increase with the proposed approach. (C) 2011 Elsevier Ltd. All rights reserved.