A new separation system representation uses stream separation vectors in the separation space. The characteristics of a separation sequence are clarified by its separation base vectors that form the sequence basis and those of a separation system by the geometrical properties of separation vectors. The optimal flowsheet of each sequence under separation vector formulation can be obtained as the solution of a linearly-constrained optimization problem. A set of simple rules determines the minimum separation toads of any sharp separation sequence by inspection. A modified cost measure,,which combines the separation cost with the savings due to stream bypass, can be used to select the optimal sequence without the overall cost analysis. The optimal separation sequence is obtained first by identifying the best sequence configuration according to modified cost measures and then by finding the actual costs of maximum-bypass and its neighboring sequence configurations. For the nonlinear objective function, the exact optimal flowsheet is determined by solving a linearly-constrained optimization problem. Since this procedure is a linearly-constrained optimization problem, the mathematical programming solution is not likely to lead to a local minimum.