Transition metal catalysts play a crucial role in many industrial applications, including the manufacture of lubricants, smoke suppressants, corrosion inhibitors and pigments. The development of novel catalysts is commonly performed using a trial-and-error approach which is costly and time-consuming. The application of computer-aided molecular design (CAMD) to this problem has the potential to greatly decrease the time and effort required to improve current catalytic materials in terms of their efficacy and biological effects. This work applies an optimization approach to redesign environmentally-benign homogeneous catalysts, specifically those which contain transition metal centers, to improve certain physical properties. Two main tasks must be achieved in order to perform the molecular design of a novel catalyst: biological and chemical properties must be estimated directly from the molecular structure, and the resulting optimization problem must be solved in a reasonable time. In this work, connectivity indices are used for the first time to predict the physical properties of a homogeneous catalyst. The existence of multiple oxidation states for transition metals requires a reformulation of the original equations for these indices. Once connectivity index descriptors have been defined for transition metal catalysts, structure-property correlations are then developed based on regression analysis using literature data for various properties of interest, including toxicity and electronegativity. These structure-property correlations are then used within an optimization framework to design novel homogeneous catalyst structures for use in a given application. The use of connectivity indices which define the topology of the molecule within the formulation guarantees that a complete molecular structure is obtained when the global optimum is found. In this work, second-order connectivity indices are used to obtain more information about steric features of the catalyst molecules, and non-linear correlations are employed to improve the accuracy of the property prediction equations. The structure-property correlations are then combined with linear structural feasibility constraints to form a mixed-integer non-linear program (MINLP), which when solved to optimality results in a catalyst molecule which most closely matches given property targets. To solve the resulting optimization problem, two methods are applied: Tabu search (a stochastic method), and outer approximation, a deterministic approach. For the outer approximation solution, a data structure is used which permits all equations except for the property prediction expressions to be written in linear forms. The computational efficiency of Tabu search is not strongly dependent on the existence of non-linear constraints, so for solution using this method, a non-linear form for the second-order connectivity index was chosen, which decreases the number of binary variables required. The solution methods are compared using three examples involving the design of environmentally-benign homogeneous catalysts containing molybdenum centers. Results show the efficacy of the formulation, and provide evidence that the Tabu search algorithm is more suitable for this type of molecular design algorithm than the commercially available deterministic approach. (C) 2004 Elsevier Ltd. All rights reserved.