This study attempts to simulate the flow and heat transfer in shell-and-tube heat exchanger with helical baffles. The effects of geometrical parameters (i.e., helix angle and baffle pitch) on the convective heat transfer and pressure drop have been investigated. The results indicate that application of helical baffles results in almost uniform distribution of temperature and pressure. Furthermore, both convective heat transfer and pressure drop increase by decreasing the helix angle and baffle pitch. In order to find optimum helix angle and baffles overlapping, two-objective optimization was performed, in which neural network was applied to obtain the objective functions. By performing optimization, 38 optimal cases were obtained. After finding the optimal cases, the compromise programming technique was utilized in order to facilitate the decision making process for selection among the optimal points. When importance of heat transfer and pressure drop is considered to be similar, using helical baffles with low overlapping (i.e., high baffle pitch) and low helix angle is suggested, while in case which heat transfer is regarded to be more important than pressure drop, application of helical baffles with high overlapping and low helix angle is recommended. (C) 2015 The Institution of Chemical Engineers. Published by Elsevier B.V. All rights reserved.