A newly proposed concept of entransy as the heat transfer potential could be useful in analyzing and optimizing the heat-work conversion systems. In this study, the work production and heat transfer processes in endoreversible Carnot cycle system are investigated by systematically establishing the balance equation of entransy along with the formulations of entransy loss, work entransy, and entransy dissipations at various positions in the system and environment, which are consistent with energy and exergy balances. Both cases of dumping and non-dumping of used source fluid are considered to analyze the optimal condition of net work production and maximization of total entransy loss. In addition, the several forms of system efficiency are introduced based on entransy input, entransy loss, and work entransy in order to appreciate the system performance. As an illustration of optimization using the concept of entransy, the effects of system parameters on source fluids and heat exchanger are investigated for optimal condition of producing maximum work. In particular, the change of thermal capacity in source fluid leads to different trends of entransy efficiencies when compared to conventional efficiencies of energy and exergy, as it shows another way to assess the effective use of heat source in power generation systems. (C) 2015 Elsevier Ltd. All rights reserved.