A novel approach for approximating the nonlinear optimal feedback control of a system with a terminal cost is proposed. To lessen the difficulty due to nonlinearity, we try to treat the system in a framework of linear theories. For this, we assume a quantum mechanical linear wave associated with the system. Since the control system is constrained by state equations, we handle the system according to quantum mechanics of constrained dynamics. A Hamiltonian is represented as a linear operator acting on a function that describes behavior of waves. Subsequently, nonlinear feedback is calculated without any time integration in the backward direction. Using eigenvalues and eigenfunctions of the linear Hamiltonian operator, an optimal feedback law is given as a combination of analytic functions of time and state variables. We take as an example a system described by two scalar variables for state and control input. Simulation studies on the system by the eigenvalue analysis show that the proposed method reduces calculation time to nearly a tenth that of a numerical calculation of a Hamilton-Jacobi equation by a finite difference method. (c) 2005 Elsevier Ltd. All rights reserved.