The authors consider infinite-horizon nonlinear optimal control problems with unbounded performance indexes using notions of overtaking optimality. In the framework of optimal control, new sufficient conditions of Caratheodory-Hamilton-Jacobi type for both weak overtaking and overtaking optimality are presented and compared with known results from the calculus of variations setting. Using local structural properties of a related control Hamiltonian, conclusions are drawn on aspects of existence of overtaking and weak overtaking optimal controls, stability of optimal trajectories, and approximation of weak overtaking optimal controls. Both similarities and differences among alternative approaches to overtaking, weak overtaking, and strong optimal control problems are discussed. The formulation and results are illustrated in the classical linear-quadratic tracking problem and in an optimization approach to gain scheduling.