The conventional hydraulic drive system for a heavy-load mobile manipulator is usually operated under single mode, such that both inlet/outlet and potential energy losses are large to lower the energy efficiency. In this paper, a novel electro-hydraulic drive system is presented to improve energy efficiency. Extended control degrees of freedom are obtained utilizing the independent metering valve and electronic controlled pump. Then, multiple operating modes are carried out pertaining to the cylinder, valve, and pump. To achieve both optimal energy efficiency and precise motion tracking, both multi-mode switching and multi-variable controller are designed to accommodate with time-varying and uncertain load characteristics. As a consequence, the inlet, outlet, and potential energy losses can be decreased simultaneously. The experimental validation is conducted by using a three joint manipulator in a 2 t excavator. A duty cycle of movement including all three actuators and covering full load quadrants is used to evaluate the efficiency improvement. Compared with the conventional load sensing system, the proposed multi-mode switching system using the pump pressure with valve meter-in control mode yields a 25.8% energy-saving ratio. Furthermore, the pump flow with valve mete-out control mode yields a 35.3% energy-saving ratio. Using this combined control mode, higher efficiency can be obtained due to the minimum inlet losses, but faster dynamic response together with higher overshoot will appear. It is proved that the energy efficiency is improved, while the motion tracking performance is not degraded by introducing the multi-mode switching.