Planetary gearboxes play an important role in wind turbine drive trains. Fault diagnosis of planetary gearboxes is a key topic for maintenance of wind turbines. Considering the spectral complexity of planetary gearbox vibration signals as well as their amplitude modulation and frequency modulation (AMFM) nature, we propose a simple yet effective method to diagnose planetary gearbox faults based on amplitude and frequency demodulations. We use the energy separation algorithm to estimate the amplitude envelope and instantaneous frequency of modulated signals for further demodulation analysis, by exploiting the adaptability of Teager energy operator to instantaneous changes in signals and the fine time resolution. However, the energy separation algorithm requires signals to be mono-components. To satisfy this requirement, we decompose signals into intrinsic mode functions (IMFs) using the ensemble empirical mode decomposition (EEMD) method as it can decompose any signal into mono-components. We further propose a criterion to guide the selection of the most relevant IMF for demodulation analysis according to the wavelet-like filter nature of EEMD and the AMFM characteristics of the planetary gearbox vibration signals. By matching the dominant peaks in the Fourier spectra of the obtained amplitude envelope and instantaneous frequency with the theoretical characteristic frequency of each gear, we can diagnose planetary gearbox faults. The principle and effectiveness of the proposed method are illustrated by simulation and the experimental analysis of signals from a planetary gearbox of a wind turbine test rig. With the proposed method, both the wear and chipping faults can be detected and located for a sun gear of the planetary gearbox test rig. (C) 2012 Elsevier Ltd. All rights reserved.