Accurately predicting the power load in certain areas is of great importance for grid management and power dispatching. A great deal of research has been conducted within the smart grid system community in developing an assortment of different algorithms that seek to increase the accuracy of these predictions. However, these predictions suffer from various sources of error, such as the variations in weather conditions, calendar effects, economic indicators, and many other sources, which are caused by the inherent stochastic and nonlinear characteristics of power demand. In order to quantify the uncertainty in load forecasting effectively, this paper proposes a comprehensive probability density forecasting method employing Gaussian process quantile regression (GPQR). GPQR is a type of Bayesian non-parametric method which can handle the uncertainties in power load data in a principled manner. Consequently, the probabilistic distribution of power load data can be statistically formulated. The effectiveness of the proposed method for short-term load forecasting has been assessed adopting the real dataset provided by American PJM electric power company. Numerical results demonstrate that the uncertainties in power load data can be effectively acquired based on the proposed method. Meanwhile, the competitive predictive performance could be yielded with respect to the conventional adopted methods.