Building energy consumption prediction plays an irreplaceable role in energy planning, management, and conservation. Data-driven approaches, such as artificial neural networks, support vector regression, gradient boosting regression and extreme learning machine are the most advanced methods for building energy prediction. However, owing to the high nonlinearity between inputs and outputs of building energy consumption prediction models, the aforementioned approaches require improvement with regard to the prediction accuracy, robustness, and generalization ability. To counter these shortcomings, a novel vector field-based support vector regression method is proposed in this paper. Through multi-distortions in the sample data space or high-dimensional feature space mapped by a vector field, the optimal feature space is found, in which the high non linearity between inputs and outputs is approximated by linearity. Hence, the proposed method ensures a high accuracy, a generalization ability, and robustness for building energy consumption prediction. A large office building in a coastal town of China is used for a case study, and its summer hourly cooling load data are used as energy consumption data. The results indicate that the proposed method achieves better performance than commonly used methods with regard to the accuracy, robustness, and generalization ability.