This paper proposes a framework to automatically conduct cloud classification on all-sky images and perform short-term solar irradiance prediction according to the classification results. The all-sky images are divided into blocks to deal with the mixed cloud type conditions. Local texture patterns and statistical texture features are extracted from the image blocks for cloud classification. Different cloud types with various heights, thickness, and opacity have different impact on the variation of solar irradiance. Therefore, several regression models are trained to capture the characteristics of irradiance changes under different cloud types. The current classified cloud type is used to select a corresponding prediction model. Such design substantially increases the prediction accuracy. The experimental results verify the effectiveness of the proposed framework. Both the proposed cloud classification method and irradiance prediction mechanism outperform existing works. Adding local texture patterns in the feature vector enhance the classification performance. Compared with non-block based methods, the proposed block-based method could increase the classification rate by 5%-10%. Utilizing multiple prediction models according cloud types could lower both the mean absolute error and the root mean squared error on short-term irradiance prediction. (C) 2015 Elsevier Ltd. All rights reserved.