The unstable light source in the printing and dyeing environment would cause the change of fabric surface color and lead to serious color difference evaluation error. To solve this problem, a dyed fabric illumination estimation with regularized random vector functional link network (RRVFL) was proposed in this study. First, the Gray-Edge framework is used to extract the color features of the sample images collected from the actual scene, and the extracted color features and the illumination information of the sample images constitute a data set. Then, considering the ill-conditioned solution of the output weight of the traditional random vector functional link network (RVFL), regularization is proposed to solve this problem. Thus, we constitute a highly robust RRVFL dyed fabric illumination model. By analyzing the parameters that affect the precision of RRVFL model, the optimal parameters of RRVFL can be selected. Finally, the traditional RVFL, extreme learning machine (ELM), back-propagation (BP), regularized extreme learning machine (RELM), support vector regression (SVR), and RRVFL estimation algorithm proposed in this study were compared and analyzed using the measurement standards of angle error, colorimetric error, and T-test, respectively. The experimental results show that RRVFL has the best predictive results and the most stable performance compared with the traditional algorithm. RRVFL is 0.00036, 2.8050, 3.3518, 4.1669, and 2.9289 less than RVFL, ELM, BP, RELM, and SVR, respectively, in terms of average angle error. RRVFL is 0.0131, 0.0763, 0.0232, 0.0241, and 0.0221 lower than RVFL, ELM, BP, RELM, and SVR, respectively, in the average colorimetric error.