In the standard prediction error framework of system identification, statistical properties of estimated models are typically derived under the assumption that the true system is in the model class. The standard model structure validation test for plant models is the sample cross-correlation test between the residuals of the model and the input. It turns out that the standard test itself is valid only under exactly those assumptions it is meant to verify, i.e. the system is in the model class. It is shown that for reliable results of the validation test a vector-valued test is required and that accurate noise modelling is indispensable for reliable model structure validation. This shows the limitation of separate validation of plant and noise model structures. Improvements of the test are presented, and it is motivated by the fact that reserving data only to be used for model validation is not efficient. (c) 2008 Elsevier Ltd. All rights reserved.