In this note, we propose partially linear models with least squares support vector machines (LS-SVMs) for nonlinear ARX models. We illustrate how full black-box models can be improved when prior information about model structure is available. A real-life example, based on the Silverbox benchmark data, shows significant improvements in the generalization ability of the structured model with respect to the full black-box model, reflected also by a reduction in the effective number of parameters.