The choice of a parametric model structure in empirical and semi-empirical non-linear modeling is usually viewed as an important and critical step. However, it is known that by augmenting the least-squares identification criterion with a term that imposes a penalty on the non-smoothness of the model, an optimal non-parametric model can be found explicitly. The optimal non-parametric model will depend on the particular form of the penalty, which can be looked upon as a priori knowledge, or the desired properties of the model. In this paper these results are extended in several directions : (i) we show how useful types of prior knowledge other than smoothness can be included as a term in the criterion or as a constraint, and how this influences the optimal model; (ii) dynamic models and a general prediction error penalty are considered; (iii) we present a practical numerical procedure for the identification of a close to optimal semi-parametric model. The numerical approach is motivated by the difficulty of deriving the optimal non-parametric model if there are complicated constraints or penalty terms in the criterion; and finally (iv) we discuss determination of the appropriate model complexity through selecting weights on the different penalties on the basis of empirical data. Since the optimal non-parametric model is identified using an augmented optimization criterion, and all prior knowledge and desired model properties may be specified through the augmented optimization criterion, it is the choice of this criterion that is critical in this approach. This elevates the empirical and semi-empirical modeling problems to a more transparent and engineering-friendly level than is achieved by directly specifying a parametric model structure. Hence, our results shed some more light on modern non-linear empirical modeling approaches like radial basis-functions, splines and neural networks. In addition to providing a fundamental insight into the role of different kinds of prior knowledge, this high-level formulation is also attractive from a practical point of view, as it lies closer to engineering thinking, and less guesswork is required. The flexibility and power of this approach are illustrated with a semi-realistic simulation example.