Surface roughness is really important to the ultra-low loss of multilayer optical coatings. Reflectance and transmittance model for multilayer optical coatings with surface roughness has been established at normal incidence by A. V. Tikhonravov et al. in 2003. However, under the condition of oblique incidence, no further model which is based on the model of Tikhonravov has ever been reported. In this paper, an approximate reflectance and transmittance model is established for the thin film system with roughness at oblique incidence by analogy and usage of the concept of equivalent refractive index. The small-scale roughness model's effectiveness has been theoretically demonstrated according to the effective medium theory. However, the large-scale roughness model's effectiveness needs other methods to be proved, as the scattering effect of different scale roughness is different. This model might play a significant role to research the scattering loss of oblique incident thin film systems, such as mirrors in ring lasers, deep ultraviolet lasers and interferometers with ultra long cavity length. (C) 2015 Elsevier B.V. All rights reserved.