We investigate the drying process of three component and polydisperse colloidal suspensions using Brownian dynamics simulations. We have previously reported (Phys. Rev. Lett. 2016, 116, 118301) on the drying of binary mixtures. For binary mixtures, we found that a gradient of colloidal osmotic pressure develops during drying and that this leads to the final film being stratified with a layer of smaller particles on top of a layer of larger particles. Here, we find that stratification by size is very general and also occurs in ternary and polydisperse mixtures. We name the segregation effect colloidal diffusiophoresis. In particular, we show that by changing the composition of a ternary mixture, different stratification morphologies can be achieved and hence the film properties can be tuned. In polydisperse spheres, colloidal diffusiophoresis leads to enrichment in the large particles at the bottom part of the film, whereas the top part is enriched with smaller particles. This segregation means that in the final film, the particle size distribution depends on height. Thus, the properties of the film will then depend on height. We propose a model that predicts a power-law dependence of the phoretic velocity on particle size. Results from the model and simulation show a good agreement.