The effective diffusion coefficients of hardened cement pastes can vary between a few orders of magnitude. The paper aims at building a homogenization model to estimate these macroscopic diffusivities and capture such strong variations. For this purpose, a three-scale description of the paste is proposed, relying mainly on the fact that the initial cement grains hydrate forming a complex microstructure with a multi-scale pore structure. In particular, porosity is found to be well connected at a fine scale. However, only a few homogenization schemes are shown to be adequate to account for such connectivity. Among them, the mixed composite spheres assemblage estimate (Stora, E., He, Q.-C., Bary, B.: J. Appl. Phys. 100(8), 084910, 2006a) seems to be the only one that always complies with rigorous bounds and is consequently employed to predict the effects of this fine porosity on the material effective diffusivities. The model proposed provides predictions in good agreement with experimental results and is consistent with the numerous measurements of critical pore diameters issued from mercury intrusion porosimetry tests. The evolution of the effective diffusivities of cement pastes subjected to leaching is also assessed by adopting a simplified scenario of the decalcification process.