The effect of the modification of the molecular structure on the stiffness of the polymeric backbones in relation to the diffusion coefficients of typical rubbery and glassy silane and siloxane polymers at different temperatures was investigated. The inflexibility in the polymeric chains as deduced by higher values for the persistence length was shown to correspond consistently to lower values for the self-diffusion coefficients. Increasing the temperature resulted in decreasing the persistence length of the silane polymers and increasing it in the case of the siloxane polymers. This was found to be due to the fact that the bond angle about the oxygen atoms is approximately 144 degrees and rotations about the oxygen atoms for the trans isomeric states will bring the side groups on the neighboring Si-atoms in close proximity to each other thus increasing the torsional energies of the trans isomeric states considerably. The obtained simulation results showed an excellent agreement to those determined experimentally.