SrFeO2.5 and SrCoO2.5 are able to intercalate oxygen in a reversible topotactic redox reaction already at room temperature to form the cubic perovskites Sr(Fe,Co)O-3, while CaFeO2.5 can only be oxidized under extreme conditions. To explain this significant difference in low temperature oxygen mobility, we investigated the homologous SrFeO2.5 and CaFeO2.5 by temperature dependent oxygen isotope exchange as well as by inelastic neutron scattering (INS) studies, combined with ab initio (DFT) molecular dynamical calculations. From O-18/O-16 isotope exchange experiments we proved free oxygen mobility to be realized in SrFeOx already below 600 K. We have also evidence that low temperature oxygen mobility relies on the existence of specific, low energy lattice modes, which trigger and amplify oxygen mobility in solids. We interpret the INS data together with the DFT-based molecular dynamical simulation results on SrFeO2.5 and CaFeO2.5 in terms of an enhanced, phonon-assisted, low temperature oxygen diffusion for SrFeO3-x as a result of the strongly reduced Fe-O-Fe bond strength of the apical oxygen atoms in the FeO6 octahedra along the stacking axis. This dynamically triggered phenomenon leads to an easy migration of the oxide ions into the open vacancy channels and vice versa. The decisive impact of lattice dynamics, giving rise to structural instabilities in oxygen deficient perovskites, especially with brown millerite-type structure, is demonstrated, opening new concepts for the design and tailoring of low temperature oxygen ion conductors.