Inspired by the promising thermoelectric properties of chalcopyrite-like quaternary chalcogenides, here we describe the synthesis and characterization of the solid solution Cu2Zn1-xFexGeSe4. Upon substitution of Zn with the isoelectronic Fe, no charge carriers are introduced in these intrinsic semiconductors. However, a change in lattice parameters, expressed in an elongation of the c/a lattice parameter ratio with minimal change in unit cell volume, reveals the existence of a three-stage cation restructuring process of Cu, Zn, and Fe. The resulting local anisotropic structural disorder leads to phonon scattering not normally observed, resulting in an effective approach to reduce the lattice thermal conductivity in this class of materials.