Dualities are mathematical mappings that reveal links between apparently unrelated systems in virtually every branch of physics(1-8). Systems mapped onto themselves by a duality transformation are called self-dual and exhibit remarkable properties, as exemplified by the scale invariance of an Ising magnet at the critical point. Here we show how dualities can enhance the symmetries of a dynamical matrix (or Hamiltonian), enabling the design of metamaterials with emergent properties that escape a standard group theory analysis. As an illustration, we consider twisted kagome lattices(9-15), reconfigurable mechanical structures that change shape by means of a collapse mechanism(9). We observe that pairs of distinct configurations along the mechanism exhibit the same vibrational spectrum and related elastic moduli. We show that these puzzling properties arise from a duality between pairs of configurations on either side of a mechanical critical point. The critical point corresponds to a self-dual structure with isotropic elasticity even in the absence of spatial symmetries and a twofold-degenerate spectrum over the entire Brillouin zone. The spectral degeneracy originates from a version of Kramers' theorem(16,17) in which fermionic time-reversal invariance is replaced by a hidden symmetry emerging at the self-dual point. The normal modes of the self-dual systems exhibit non-Abelian geometric phases(18,19) that affect the semiclassical propagation of wavepackets(20), leading to non-commuting mechanical responses. Our results hold promise for holonomic computation(21) and mechanical spintronics by allowing on-the-fly manipulation of synthetic spins carried by phonons.