Polymer-metal hybrids are interesting materials in general and serve as precursors for a wide range of possible applications. Herein, we present the controlled radical polymerization of 2-acrylamidoglycolic acid (AGA), a polyelectrolyte capable of chelating various metals, using reversible addition-fragmentation chain transfer (RAFT) polymerization. Polymerization kinetics and behavior in aqueous solution were investigated for PAGA homopolymers, and we further show that double hydrophilic poly(N-acryloylmorpholine)-block-PAGA (PNAM-b-PAGA) or poly(ethylene oxide)-block-PAGA (PEO-b-PAGA) block copolymers with varying weight fractions of PAGA can be prepared. Upon addition of different metal cations, these block copolymers form wormlike and spherical hybrid assemblies in aqueous media. The aggregates react reversibly to changes in pH, and their size and shape are influenced by the PAGA/metal ratio, the overall PAGA weight fraction, the type of metal cation, and the block copolymer concentration.