Single-atom impurities and other atomic-scale defects can notably alter the local vibrational responses of solids and, ultimately, their macroscopic properties. Using high-resolution electron energy-loss spectroscopy in the electron microscope, we show that a single substitutional silicon impurity in graphene induces a characteristic, localized modification of the vibrational response. Extensive ab initio calculations reveal that the measured spectroscopic signature arises from defect-induced pseudo-localized phonon modes-that is, resonant states resulting from the hybridization of the defect modes and the bulk continuum-with energies that can be directly matched to the experiments. This finding realizes the promise of vibrational spectroscopy in the electron microscope with single-atom sensitivity and has broad implications across the fields of physics, chemistry, and materials science.