How much translational energy atoms and molecules lose in collisions at surfaces determines whether they adsorb or scatter. The fact that hydrogen (H) atoms stick to metal surfaces poses a basic question. Momentum and energy conservation demands that the light H atom cannot efficiently transfer its energy to the heavier atoms of the solid in a binary collision. How then do H atoms efficiently stick to metal surfaces? We show through experiments that H-atom collisions at an insulating surface (an adsorbed xenon layer on a gold single-crystal surface) are indeed nearly elastic, following the predictions of energy and momentum conservation. In contrast, H-atom collisions with the bare gold surface exhibit a large loss of translational energy that can be reproduced by an atomic-level simulation describing electron-hole pair excitation.