As environmentally-friendly materials, geopolymers have the potential to replace ordinary Portland cement (OPC) for the construction of railway sleepers and multi-flue chimneys, where the vibration control capabilities of the material must be considered. The critical damping value (xi) is the main parameter in relation to vibration reduction. In this study, the traditional logarithmic decrement technique was used to measure the xi of geopolymers. Geopolymers were prepared by activating fly ash using alkali solutions with different SiO2/Na2O ratios. The results show that the xi of the geopolymers is similar to that of the OPC counterpart. Finite element analysis (FEM) based on the Rayleigh damping model was conducted to replicate the test results, and scanning electron microscopy and mercury-intrusion porosimetry were used to study the microstructure of the geopolymers. A discussion of the possible damping mechanisms based on the microstructural investigation and the FEM analysis is presented.