The vapor-phase elemental mercury removal efficiency of the heavy-oil-fired fly ash (HOFA) discharged from heavy-oil-fired power plants was first tested to evaluate its suitability as a base material for the development of a low-cost novel sorbent to capture vapor-phase mercury in coal combustion flue gases. Raw, CO2-activated, and sulfur-impregnated HOFAs were prepared and tested. The morphology, specific surface area, particle size, and chemical composition were analyzed for the tested samples. A bench-scale fixed-bed reactor system was used to determine the mercury removal efficiencies of the HOFAs and commercially available activated carbons for comparison. The CO2-activated HOFA showed slightly higher mercury removal efficiency than the raw HOFA, resulting from the increase of active sorption sites by the enlarged surface area. The mercury removal efficiencies of the HOFAs modified by the sulfur impregnation process significantly increased with increasing sulfur content and were comparable to those of the commercially available sulfur-impregnated activated carbons, despite their much smaller surface area. These results suggest that the sulfur sites formed on the surface of the HOFAs during the impregnation process are highly active in capturing vapor-phase elemental mercury.