Adhesive carrier particles have important technical and economic influence on circulating fluidized bed flue gas desulfurization (CFB-FGD) systems that use rapidly hydrated sorbent. Several cheap and easily obtained materials with rough surfaces and porous structures were used as adhesive carrier particles to prepare rapidly hydrated sorbent: the fly ash from the first electrical field in the electrostatic precipitator (ESP) of a circulating fluidized bed (CFB) boiler, the fly ash from a chain boiler, and river sand. Scanning electron microscopy (SEM), particle size distribution (PSD) analysis, surface and pore analysis, abrasion tests, and thermal gravimetric analysis (TGA) tests were used to examine the influence of the adhesive carrier particles' surface and pore characteristics on the adhesion, abrasion, and desulfurization characteristics of the rapidly hydrated sorbent. Experimental results showed that the CFBB ESP ash, chain boiler ash, and river sand all improved the abrasion and desulfurization characteristics of the rapidly hydrated sorbent compared to the coal fly ash. Specifically, the abrasion ratios of the experimental sorbents were all less than 25% compared to 45% for the coal fly ash sorbent, while the desulfurization abilities of the three experimental sorbents were more than 830 mg SO2/g lime compared to 725 mg SO2/g lime for the coal fly ash sorbent. The river sand sorbent had the greatest desulfurization ability and the lowest abrasion ratio, and it possesses the best industrial application potential of all the tested materials.