We compare the dynamics of Fewest Switches Surface Hopping (FSSH) in different parameter regimes of the spin-boson model. We show that for exceptional regions of the spin-boson parameter space, FSSH dynamics are in fact time-reversible. In these rare instances, FSSH does recover the correct Marcus rate scaling (as a function of diabatic coupling) without the addition of decoherence. In regions where dynamics are irreversible, however, FSSH does not recover the correct Marcus rate scaling. Finally, by comparing the friction dependence of rates predicted by various decoherence schemes to an analytic result by Zusman, we provide yet more evidence that the method of introducing decoherence has a qualitative effect on the accuracy of results and this effect must be treated carefully.