We use a modified conducting atomic force microscope to simultaneously probe the conductance of a single-molecule junction and the force required to rupture the junction formed by alkanes terminated with four different chemical link groups which vary in binding strength and mechanism to the gold electrodes. Molecular junctions with amine, methylsulfide, and diphenylphosphine terminated molecules show clear conductance signatures and rupture at a force that is significantly smaller than the measured 1.4 nN force required to rupture the single-atomic gold contact. In contrast, measurements with a thiol terminated alkane which can bind covalently to the gold electrode show conductance and force features unlike those of the other molecules studied. Specifically, the strong Au-S bond can cause structural rearrangements in the electrodes, which are accompanied by substantial conductance changes. Despite the strong Au-S bond and the evidence for disruption of the Au structure, the experiments show that on average these junctions also rupture at a smaller force than that measured for pristine single-atom gold contacts.