Fatigue cracking is one of the major distresses of hot mix asphalt (HMA) at moderate temperatures. Two of the main properties of asphalt mixtures affecting fatigue are the cohesive bond energy of asphalt binder and the adhesive bond energy between asphalt binder and aggregate. These two parameters were calculated using surface free energy (SFE) theory. Furthermore, the fatigue life of asphalt mixtures was measured by indirect tensile fatigue test. The results showed that asphalt mixtures with limestone aggregates, with the most specific surface area, and magnitude of adhesion had the highest fatigue life. Moreover, asphalt mixtures made with asphalt binder having the highest penetration had greater fatigue life than the other mixtures. The enhanced fatigue life was attributed to the greater cohesion energy and higher resistance to fatigue cracking in asphalt film. Also, these mixtures had the highest adhesion energy on the contact surface between asphalt binders and aggregates, which increased the energy required to separate the asphalt binder from the aggregate surface and the occurrence of adhesion rupture distress.