Heavy oil is an abundant and important energy source that has the potential to meet the increasing energy demand of the world. However, its economic recovery is challenging and having a thorough molecular understanding of the heavy oil-rock interface will be advantageous. Here, we have used molecular dynamics simulations to probe the adhesion of heavy oil at commonly found carbonate (calcite), sandstone (silica), and clay (mica)-type rock reservoirs. The calcite surface has the highest adhesion energy with heavy oil followed by mica and silica surfaces. The dispersion interaction is dominant as compared to electrostatic interactions in calcite and silica systems. In contrast, an equal contribution of both interactions is found in the heavy oil-mica interface. The structure of heavy oil adjacent to the rock surface is characterized by a dense interfacial layer (IL) followed by a bulk zone. We found that the areal density of exposed atoms, roughness, and dipole moment of the rock surface has a significant effect on the structure of the IL and adhesion energy. A higher areal density of surface atoms coupled with low roughness leads to stronger adsorption of heavy oil molecules favored by dispersion interactions. The adsorption of aromatic carbons (from aromatic and resin fractions) of heavy oil is more than that of aliphatic carbons (from saturates) in the calcite and mica system, whereas a reverse trend is observed in the silica system. The charge distribution is non-uniform in the IL; however, the electrostatic interaction between the heavy oil and rock surface is dictated by the dipole moment of the rock surface.