The linear instability characteristics of a core-annular pipe flow of a Newtonian fluid and a Bingham fluid separated by a mixed region is investigated. The main objective of this study is to investigate the effect of the yield stress of the annular fluid on the stability behaviour. The present linear stability analysis reveals that the corkscrew mode is the most dangerous mode for the parameters considered. The Bingham number seems to destabilise the flow via modifying the velocity profile. The mechanism of instability is found to be inviscid in nature. An energy budget analysis shows that the energy due to the gradient of viscosity perturbation in the radial direction, and the transfer of energy from the basic flow to the perturbation via the "Reynolds stress" term are responsible for the instability observed. The positive contributions of these terms towards instability increases with increasing the yield stress. The critical Reynolds number also decreases with increasing the Bingham number for the corkscrew mode, which indicates the destabilising influence of the yield stress.