In this paper we discuss the control of a stochastic process for which the driving noise is provided by a martingale associated with a Markov chain. A duality between stochastic differential equations and backward stochastic differential equations is used, as well as a duality between stochastic differential delay equations and anticipated backward stochastic differential equations.The proof involves some delicate analysis to establish comparison results.