This work develops asymptotically optimal controls for a class of discrete-time hybrid systems involving singularly perturbed Markov chains having weak and strong interactions. The state space of the underlying Markov chain is decomposed into a number of recurrent classes and a group of transient states. Using a hierarchical control approach, by aggregating the states in each recurrent class into a single state, a continuous-time quadratic limit control problem in which the resulting limit Markov chain has much smaller state space is derived. Using the optimal control for the limit problem, a control for the original problem is constructed, which is shown to be nearly optimal. Finally, a numerical example is given to demonstrate the effectiveness of the approximation scheme.