This paper studies the quadratic regulation problem for discrete-time switched linear systems (DSLQR problem) on an infinite time horizon. A general relaxation framework is developed to simplify the computation of the value iterations. Based on this framework, an efficient algorithm is developed to solve the infinite-horizon DSLQR problem with guaranteed closed-loop stability and suboptimal performance. Due to these guarantees, the proposed algorithm can be used as a general controller synthesis tool for switched linear systems.