This technical note addresses a class of discrete-time stochastic systems with infinite Markov jump parameter. First of all, spectral criterion and Lyapunov type criteria are presented for the exponential stability of considered systems. Further, under the condition of strong detectability, a Barbashin-Krasovskii stability theorem is shown in terms of a generalized Lyapunov equation with a positive semi-definite term. As an application of the proposed Lyapunov stability criterion, a sufficient condition of l(2) input-state stability is derived for the infinite Markov jump systems disturbed by finite-energy random disturbance.