This paper discusses the linear asymptotic consensus problem for a network of dynamic agents whose communication network is modeled by a randomly switching graph. The switching is determined by a finite state Markov process, each topology corresponding to a state of the process. We address the cases where the dynamics of the agents is expressed both in continuous time and in discrete time. We show that, if the consensus matrices are doubly stochastic, average consensus is achieved in the mean square sense and the almost sure sense if and only if the graph resulting from the union of graphs corresponding to the states of the Markov process is strongly connected. The aim of this paper is to show how techniques from the theory of Markovian jump linear systems, in conjunction with results inspired by matrix and graph theory, can be used to prove convergence results for stochastic consensus problems.