Motivated by the recent advances in the field of quantum computing, quantum systems are modeled and analyzed as networks of decentralized quantum nodes which employ distributed quantum consensus algorithms for coordination. In the literature, both continuous and discrete-time models of the algorithm have been proposed. This paper aims at optimizing the convergence rate of the discrete-time quantum consensus algorithm over a quantum network with N qudits. The induced graphs are categorized in terms of the partitions of integer N by arranging them as the Schreier graphs. It is shown that the original optimization problem reduces to optimizing the Second Largest Eigenvalue Modulus of the weight matrix. Exploiting the Specht module representation of partitions of N, the Aldous' conjecture is generalized to all partitions of integer N implying that the spectral gap of all resultant induced graphs is the same. The spectral radius of the Laplacian is obtained from the feasible least dominant partition in the Hasse diagram of integer N. By analytically addressing the semidefinite programming formulation of the problem, closed form expressions for the optimal results are provided for a wide range of topologies. (C) 2016 Elsevier Ltd. All rights reserved.