The effects of Co2+ doping on the structural, magnetic and dielectric properties of the multiferroic frustrated antiferromagnet Mn3TeO6 have been investigated. Ceramic samples of the solid solution series Mn3-xCoxTeO6 were prepared by a solid-state reaction route. X-ray and neutron powder diffraction and electron microscopy techniques were combined with calorimetric, dielectric and magnetic measurements to investigate the dependence of the crystal structure and physical properties on temperature and composition. It is shown that the compounds with x <= 2.4 adopt the trigonal corundum-related structure of pure Mn3TeO6 (space group 18) in the temperature range 5-295 K and that the lattice parameters a and c and the unit-cell volume V decrease linearly with increasing Co2+ concentration. The low-temperature magnetic susceptibility and heat capacity data evidence the antiferromagnetic ordering of all samples. The Neel temperature linearly increases with Co2+ concentration x. Curie-Weiss fits of the high temperature susceptibility indicate that the magnetic frustration decreases with x. The derived magnetic structure of Mn3TeO6 can be described as an incommensurately modulated magnetic spin state with k = [0, 0, k(z)] and an elliptical spin-spiral order of spins within the chains of MnO6 octahedra. With increasing Co2+ concentration the propagation vector kz changes from 0.453 (x = 0) to 0.516 (x = 2.4). The magnetic anisotropy changes as well, leading to a reorientation of the spiral-basal plane. A possible coexistence of long-range order of electrical dipoles and magnetic moments in Mn3-xCoxTeO6 is discussed. (C) 2013 Elsevier Ltd. All rights reserved.