Interest in assemblies of colloidal particles(1-4) has long been motivated by their applications in photonics(5,6), electronics(7,8), sensors(8) and microlenses(9). Existing assembly schemes(10-15) can position colloids of one type relatively flexibly into a range of desired structures, but it remains challenging to produce multicomponent lattices, clusters with precisely controlled symmetries and three-dimensional assemblies(16). A few schemes can efficiently produce complex colloidal structures(2,17,18), but they require system-specific procedures. Here we show that magnetic field microgradients established in a paramagnetic fluid(19,20) can serve as 'virtual moulds' to act as templates for the assembly of large numbers (similar to 10(8)) of both non-magnetic and magnetic colloidal particles with micrometre precision and typical yields of 80 to 90 per cent. We illustrate the versatility of this approach by producing single-component and multicomponent colloidal arrays, complex three-dimensional structures and a variety of colloidal molecules from polymeric particles, silica particles and live bacteria and by showing that all of these structures can be made permanent. In addition, although our magnetic moulds currently resemble optical traps in that they are limited to the manipulation of micrometre-sized objects, they are massively parallel and can manipulate non-magnetic and magnetic objects simultaneously in two and three dimensions.