The relativistic Dirac Hamiltonian that describes the motion of electrons in a magnetic field contains only paramagnetic terms (i.e., terms linear in the vector potential A) while the corresponding nonrelativistic Schrodinger Hamiltonian also contains diamagnetic terms (i.e., those from an A(2) operator). We demonstrate that all diamagnetic terms relativistically arise from second-order perturbation theory and that they correspond to a "redressing'' of the electrons by the magnetic field. If the nonrelativistic limit is taken with a fixed no-pair Hamiltonian (no redressing), the diamagnetic term is missing. The Schrodinger equation is normally obtained by taking the nonrelativistic limit of the Dirac one-electron equation, we show why nonrelativistic use of the A(2) operator is also correct in the many-electron case. In nonrelativistic approaches, diamagnetic terms are usually considered in first-order perturbation theory because they can be evaluated as an expectation value over the ground state wave function. The possibility of also using an expectation value expression, instead of a second-order expression, in the relativistic case is investigated. We also introduce and discuss the concept of "magnetically balanced'' basis sets in relativistic calculations.