The paper is concerned with numerical algorithms for the optimal control of diffusion-type processes when the noise variance also depends on the control. This problem is of increasing importance in applications, particularly in financial mathematics. We discuss the construction of numerical algorithms guaranteed to converge to the true minimum as the discretization level decreases and with acceptable numerical properties. The algorithms are based on the popular Markov chain approximation method. The basic criterion the algorithms must satisfy is a weak "local consistency" condition, which is essential for convergence to the true optimal cost function. This condition is often hard to satisfy by simple algorithms (with essentially only local transitions) when the variance is also controlled, Numerical "noise" can be introduced by the more convenient approximations. This question of "numerical noise" (also called "numerical viscosity") is dealt with in detail, and methods for eliminating or greatly reducing it are discussed.