A standard assumption in traditional (deterministic and stochastic) optimal (minimizing) linear quadratic regulator (LQR) theory is that the control weighting matrix in the cost functional is strictly positive definite. In the deterministic case, this assumption is in fact necessary for the problem to be well-posed because positive definiteness is required to make it a convex optimization problem, However, it has recently been shown that in the stochastic case, when the diffusion term is dependent on the control, the control weighting matrix may have negative eigenvalues but the problem remains well-posed! In this paper, the completely observed stochastic LQR problem with integral quadratic constraints is studied. Sufficient conditions for the well-posedness of this problem are given. Indeed, we show that in certain cases, these conditions may be satisfied, even when the control weighting matrices in the cost and all of the constraint functionals have negative eigenvalues. It is revealed that the seemingly nonconvex problem (with indefinite control weights) can actually be a convex one by virtue of the uncertainty in the system. Finally, when these conditions are satisfied, the optimal control is explicitly derived using results from duality theory.