This note investigates the optimal linear quadratic control problem in the discrete-time framework, for stochastic systems affected by disturbances generated by a nonlinear stochastic exosystem. The application of the maximum principle to nonlinear optimal control problems does not admit, in general, implementable solutions. Therefore, it is worthwhile to look for finite-dimensional approximation schemes. The approach followed in this note is based on the v-degree Carleman approximation of a stochastic nonlinear system applied to the exosystem and provides a real-time algorithm to design an implementable control law. Simulations support theoretical results and show the improvements when the approximation index v is increased.