In this work, we have studied the magnetorheological (MR) fluid rheology in the magnetic field parallel to the fluid vorticity. Experimentally, the MR fluid flow was realized in the Couette coaxial cylinder geometry with the magnetic field parallel to the symmetry axis. The rheological measurements were compared to those obtained in the cone-plate geometry with the magnetic field perpendicular to the lower rheometer plate. Experiments revealed a quasi-Bingham behavior in both geometries with the stress level being just a few dozens of percent smaller in the Couette cylindrical geometry at the same internal magnetic field. The unexpectedly high MR response in the magnetic field parallel to the fluid vorticity is explained by stochastic fluctuations of positions and orientations of the particle aggregates. These fluctuations are induced by magnetic interactions between them. Once misaligned from the vorticity direction, the aggregates generate a high stress independent of the shear rate, and thus assimilated to the suspension apparent (dynamic) yield stress. Quantitatively, the fluctuations of the aggregate orientation are modeled as a rotary diffusion process with a diffusion constant proportional to the mean square interaction torque. The model gives a satisfactory agreement with the experimental field dependency of the apparent yield stress and confirms the nearly quadratic concentration dependency sigma(Y) proportional to Phi(2.2), revealed in experiments. The practical interest of this study lies in the development of MR smart devices with the magnetic field nonperpendicular to the channel walls. (C) 2014 The Society of Rheology.