In this paper we study the bulk stress of a suspension of rigid particles in viscoelastic fluids. We first apply the theoretical framework provided by Batchelor [J. Fluid Mech. 41 (1970) 545] to derive an analytical expression for the bulk stress of a suspension of rigid particles in a second-order fluid under the limit of dilute and creeping flow conditions. The application of the suspension balance model using this analytical expression leads to the prediction of the migration of particles towards the centerline of the channel in pressure-driven flows. This is in agreement with experimental observations. We next examine the effects of inertia (or flow Reynolds number) on the rheology of dilute suspensions in Oldroyd-B fluids by two-dimensional direct numerical simulations. Simulation results are verified by comparing them with the analytical expression in the creeping flow limit. It is seen that the particle contribution to the first normal stress difference is positive and increases with the elasticity of the fluid and the Reynolds number. The ratio of the first normal stress coefficient of the suspension and the suspending fluid decreases as the Reynolds number is increased. The effective viscosity of the suspension shows a shear-thinning behavior tin spite of a non-shear-thinning suspending fluid) which becomes more pronounced as the fluid elasticity increases.