We present a numerical approach to simulate the dynamics of viscous vesicles (their internal and external fluids have different viscosities). The flow is computed using the lattice Boltzmann method and the fluid-vesicle two-way coupling is achieved using the immersed boundary method. The viscosity contrast (defined as the ratio of the internal to the external viscosities) is included using a geometrical algorithm that detects if a fluid node is either located inside or outside a vesicle. Our two-dimensional simulations successfully reproduce the tank-treading and tumbling dynamical states known for a viscous vesicle when it is subjected to simple shear flow. A good qualitative agreement between our simulation results and literature data is obtained. Moreover, we quantitatively analyze how inertia influences the dynamics of a vesicle and as an outlook we present an application of our method to the flow of multiple viscous vesicles in a microfluidic constriction.