Carbon dioxide (CO2) enhanced oil recovery is a green and promising way to produce oil and reduce the rapid growth of carbon dioxide released to atmosphere. A pore-scale understanding of CO2 displacement phenomena is important to enhance oil recovery in porous media. In this work, a direct numerical simulation method is employed to investigate the drainage process of CO2 in an oil-wet porous medium. The interface between the oil and CO2 is tracked by the phase field method. The capacity and accuracy of the model are validated using a classic benchmark: the process of a bubble rising. A series of numerical experiments were performed over a large range of values of the gravity number, capillary number, and viscosity ratio to investigate the flooding process of CO2 in a porous medium. The results show that the pressure in the main CO2 flow path decreases dramatically after CO2 breaks through the outlet. Oil begins to reflow into large pores that were previously occupied by CO2. This phenomenon has an important impact on the final saturation distribution of CO2. Increasing the viscous force is the dominant mechanism for improving oil recovery. Selecting an appropriate depth is the primary consideration for reaching the maximum recovery before CO2 is injected into the subsurface. Abnormal high-pressure formations represent a good choice for CO2 sequestration. Gravity fingers improve the sweep area of CO2 when the viscous force is small. The oil recovery increases with increasing contact angle. It is difficult to reach the final steady state of saturation because of the "snap-off and supplement" dynamic balance in porous media when both the injection velocity and the contact angle are small.