To develop an efficient material for the cathode of the lithium oxygen (Li-O-2) secondary battery, the oxygen reduction and evolution reactions (ORR and OER) on a well-defined graphene electrode have been investigated in a typical organic solvent, dimethyl sulfoxide (DMSO). The adsorption and desorption behaviors of the solvents on the graphene electrode surface were evaluated by an intrinsically surface-selective vibrational spectroscopy of sum frequency generation (SFG) during the ORR and OER. After the initial ORR depositing lithium peroxide (Li2O2) on the graphene electrode surface in a LiCIO4/DMSO solution, the SFG spectroscopy revealed that the subsequent OER oxidizing the Li2O2 preferentially proceeds at the interface between the Li2O2 and graphene rather than that between the Li2O2 and bulk solution. Therefore, the OER tends to reduce the electric conductivity between the Li2O2 and graphene by decreasing their contact area before a large part of the deposited Li2O2 was oxidized, which elucidates the origin of the high overpotential for the OER.