Upgrading the water-soluble fraction, which was obtained by water extraction of rice husk fast pyrolysis oil, was investigated with simultaneous esterification and acetalation with online solvent extraction (SEAWOSE) in butanol. It was found that, in comparison to direct esterification and acetalation without extraction, almost all of the acids and aldehydes in the water-soluble fraction can be converted to the corresponding esters, hemiacetals, and acetals by SEAWOSE. With the aid of online extraction, the saccharides could be transformed into the upgraded oil gradually via first hydrolysis into aldehyde derivatives and then acetalation. As a result, the char formation was significantly suppressed. The effect of oxidation and reduction of the water-soluble fraction as pretreatment before SEAWOSE was also investigated. By hydrogen peroxide oxidation, the aldehydes could be first converted into acids and subsequently esterified to esters, consequently without char formation. The upgraded oil was with high oil quality, less than 3% in moisture, higher than 30 MJ/kg in high heating value, and less than 2 mg of KOH/g in acidity.