This paper describes the buildup of hydrogen-bonding-directed poly(4-vinylpyridine)/poly(4-vinylphenol) (PVPy/PVPh) multilayer film that was fabricated by layer-by-layer (LbL) assembly of PVPy and PVPh from an ethanol solution. UV-visible spectroscopy and Fourier transform infrared (FT-IR) spectroscopy revealed a uniform deposition process. The interaction between PVPy and PVPh was identified as hydrogen bonding through FT-IR spectroscopy and temperature-dependent IR spectral changes of the hydrogen-bonded multilayer. Notably, we discussed the effect of solvent conditions on the growth of PVPy/PVPh multilayer films monitored by UV-visible spectroscopy. It was found that increasing the ratio of N,N-dimethylformamide (DMF) in the mixed ethanol/DMF solvents resulted in a marked decrease of the amount of polymers adsorbed, which was attributed to the increased polarity of the adsorption solutions. Furthermore, the solvent stability of PVPy/PVPh multilayer film in mixed ethanol/DMF solvents with different DMF ratios was also investigated. As a result, a new method for tuning the structure of hydrogen-bonding-directed multilayer film was developed.