It remains a great challenge for the simple and affordable production of membranes for oil/water separation. We prepare low-cost but highly efficient paper-based membranes for oil/water separation through hydrophobic modification to filter papers. The simple modification contains only two steps: a thin layer of aluminum oxide is first coated on the surface of the filter paper by atomic layer deposition, and silane molecules are subsequently coupled on the precoated aluminum oxide layer via their reaction with hydroxyl groups on the surface. Both the alumina layer and the silanization layer are very thin with a total thickness less than 10 nm. The modified filter paper is endowed with strong hydrophobicity and oleophilicity, therefore exhibits strongly retarded permeation to water and enhanced permeation to nonpolar oils. The modified filter paper is demonstrated to show excellent separation efficiencies greater than 90% in the separation of various types of oils and organic solvents from their mixtures with water. The paper-based membranes prepared in this work are distinguished among others for their low-cost substrates and simple modification route. This modification method is expected to be easily extended to hydrophobize a diversity of other substrates.