Two-dimensional materials, such as graphene and MoS2, are films of a few atomic layers in thickness with strong in-plane bonds and weak interactions between the layers. The in-plane elasticity has been widely studied in bending experiments where a suspended film is deformed substantially; however, little is known about the films' elastic modulus perpendicular to the planes, as the measurement of the out-of-plane elasticity of supported 2D films requires indentation depths smaller than the films' interlayer distance. Here, we report on sub-angstrom-resolution indentation measurements of the perpendicular-to-the-plane elasticity of 2D materials. Our indentation data, combined with semi-analytical models and density functional theory, are then used to study the perpendicular elasticity of few-layer-thick graphene and graphene oxide films. We find that the perpendicular Young's modulus of graphene oxide films reaches a maximum when one complete water layer is intercalated between the graphitic planes. This non-destructive methodology can map interlayer coupling and intercalation in 2D films.