Using the conventional volumetric method in unsteady-state relative permeability measurements for unconventional gas reservoirs, such as coal and gas shale, is a significant challenge because the movable water volume in coal or shale is too small to be detected. Moreover, the dead volume in the measurement system adds extra inaccuracy to the displaced water determination. In this study, a low-field nuclear magnetic resonance (NMR) spectrometer was introduced into a custom-built relative permeability measurement apparatus, and a new method was developed to accurately quantify the displaced water, avoiding the drawback of the dead volume. The changes of water in the coal matrix and cleats were monitored during the unsteady-state displacement experiments. Relative permeability curves for two Chinese anthracite and bituminous coals were obtained, matching the existing research results from the Chinese coalbed methane area. Moreover, the influences of confining pressure on the shape of the relative permeability curve were evaluated. Although uncertainties and limits exist, the NMR-based method is a practical and applicable method to evaluate the gas/water relative permeability of ultra-low permeability rocks.