In this work, we investigate the configuration of the contact line of a water drop lying on an ultrahydrophobic post surface using the numerical algorithm Surface Evolver. For the special situation of Cassie wetting, we propose a modified definition of the contact line as the line in space where the meniscus starts to curve upward out of the plane of the composite surface. In our simulations, it is found that the contact line is very strongly distorted, indicating a strong tendency of the drop to "ball up" in those areas where it is not in contact with the solid surface. The distortion of the contact line corresponds to a pronounced deformation of the liquid-air interface around the base of the drop. We discuss the consequences of this distortion for the definition and practical measurement of the contact angle on ultrahydrophobic surfaces.