We present observations of cavitation that occur inside a capillary die during extrusion of polyethylene. This phenomenon was observed over the last 1.5 mm of the capillary tube immediately upstream of the exit. We observed spontaneous formation of voids near the wall that grew to a typical length and width of 150 mum, and then shrank and disappeared over a time frame of approximately 20 ms. From velocity measurements of these structures, we concluded that their width in the radial direction was smaller than in the axial and lateral directions, and they were near the wall. The shape of the cavities was highly irregular. We assessed the roles of extensional stress and shear stress at the exit region and concluded that they were not the direct cause of cavitation. Rather, cavitation occurs in conjunction with an upstream rupture of the polymer that occurs in the contraction region leading into the capillary tube (gross melt fracture). We argue that the exit region does, however, serve as the initiation point of the cavitation because of a combination of the reduced pressure and extensional stress.