Using a novel microcalorimeter, we have performed the first specific heat measurements of C-84, Sc-2@C-84, C-82, and La@C-82 (10-300 K). We analyze these results using a framework based on the phonon modes in these materials. C-84 compares qualitatively to C-60 with a clear separation between inter-and intraball modes, although the interball modes are significantly softer in C-84. For Sc-2@C-84 the added optical modes due to the metal atoms are high-energy Einstein modes comparable to the on-ball modes. Thus, the specific heat of Sc-2@C-84 is very similar to that of C-84; and likewise, the specific heat of La@C-82 resembles that of C-82. Remarkably, however, C-82 contrasts sharply with the other empty fullerenes in that it shows no separation of energy levels between inter-and intraball modes. We speculate about possible causes of this anomalous behavior.