Energy storage plays a crucial role in saving energy and protecting the environment. The research on phase change latent heat storage materials has been in the forefront of the thermal storage research. However, the low thermal conductivity and the leakage during phase transition have limited the application of the phase change materials (PCMs). Carbon materials can be used to improve the properties of PCMs because of their excellent properties. Carbon materials have high thermal conductivity and excellent adsorption properties. Some carbon materials can improve the mechanical properties, electrical properties and flame retardant properties of PCMs. In this paper, performances and applications of one-dimensional carbon (carbon nanotubes, carbon fibers and graphite fibers) based composite PCMs, two-dimensional carbon (graphite, graphene, graphene oxide and exfoliated graphite nanoplates) based composite PCMs and three-dimensional carbon (expanded graphite, carbon based foam, graphene aerogel material and activated carbon) based composite PCMs were summarized and discussed. The advantages and disadvantages of the carbon materials with various dimensions in the application were also pointed out. References for thermal energy storage application were provided by this paper.