A hierarchical carbon/reduced graphene oxide (rGO)/FeOx (CGF) composite has been successfully prepared utilizing a sequential process of immersion and carbonization. During the immersion of melamine foam in a liquid precursor, graphene oxide (GO) is covered on the interconnected framework of the foam and is decorated by metallic ions simultaneously. Therefore, the hierarchical structure is constructed in one step. The CGF composite exhibits a maximum reflection loss of as much as -51.2 dB and an ultrawide effective absorption bandwidth of 8.2 GHz. Detailed investigation suggests that the hierarchical structure contributes to the superb microwave absorption performance significantly through enhancing the interfacial polarization and inducing multireflection of microwave. The simple and cost-effective fabrication process together with the excellent microwave absorption performance endows the CGF composite with great potential for practical application.