Nanocarbon (carbon nanotubes (CNTs) and graphene (GN)) composites attract considerable research interest due to their fascinating applications in many fields. Here, recent developments in the field of in situ chemical vapor deposition (CVD) for the design and controlled preparation of advanced nanocarbon composites are highlighted, specifically, CNT-reinforced bulk structural composites, as well as CNT, GN, and CNT/GN functional composites, together with their practical and potential applications. In situ CVD is a very attractive approach for the fabrication of composites because of its engaging features, such as its simplicity, low-cost, versatility, and tunability. The morphologies, structures, dispersion, and interface of the resulting nanocarbon composites can be easily modulated by varying the experimental parameters (such as temperature, catalysts, carbon sources, templates or template catalysts, etc.), which enables a great potential for the in situ synthesis of high-quality nanocarbons with tailored size and dimension for constructing high-performance composites, which has not yet been achieved by conventional methods. In addition, new trends of the in situ CVD toward nanocarbon composites are discussed.