Since the first discovery of graphene in 2004, the scientific and industrial community is still finding fields to bring the excellent mechanical performance of graphene into full play. Herein, this review summarizes the recent advances in graphene and graphene derivatives toughened polymer nanocomposites, which gives full play to the unique two-dimensional nanostructure, extra specific surface area and ultrahigh mechanical properties of graphene and graphene derivatives. Using graphene and graphene derivatives as toughening modifier of polymers can not only enhance the toughness of the polymer matrix but also endow the polymer nanocomposites with enhanced strength. The preparation, performance and mechanism of graphene and graphene derivatives toughened polymer nanocomposites are highlighted in this review. The factors influencing the efficiency of graphene and graphene derivatives toughening polymers are discussed in detail. Current challenge and future direction in graphene and graphene derivatives toughened polymer nanocomposites are proposed for promoting the application of graphene in toughening polymers. This review provides a guidance for fabricating high-performance polymer/graphene nanocomposites.