Polymer-based materials with high electrical conductivity are of considerable interest because of their wide range of applications. The construction of a 3D, compactly interconnected graphene network can offer a huge increase in the electrical conductivity of polymer composites. However, it is still a great challenge to achieve desirable 3D architectures in the polymer matrix. Here, highly conductive polymer nanocomposites with 3D compactly interconnected graphene networks are obtained using a self-assembly process. Polystyrene (PS) and ethylene vinyl acetate (EVA) are used as polymer matrixes. The obtained PS composite film with 4.8 vol% graphene shows a high electrical conductivity of 1083.3 S/m, which is superior to that of the graphene composite prepared by a solvent mixing method. The electrical conductivity of the composites is closely related to the compact contact between graphene sheets in the 3D structures and the high reduction level of graphene sheets. The obtained EVA composite films with the 3D graphene structure not only show high electrical conductivity but also exhibit high flexibility. Importantly, the method to fabricate 3D graphene structures in polymer matrix is facile, green, low-cost, and scalable, providing a universal route for the rational design and engineering of highly conductive polymer composites.