Diarylbibenzofuranone (DABBF), which contains an exchangeable dynamic covalent carbon-carbon bond was introduced into the polymer or silica domains of two series of polymer/silica composites with a network structure in order to investigate the correlation between the structural reorganization and the location of the DABBF units in these polymer/silica composites. The thermally induced structural reorganization behavior of the two polymer/silica composites was investigated by optical microscopy, dynamic mechanical analysis, and variable-temperature electron paramagnetic resonance spectroscopy. The thus obtained results revealed that the cross-linked polymer/silica composites that contain DABBF units in the polymer domain exhibit network reorganization, and that the efficiency of the network reorganization increases with the molecular weight between the cross-linking points composed of silica domains. Conversely, the cross-linked composite that contains the DABBF units in the silica domains does not show a network reorganization, as the densely cross-linked silica structure prevents the bond exchange reactions of the DABBF units. In their entirety, these results clearly indicate that the incorporation of dynamic covalent linkages in the high mobile polymer domain of cross-linked polymers is advantageous for structural reorganization, which affords a new guiding principle for the design of dynamic-covalent-chemistry-based materials.