CO2 emissions mitigation in iron and steel industry (ISI) and construction material industry (CMI), including cement, glass, and ceramics materials, is crucial for the realization of CO2 emission peak targets in China, given their great contributions to China's emission structure. Great transitions have occurred in the two industries recently, including scale expansion, efficiency improvement, and changes in production and demand structures. By developing an integrated framework for inter-sector linkage analysis, we investigated the impact of recent transitions in the ISI and CMI on China's CO2 emissions between 1992 and 2012. Results show that the CO2 emissions from ISI and CMI increased by 4.2 and 6.8 times over two decades, respectively, and the two key sectors have significantly higher backward and forward linkages than average in terms of CO2 emissions. The internal efficiency improvement of the ISI and CMI are crucial factors curbing the rising CO2 emissions in these two sectors. The total CO2 intensity of the ISI and CMI have declined by 78% and 68%, separately, cumulatively reducing 517 Mt and 704 Mt CO2 emissions during the studied period. The external final demand growth and its structure changes of the ISI and CMI have had a significant impact on their CO2 emissions. The construction sector is the greatest consumer, responsible for 53% and 86% emissions increase of ISI and CMI during 2002-2012, respectively. Emerging manufacturing and machinery also became substantial emissions sources, generating 536 Mt CO2 emissions in 2012 by consuming iron and steel products. Based on these findings, policy recommendations for CO2 emissions mitigation in the two key sectors and their related sectors are also discussed.