In general, cross-link is applied into conjugated polymers for improving stability of organic solar cells, but its effect on the photovoltaic performance received little attention. Particularly, cross-linked conjugated polymers with poor solubility show strong aggregation in bulk-heterojunction thin films, resulting in low charge generation efficiencies and hence poor performance in solar cells. Herein, we were able to develop a series of cross-linked conjugated polymers as electron donor for application in organic solar cells with a non-fullerene acceptor IT-4F, in which the photovoltaic performance and morphological stability in a nitrogen environment could be significantly improved. The polymers contain an electron-deficient thieno[3,4-c]pyrrole-4,6-dione (TPD) unit alternated with two-dimensional benzodithiophene, in which a four brominated-TPD monomer was applied into polymerization to obtain cross-linked polymers. All these cross-linked polymers perform identical absorption spectra, energy levels, and hole mobilities with the non-cross-linked polymer, but their photovoltaic performance was different. The cross-linked polymer containing 3% cross-linker as electron donor realized a high power conversion efficiency of 12.18% in non-fullerene organic solar cells, while the polymer donor without cross-linker only provided a low PCE of 7.56%. The greatly enhanced performance in cross-linked polymer solar cells was due to optimized nanophase separation with crystalline and small domain in blended thin films, resulting in efficient charge generation. Our results demonstrate that by rationally designing cross-linked conjugated polymers, it is possible to simultaneously obtain high performance and morphological stability in a nitrogen environment in organic solar cells, enabling their great potential application in large-area devices.