A series of proton exchange membranes were prepared by incorporating phosphonic functionalized siloxane into sulfonic poly(2,6-dimethyl-1,4-phenyleneoxide) (SPPO) and imidazole functionalized poly(2,6-dimethyl-1,4-phenyleneoxide) grafted with siloxane (IPPO-Si). Phosphonic acid functionalized siloxane was synthesized from amino trimethyl phosphonic acid (ATMP) and (3-aminopropyl)triethoxysilane (APTES). FTIR results showed that 1-methylimidazole and APTES were successfully grafted onto polyphenylene oxide, and APTES successfully formed Si-O-Si crosslinked networks through hydrolytic crosslinking. The membranes were thermally stable up to 220 °C and exhibited excellent oxidative stability and mechanical performance. We also measured the proton conductivity of the membranes. The results showed that the proton conductivities of the composite membranes increased with the increasing of SPPO content at different degrees under high (100 °C-160 °C) and low (25 °C-80 °C) temperature conditions. Furthermore, the IPPO-SI-P/SPPO-30 has the best conductivity, reaching to 0.1131 S cm-1 at 80 °C, 100%RH and 0.1049 S cm-1 at 160 °C, 5%RH, respectively. Therefore, this novel membrane acts as a potential candidate for proton exchange membranes with a wider applicable temperature (25 °C-160 °C).