Three acceptor-acceptor (A-A) type conjugated polymers based on isoindigo and naphthalene diimide/perylene diimide are designed and synthesized to study the effects of building blocks and alkyl chains on the polymer properties and performance of all-polymer photoresponse devices. Variation of the building blocks and alkyl chains can influence the thermal, optical, and electrochemical properties of the polymers, as indicated by thermogravimetric analysis, differential scanning calorimetry, UV-vis, cyclic voltammetry, and density functional theory calculations. Based on the A-A type conjugated polymers, the most efficient all-polymer photovoltaic cells are achieved with an efficiency of 2.68%, and the first all-polymer photodetectors are constructed with high responsivity (0.12 A W-1) and detectivity (1.2 x 10(12) Jones), comparable to those of the best fullerene based organic photodetectors and inorganic photodetectors. Photoluminescence spectra, charge transport properties, and morphology of blend films are investigated to elucidate the influence of polymeric structures on device performances. This contribution demonstrates a strategy of systematically tuning the polymeric structures to achieve high performance all-polymer photoresponse devices.