The collapse and swelling behavior of a generic homopolymer is studied using implicit-solvent, explicit-cosolvent Langevin dynamics computer simulations for varying interaction strengths. The systematic investigation reveals that polymer swelling is maximal if both monomer-monomer and monomer-cosolute interactions are weakly attractive. In the most swollen state the cosolute density inside the coil is remarkably bulk-like and homogeneous. Highly attractive monomer-cosolute interactions, however, are found to induce a collapse of the chain, which, in contrast to the collapsed case induced by purely repulsive cosolvents, exhibits a considerably enhanced cosolute density within the globule. Thus, collapsed states, although appearing similar on a first glance, may result from very different mechanisms with distinct final structural and thermodynamic properties. Two theoretical models, the first based on an effective one-component description where the cosolutes have been integrated out and, second, a fully two-component Flory-de Gennes like model, support the simulation findings above and serve for interpretation. In particular, the picture is supported that collapse in highly attractive cosolvents is driven by cross-linking-like bridging effects, while the ratio of attraction range to cosolute size plays a critical role behind this mechanism. Only if polymer-cosolute interactions are not too short-ranged, swelling effects should be observable. Our findings may be important for the interpretation of the effects of cosolutes on polymer and protein conformational structure, in particular for highly attractive interaction combinations, such as provided by urea, GdmCl, NaI, or NaClO4 near peptide-like moieties.