We report on the aggregation behavior of asymmetric amphiphilic poly(ethylene oxide)-block-poly(n-butyl acrylate) (PEO-PnBA) diblock copolymers into vesicles and micelles in aqueous media. These copolymers are composed of a hydrophilic PEO block of a constant chain length of 113 monomer units and longer soft hydrophobic PnBA blocks of 163 and 243 units, respectively. This study presents the first insight into the solution properties of such copolymers examined by dynamic light scattering (DLS), cryogenic and conventional transmission electron microscopy (cryo-TEM, TEM), field-flow fractionation (FFF), and turbidity measurements. Whereas the block copolymer with the shorter PnBA block forms crew-cut micelles, for the polymer with the longer hydrophobic block, the morphology of the aggregates strongly depends on concentration. Besides crew-cut micelles, we observed very peculiar vesicular structures, in particular entrapped, multilamellar, and multicompartment vesicles. Although PnBA has a low glass transition temperature, the aggregates formed in water are kinetically frozen.