We report kinetic studies of therapeutically highly potent polymer drug conjugates consisting of amphiphilic N-(2-hydroxypropyl) methacrylamide (HPMA)-based copolymers bearing the anticancer drug doxorubicin (Dox). Highly hydrophobic cholesterol moieties as well as the drug were attached to the polymer backbone by a pH-sensitive hydrazone bond. Moreover, the structure of the spacer between the polymer carrier and the cholesterol moiety differed in order to influence the release rate of the hydrophobic moiety, and thus the disintegration of the high-molecular-weight micellar nanoparticle structure. We performed time-dependent SAXS/SANS measurements after changing pH from a typical blood value (pH 7.2) to that of tumor cells (pH 5.0) to characterize the drug release and changes in particle size and shape. Nanoparticles composed of the conjugates containing Dox were generally larger than the drug-free ones. For most conjugates, nanoparticle growth or decay was observed in the time range of several hours. It was established that the growth/decay rate and the steady-state size of nanoparticles depend on the spacer structure. From analytical fitting, we conclude that the most probable structure of the nanoparticles was a core shell or a core with attached Gaussian chains. We concluded that the spacer structure determined the fate of a cholesterol derivative after the pH jump. Fitting results for 5a-cholestan-3-onecholestan-3-one and cholestery1-4-oxopentanoate (Lev-chol) implied that cholesterol moieties continuously escape from the core of the nanoparticle core and concentrate in the hydrophilic shell. In contrast, cholest-4-en-3-one spacer prevent cholesterol escaping. Dox moiety release was only observed after a change in pH. Such findings justify the model proposed in our previous paper. Lastly, the cholesteryl 4-(2-oxopropypbenzoate (Opb-Chol) was a different case where after the release of hydrophobic Opb-Chol moieties, the core becomes more compact. The physicochemical mechanisms responsible for the scenarios of the different spacers are discussed.