Throughout life, neural stem cells (NSCs) generate neurons in the mammalian brain. Using photobleaching experiments, we found that during cell division in vitro and within the developing mouse forebrain, NSCs generate a lateral diffusion barrier in the membrane of the endoplasmic reticulum, thereby promoting asymmetric segregation of cellular components. The diffusion barrier weakens with age and in response to impairment of lamin-associated nuclear envelope constituents. Weakening of the diffusion barrier disrupts asymmetric segregation of damaged proteins, a product of aging. Damaged proteins are asymmetrically inherited by the nonstem daughter cell in embryonic and young adult NSC divisions, whereas in the older adult brain, damaged proteins are more symmetrically distributed between progeny. Thus, these data identify a mechanism of how damage that accumulates with age is asymmetrically distributed during somatic stem cell division.