A segmented poly(ester urethane) shape-memory polymer with poly(epsilon-caprolactone) (PCL) soft segment and urethane hard segment was studied by FTIR to investigate the structural evolution in the shape-memory cycle. It was revealed that in the cold drawing programming process, first the amorphous PCL chains start to orient along the drawing direction, and then, the hard segment and the crystalline PCL chains orient upon further extension, accompanied by the weakening of the hydrogen bonds along the drawing direction between the hard segments and stress-induced disaggregation and recrystallization of the crystalline PCL. In the recovery process, the hard segments restore first with the parallel hydrogen bonds strengthening and then the amorphous PCL chains restore the original random orientation, and finally the crystalline PCL chains lose their alignment to a less-oriented state.