The shape memory effect in an Fe-14Mn-5Si-9Cr-5Ni alloy with different original microstructures was studied by transmission electron microscopy (TEM) observations and X-ray diffraction (XRD) analysis. By using a series of heat treatments, dislocation tangles, dislocation pile-up, second phase precipitation, segregation of the alloy atoms, and alpha＇-martensite were introduced into the original microstructures, to strengthen the parent phase austenite. With the strengthening of the matrix, the irreversible plastic deformation in austenite during prestrain was put back, which resulted in a significant improvement of the shape memory effect (SME).