To obtain a satisfying anticancer effect, rationally designed nanocarriers are intensively studied. In this field, heparin and its derivatives have been widely attempted recently as potential component of nanocarriers due to their unique biological and physiochemical features, especially the anticancer activity. This review focuses on state-of-the-art nanocarriers with heparin/heparin derivatives as backbone or coating material. At the beginning, the unique advantages of heparin used in cancer nanotechnology are discussed. After that, different strategies of heparin chemical modification are reviewed, laying the foundation of developing various nanocarriers. Then a systematic summary of diverse nanoparticles with heparin as component is exhibited, involving heparin-drug conjugate, polymeric nanoparticles, nanogels, polyelectrolyte complex nanoparticles, and heparin-coated organic and inorganic nanoparticles. The application of these nanoparticles in various novel cancer therapy (containing targeted therapy, magnetic therapy, photodynamic therapy, and gene therapy) will be highlighted. Finally, future challenges and opportunities of heparin-based biomaterials in cancer nanotechnology are discussed.