Lithium slag (LS) is discharged as a byproduct in the process of the lithium carbonate, and it is very urgent to explore an efficient way to recycle LS in order to protect the environments and save resources. Many available supplementary cementitious materials for partial replacement of cement and/or silica fume (SF) can be used to prepare ultra high performance concrete (UHPC). The effect of LS to replace SF partially by weight used as a supplementary cementitious material (0%, 5%, 10% and 15% of binder) on the compressive strengths and microstructure evolution of UHPC has experimentally been studied by multi-techniques including mercury intrusion porosimetry, scanning electron microscope and nanoindentation technique. The results show that the use of IS degrades the microstructure of UHPC at early ages, and however, the use of LS with the appropriate content improves microstructure of UHPC at later ages. The hydration products of UHPC are mainly dominated by ultrahigh density calcium-silicate-hydrate (UHD C-S-H) and interfacial transition zone (ITZ) in UHPC has similar compact microstructure with the matrix. The use of IS improves the hydration degree of UHPC and increases the elastic modulus of ITZ in UHPC. LS is a promising substitute for SF for preparation UHPC.