Red mud is a kind of by-product generated in alumina production and typically contains some rare metals like Sc and Ga. In this study, Time of Flight-Secondary Ion Mass Spectrometry (ToF-SIMS) and Electron Probe Micro Analysis (EPMA) were used to investigate the affinities of Sc and Ga with the major elements including Fe, Al, Si, Ti and Ca existed in red mud. Combined the mapping analyses of high mass resolution of ToF-SIMS with individual spot and mapping analyses of EPMA, the substitution of scandium and gallium for these major elements should follow the order of Ti > Fe > Al > Si and Fe > Al > Ti > Ca > Si, respectively. Scandium has an apparent enrichment in the mineral phases of anatase, hematite and goethite. The hetero-valent isomorphism of 4Sc(3+) -> 3Ti(4+) plays a critical role in substitution of Sc3+ into the anatase structure and is different from that of Sc3+ -> Fe3+ occurred in hematite or goethite. Part of scandium is also found to be existed in silicon-rich minerals of quartz or zeolite. Gallium is closely coordinated with ferrum and aluminum, suggesting that it can proxy for these elements in the crystal lattices of hematite, goethite and gibbsite. All the observations are fundamental clues to the extraction technology of scandium and gallium from red mud.