Depth resolution for secondary ion mass spectrometry (SIMS) profiling of poly Si/SiO2/Si gate stacks is often hindered by roughness of poly Si surfaces. In order to maintain the high depth resolving power of low-energy SIMS for the gate stacks, an ion-milling method was developed to smoothen and thin poly Si films by O-2(+) beam bombardment at normal incidence. The ion milling has led to a remarkable smoothening in the sputtered area: the rms roughness in poly Si was reduced from 4.5 to 0.6 nm, after removal of 120 nm poly Si by 5 keV O-2(+) bombardment at normal incidence. SIMS profile analyses with the ion milling provided detailed in-depth distributions of As and Si in a gate stack, showing vividly the pile up of As at the poly Si/SiO2 interface and the difference in the beam-induced diffusion of As in Si and SiO2.