The band gaps and spectral shifts of CdS, CdSe, CdTe, AlP, GaP, GaAs, and InP semiconductor clusters are calculated from band structure calculations using accurate local and nonlocal empirical pseudopotentials. The effect of spin-orbit coupling on the band structures is included in the calculations when they are important. The complete set of pseudopotential parameters and full computational details are reported for all these semiconductors. The calculated spectral shifts of zinc-blende and wurtzite CdS, wurtzite CdSe, zinc-blende CdTe, and zinc-blende InP clusters are in good agreement with experiments over a range of cluster sizes. The effect of crystal structure on the band gaps is small in large clusters but becomes important in small clusters. Spin-orbit coupling splits the valence band into A, B, and C sub-bands and we identify transitions arising from these sub-bands in the spectra of both CdSe and CdTe clusters. These results demonstrate that the empirical pseudopotential method yields unique insights into the quantum confinement effects and is a powerful quantitative tool for calculating the spectral shifts of semiconductor clusters.