We recorded photoelectron spectra of antimony Sb-N=2-9(-) and of bismuth clusters Bi-N=2-9(-) with a photon energy of 4.03 eV, as well as of Bi-N=2-21(-) with a photon energy of 5.0 eV. The experimentally determined photoelectron thresholds and peak positions of Sb-N=2-5(-) and Bi-N=2-5(-) are compared with the results of ab initio density-functional (LCAO) calculations. The agreement between the experimental thresholds and the calculated adiabatic electron affinities, as well as between the first maxima in the spectra and the calculated vertical detachment energies is fair to good for the antimony clusters and qualitative for the bismuth systems. For the calculation of the ionization (detachment) energies we determined for the neutral and anionic clusters the most stable structures by LCAO calculations; In particular, the tetramer cluster anions have a "roof" structure, while the negatively charged pentamers are planar rings [with similarities to the (C5H5)(-) anion]; positive and negative trimers are nonlinear. Furthermore, the ionization energies and affinities of larger antimony and bismuth clusters are discussed qualitatively and compared to jellium calculations of Seidl and Brack.