Experiments in coherent magnetic resonance, microwave, and optical spectroscopy control quantum-mechanical ensembles by guiding them from initial states toward target states by unitary transformation. Often, the coherences detected as signals are represented by a non-Hermitian operator. Hence, spectroscopic experiments, such as those used in nuclear magnetic resonance, correspond to unitary transformations between operators that in general are not Hermitian, A gradient-based systematic procedure for optimizing these transformations is described that finds the largest projection of a transformed initial operator onto the target operator and, thus, the maximum spectroscopic signal. This method can also be used in applied mathematics and control theory.