In this article, we investigate the problem of simultaneously steering an uncountable family of finite-dimensional time-varying linear systems with the same control signal. This class of control problems motivates further research in the new subject of control theory called Ensemble Control, a notion coming from the study of complex spin dynamics in nuclear magnetic resonance spectroscopy and imaging. We derive the necessary and sufficient controllability conditions and an accompanying analytical optimal control law for an ensemble of finite-dimensional time-varying linear systems. We show that ensemble controllability is in connection with singular values of the operator characterizing the system dynamics. In addition, the problem of optimal ensemble control of harmonic oscillators is studied to demonstrate the controllability results. We show that the optimal solutions are pertinent to the study of time-frequency limited signals and prolate spheroidal wave functions. A systematic study of ensemble control systems has immediate applications to dynamical systems with parameter uncertainty as well as to wide-ranging areas such as neuroscience and quantum control. The work in ensemble control will foster further developments in mathematical control and systems theory.