Knowing when a physical system has reached sufficient size for its macroscopic properties to be well described by many-body theory is difficult. We investigated the crossover from few-to many-body physics by studying quasi-one-dimensional systems of ultracold atoms consisting of a single impurity interacting with an increasing number of identical fermions. We measured the interaction energy of such a system as a function of the number of majority atoms for different strengths of the interparticle interaction. As we increased the number of majority atoms one by one, we observed fast convergence of the normalized interaction energy toward a many-body limit calculated for a single impurity immersed in a Fermi sea of majority particles.