A new parameter estimation method is proposed in the framework of composite model reference adaptive control for improved parameter convergence without persistent excitation. The regressor filtering scheme is adopted to perform the parameter estimation with signals that can be obtained easily. A new framework for residual signal construction is proposed. The incoming data are first accumulated to build the information matrix, and then its quality is evaluated with respect to a chosen measure to select and store the best one. The information matrix is built to have full rank after sufficient but not persistent excitation. In this way, the exponential convergence of both tracking error and parameter estimation error can be guaranteed without persistent oscillation in the external command, which drives the system. Numerical simulations are performed to verify the theoretical findings and to demonstrate the advantages of the proposed adaptation law over the standard direct adaptation law.