We explore the ability of a concentrating solar power (CSP) plant with thermal energy storage (TES) to provide peaking capacity. We focus on future power systems, wherein net load patterns may be significantly different than they are today (e.g., due to higher renewable-energy penetrations). We examine 28 locations in the southwestern United States over an 18-year period. The hourly operation of the CSP plants are simulated to determine their potential to provide energy during an eight-hour peak-load window for each day up to 365 days per year. Our result shows that for the large majority of locations and years, CSP plants with certain configurations (i.e., in terms of solar field and TES sizes) can provide nearly 100% peak-load capacity. We examine also the amount of supplemental energy (e.g., by using natural gas as a supplemental thermal-energy source) that would be required to ensure that a CSP plant could serve the eight highest-load hours of every day of the year. We find that in most cases, a CSP plant supplemented with natural gas would require less than 5% of the fuel that is used by a natural-gas fired power plant providing the same level of reliable capacity. A series of sensitivity analyses show that these results are robust to the number of peak-load hours and days that are considered and the configuration of the CSP plant.