The effect of filler addition on the grain coarsening characteristics of block copolymer materials is analyzed for the particular case of a lamellar poly(styrene-b-isoprene)-type block copolymer and polystyrene as well as polystyrene-grafted nanoparticle fillers. Filler addition is shown to reduce the rate of grain growth and to induce grain size distributions that deviate from the log-normal type that is characteristic for pristine block copolymer systems. The retardation of grain growth is shown to be associated with the segregation of filler additives into high energy grain boundary defects-a process that bears similarities to the segregation of impurity atoms within grain boundary structures in ceramics or metals. The analysis of grain boundary energy, grain size distribution, and grain coarsening kinetics suggests two major mechanisms for the interference of filler additives with grain coarsening: First, the segregation of fillers into boundary regions lowers the relative grain boundary energy and hence the driving pressure for grain growth. Second, the formation of particle aggregates along grain boundaries gives rise to a "pinning pressure" that counteracts grain growth and that limits the ultimate grain size during thermal annealing. This is in contrast to pristine block copolymer systems in which continuous grain growth is observed during thermal annealing. The results highlight the fundamental differences between structure evolution in pristine and mixed block copolymer systems and suggest that thermal annealing (in the absence of structure-guiding fields) is an inefficient path to facilitate the controlled growth of large grains in athermal block copolymer blend materials.