The competition between intergranular (IG) and transgranular (TG) fracture in fcc polycrystalline aggregates with physically representative GB misorientation distributions comprised of random low-angle, random high-angle, and coincident site lattice (CSL) GBs has been investigated. Physically-based critical conditions for IG fracture, due to the formation of dislocation pileups, and TG fracture, due to the propagation of cracks on cleavage planes, were coupled to a dislocation-density-based crystal plasticity formulation and a computational fracture scheme for crack branching to investigate how dislocation-GB interactions influence dislocation transmission, pileup formation, and local failure modes. The predictions indicate that aggregates with a large fraction of random and CSL high-angle GBs are dominated by IG fracture, as low GB transmission leads to extensive dislocation-density pileup formation and localized stress accumulations that induce IG fracture. Aggregates with a majority of low-angle GBs are dominated by TG failure, which is consistent with experimental observations. This investigation provides a fundamental understanding of the physical mechanisms governing IG and TG fracture in polycrystalline aggregates.