By in situ combining the dual cross-linking matrices of the carboxylated agarose (CA) and the silk fibroin (SF) with the hydroxyapatite (HA) crystals, the CA-SF/HA composites with optimal physicochemical and biological properties were obtained, which were designed to meet the clinical needs of load-bearing bone repair. With the synergistic modulation of the dual organic matrices, the HA nanoparticles presented sheet and rod morphologies due to the preferred orientation, which successfully simulated the biomineralization in nature. The chemical reactivity of the native agarose (NA) was significantly enhanced via carboxylation, and the CA exhibited higher thermal stability than the NA. In the presence of SF, the composites showed optimal mechanical properties that could meet the standard of bone repair. The degradation of the composites in the presence of CA and SF was significantly delayed such that the degradation rate of the implant could satisfy the growth rate of the newly formed bone tissue. The in vitro tests confirmed that the CA-SF/HA composite scaffolds enabled the MG63 cells to proliferate and differentiate well, and the CA/HA composite presented greater capability of promoting the cell behaviors than the NA/HA composite. After 24 days of implantation, newly formed bone was observed at the tibia defect site and around the implant. Extensive osteogenesis was presented in the rats treated with the CA-SF/HA composites. In general, the CA-SF/HA composites prepared in this work had the great potential to be applied for repairing large bone defects.