This study sought to investigate the effect of sulfur levels on changes in the fungal community composition of arbuscular mycorrhizae (AM) at the pod-setting stage and the relationship between the amount of applied sulfur and AM fungal diversity in different soybean cultivars. The objective of the research was to determine the optimal sulfur application level for different soybean cultivars and to improve soybean yield and quality from the perspective of AM fungal diversity. Three soybean cultivars, Heinong 44, Heinong 48, and Heinong 37, were selected as study materials. In addition to 0.033 g each of N, P2O5 and K2O per kg of soil, 0, 0.02, 0.04, or 0.06 g of elemental sulfur was applied to each kg of soil for the four treatment groups, S1, S2, S3, and S4, respectively. The AM fungal community structure was analyzed in the soil and root of different soybean cultivars using the PCR-DGGE technology. The results indicated a significant effect of sulfur on the AM fungal community structure in the roots and rhizospheric soil of different soybean cultivars. The three soybean cultivars in group S2 exhibited the highest diversity in AM fungus. Significant changes in the dominant fungal species were observed in the DGGE fingerprints of each sample, and Glomus, Funneliformis, Rhizophagus, and Claroideoglomus fungi were the dominant species of AM fungus in the roots and soil of soybean. The application of an appropriate amount of sulfur improved the diversity of AM fungi in roots and rhizospheric soil of different soybean cultivars.