American ginseng (Panax quinquefolium L.) is a perennial plant that is cultivated for medicinal purposes. Unfortunately, outbreaks of root rot disease in American ginseng (AG) reduce yields and result in serious economic losses. Information on the dynamics of soil microbial communities associated with healthy and diseased AG of different ages is limited. The present study explored the differences in field soil microbial community structure, composition, interaction, and their predictive functions associated with healthy and diseased AG at different growth ages. Changes in soil physicochemical properties were also examined to determine the possible reasons for disease outbreaks. Results revealed that in different growth years, the genera of soil-borne pathogens, such as Alternaria, Botrytis, Cladosporium, Sarocladium, and Fusarium, were increased in diseased AG soil samples in comparison with those in the healthy AG soil samples. In contrast, the abundance of some key and potentially beneficial microbes, such as Bacillus, Chaetomium, Dyella, Kaistobacter, Paenibacillus, Penicillium, and Trichoderma, was decreased. Additionally, as AG plants age, the relative abundance of symbiotic fungi tended to decrease, while the relative abundance of potential plant pathogenic fungi gradually increased. Various soil properties, such as available phosphorus, the ratio of total nitrogen to total phosphorus (N/P), and pH, were significantly (P < 0.05) associated with microbial community composition. Our findings provide a scientific basis for understanding the relationship among the root rot disease outbreaks in American ginseng as well as their corresponding soil microbial communities and soil physicochemical properties.