Black soils (Mollisols) of northeast China are highly productive and agriculturally important for food production. Ammonia-oxidizing microbes play an important role in N cycling in the black soils. However, the information related to the composition and distribution of ammonia-oxidizing microbes in the black soils has not yet been addressed. In this study, we used the amoA gene to quantify the abundance and community composition of ammonia-oxidizing archaea (AOA) and ammonia-oxidizing bacteria (AOB) across the black soil zone. The amoA abundance of AOA was remarkably larger than that of AOB, with ratios of AOA/AOB in the range from 3.1 to 91.0 across all soil samples. The abundance of AOA amoA was positively correlated with total soil C content (p < 0.001) but not with soil pH (p > 0.05). In contrast, the abundance of AOB amoA positively correlated with soil pH (p = 0.009) but not with total soil C. Alpha diversity of AOA did not correlate with any soil parameter, however, alpha diversity of AOB was affected by multiple soil factors, such as soil pH, total P, N, and C, available K content, and soil water content. Canonical correspondence analysis indicated that the AOA community was mainly affected by the sampling latitude, followed by soil pH, total P and C; while the AOB community was mainly determined by soil pH, as well as total P, C and N, water content, and sampling latitude, which highlighted that the AOA community was more geographically distributed in the black soil zone of northeast China than AOB community. In addition, the pairwise analyses showed that the potential nitrification rate (PNR) was not correlated with alpha diversity but weakly positively with the abundance of the AOA community (p = 0.048), whereas PNR significantly correlated positively with the richness (p = 0.003), diversity (p = 0.001) and abundance (p < 0.001) of the AOB community, which suggested that AOB community might make a greater contribution to nitrification than AOA community in the black soils when ammonium is readily available.