Straw open burning is common practice in agricultural production, and has been identified as highly relevant to regional haze. A variety of control measures for straw open burning have been adopted in agricultural areas to mitigate air pollution, but their contribution to air quality improvement is difficult to estimate, and has been addressed by few studies. Using Jilin Province during the post-harvest season (PHS) as a study area, we empirically evaluated the contribution of the Straw Open Burning Prohibition Plan (SOBP) to air quality improvement by response surface methodology (RMS) modeling combined with scenario analysis, and constructed and introduced a stagnant index to the RMS model. The results indicated that the adoption of the SOBP by Jilin Province in the 2018 PHS mitigated air pollution effectively. The contribution of the plan to air quality improvement was also evaluated under two scenarios: if the SOBP had been implemented during the 2015 PHS, 2016 PHS, and 2017 PHS, the cumulative PM 2.5 concentration would have declined in these years by 23%, 15%, and 22%, respectively, compared with their actual values; if the SOBP had not been implemented in the 2018 PHS, the cumulative PM 2.5 concentration would have increased by 10%-14% compared with the actual value. The results can provide decision makers with a reference to assess the environmental performance of the SOBP, and guidance for establishing scientific environmental regulations for cleaner agricultural production and sustainable development.