High temperatures and drought that accompany climate changes decrease food production and threaten food security globally. On the other hand, elevated atmospheric CO2 (eCO2) can enhance the yield of vegetables that are more nutrient-demanding compared to grain crops. Therefore, it is expected that vegetables would respond in another way to eCO2 and interact with environmental changes differently. In this study, we reviewed the effect of eCO2 on yield and yield components of vegetables by a meta-analysis using 107 selected articles. Elevated CO2 (827 μmol mol−1) increased yield of vegetables as a whole by 34% (n = 8877) but did not affect the dry matter content. Elevated CO2 increased the harvest index by 23% whilst it increased the root to shoot biomass ratio by 8%. The increased vegetable yield was associated with increased vegetable number of the organs largely by 32% and vegetable mass to a lesser extent (11%). The least increases in the mass of fruit vegetables (10%) could be the reason for the least yield improvement (30%, n = 5778) when compared to leafy vegetables, stem vegetables, root vegetables and flower vegetables. In addition, the effect of eCO2 on yield tended to differ between the levels of temperature, light, water and N availability, with the differences being greater than those of salinity, O3 and N form. Drought stress appears to be a major constraint limiting the increases in number and yield of vegetables under eCO2. We conclude that environments have to be optimized to improve the yield benefit from high CO2 for a sustainable vegetable production. In scenarios of high CO2 environment, due to climate change, a “wise use” of atmospheric CO2, without implementing an extra source for CO2 is a good step for a cleaner vegetable production.