Strong correlations between aboveground net primary productivity (ANPP) of grasslands and mean annual temperature or precipitation have been widely reported across regional or continental scales; however, inter-annual variation in these climate factors correlates poorly with site-specific ANPP. We hypothesize that the reason for these weak correlations is that the impacts of climatic variation on grassland productivity depend on the timing and intensity of variation in temperature and precipitation. In this study, long-term records of grassland productivity on the Loess Plateau in China were related with daily temperature and precipitation during 1992–2011 using Partial Least Squares (PLS) regression to test the above-mentioned hypothesis. Our results suggested that temperature increases during the early stage of the growing season (April–May) were positively correlated with ANPP. However, these effects were canceled out when this phase was followed by a hot and dry summer (June–July). Impacts of drought and heat in August on productivity were negligible. Increased temperature and precipitation during the senescence period (September–October) and a warmer dormancy phase (November–March) were negatively correlated with productivity in the following year, while precipitation during the dormancy period had no detectable effects. Climatic variability in summer has thus far been the dominant driver of temporal variation in grassland productivity. Warming during winter and spring currently play minor roles, but it seems likely that the importance of these secondary impacts may increase as warming trends continue. This evaluation of climate variability impacts on ecosystem function (e.g. grassland productivity) implies that not only the magnitude but also the timing of changes in temperature and precipitation determines how the impacts of climate changes on ecosystems will unfold.