Research on the relationship between runoff and sediment in the watershed is the foundation of constructing the models of regional water loss and soil erosion and evaluating the benefit of soil and water conservation measures. To investigate the effect of climate and land use change on runoff-sediment relationship, we selected Luoyugou Watershed, a typical small watershed in the hilly loess region, northwestern China, as study object. Hydrological element cumulative anomaly, Mann-Kendall trend test, double mass curve and other methods were applied to analyze the evolution process of the relationship between runoff and sediment and its driving forces in the period of 1986—2010 as well as the quantitative effects of driving factors. The results were as follows: 1) An abrupt change of runoff in the watershed happened in 1993, and runoff and sediment yield during the period of 1994—2010 was reduced by more than 50% compared with that during the period of 1986—1993 under different rainfall conditions. 2) Optimization on land use structures would positively influence the hydrological process and sediment yield. Those warping dams are helpful to reduce significantly the sediment yield in a given unit of runoff through altering the relationship between runoff and sediment. 3) Precipitation and the change of landuse caused by human activities contributed to reductions in runoff by 18.48% and 81.52%, and in sediment yield by 27.15% and 72.85%, respectively.