Objective  This paper aims to establish a single tree DBH growth model of Picea schrenkiana in Xinjiang of northwestern China in order to predict the DBH growth of Picea schrenkiana and provide a theoretical basis for the forestry department to manage P. schrenkiana forest.

  Method  Taking Picea schrenkiana as the research object, based on the 70 pieces of Tianshan Mountain pure forest retesting sample plots in Xinjiang, a total of 1 914 viable standing trees were measured in the sample plots, and 1 531 sets of data were randomly selected for training data, 383 sets of data for test data. Contrasting and analyzing the application of traditional single-tree DBH model and mixed effects model in the spruce single-tree DBH model, considering the density level effect, sample plot effect and nesting two-level effect when using the R language nlme module to construct the mixed effects model, and using the average absolute error (\left|\bar E\right|) , root mean square error\rm (RMSE), average prediction error (\rm MPE), total relative error (\rm TRE) to test the fitting effects of the model.

  Result  The mixed effects model (R_\rm adj^2 = 0.762) was superior to the traditional breast diameter model (R_\rm adj^2 = 0.505). In the mixed effects model, that based on the nesting two-level was the best. The average absolute error (\left|\bar E\right|) , the root mean square error (\rm RMSE), the average prediction error (\rm MPE), the total relative error (\rm TRE), and the adjustment decision coefficient (R_\rm adj^2) were 0.589 cm, 0.804 cm, 0.966%, − 0.042%, 0.899, respectively. The fitting effect of mixed effects model from high to low was: nesting two-level mixed effects model (R_\rm adj^2 = 0.899) > sample plot mixed effects model (R_\rm adj^2 = 0.766) > density level mixed effects model (R_\rm adj^2 = 0.762). The power function can effectively eliminate the influence of heteroscedastic structure. The first-order autoregressive matrix AR (1) can effectively eliminate the time-dependent effect of the data.

  Conclusion  The mixed model of DBH growth of Picea schrenkiana can be used as the main model for the prediction of DBH diameter in the Picea schrenkiana of Xinjiang, in which the mixed effects model of nesting density level effect and sample plot effect is the best for predicting the DBH diameter (R_\rm adj^2 = 0.899). This study shows that the mixed effects model is an effective method for predicting the single tree DBH of the Picea schrenkiana in Xinjiang, and provides a theoretical basis and a new method for predicting the single tree DBH of the large-scale Xinjiang Picea schrenkiana.