Abstract: Accurately identifying the key feature factors that influence short-term power load forecasting is an effective means for enhancing forecast accuracy. To address the issue in multidimensional datasets where non-critical features can easily lead to poor fitting capability of prediction models, consequently reducing model accuracy, this paper proposes a short-term power load forecasting method that combines the eXtreme Gradient Boosting (XGBoost) decision tree with an improved Informer model. Firstly, to evaluate the importance of feature factors from multi-dimensional historical load data, the coverage metric of XGBoost decision tree is adopted as an indicator to assess feature importance, thereby enabling accurate screening of the feature factors participating in model training. Subsequently, an improved Informer short-term load forecasting model is constructed. By optimizing the positional encoding design, the selected key features are combined with positional markers of different time scales to form input vectors for the encoder. Finally, ablation experiments are designed to conduct a comparative analysis of model convergence speed and prediction accuracy across different time scales. Experimental results indicate that, compared to other models, the XGB-Informer model demonstrates significant advantages in both prediction accuracy and convergence speed, verifying the effectiveness and superiority of the proposed method.