Hybrid XGBoost and LSTM Model Design Based on Attention Mechanism for Real-Time Stock Needs Prediction

Julia Totong; Albert Sanders; Nethanie Paramita Wijaya Kusumah

doi:10.55927/famq9b54

Authors

Julia Totong Program Studi Ilmu Kesehatan Masyarakat, Fakultas Farmasi, Universitas Indonesia Maju Author
Albert Sanders Program Studi Sistem Informasi, Fakultas Teknik dan Informatika, Universitas Multimedia Nusantara Author
Nethanie Paramita Wijaya Kusumah Program Studi Sistem Informasi, Fakultas Teknik dan Informatika, Universitas Multimedia Nusantara Author

DOI:

https://doi.org/10.55927/famq9b54

Keywords:

Stock Prediction, XGBoost-LSTM Hybrid, Attention Mechanism, Supply Chain, Time Series

Abstract

Uncertainty in the supply chain and high fluctuations in market demand push companies to have a stock demand forecasting system that is accurate and responsive in real-time. Conventional forecasting models often fail to capture non-linear patterns and long-term temporal relationships in inventory transaction time series data. This study proposes a new hybrid model architecture that combines Extreme Gradient Boosting (XGBoost) for non-linear feature extraction and Long Short-Term Memory (LSTM) reinforced with an Attention Mechanism to selectively capture long-term temporal dependencies. The model evaluation is tested using Root Mean Square Error (RMSE), Mean Absolute Error (MAE), and Mean Absolute Percentage Error (MAPE) metrics. The experiment results show that the hybrid XGBoost-LSTM-Attention model can outperform single models (pure LSTM or XGBoost alone) with a 14.2% decrease in MAPE, making it very effective to implement for real-time stock management systems

References

T. Chen and C. Guestrin, "XGBoost: A scalable tree boosting system," in Proceedings of the 22nd ACM SIGKDD International Conference on Knowledge Discovery and Data Mining, 2016, pp. 785-794.

A. Vaswani et al., "Attention is all you need," in Advances in Neural Information Processing Systems (NeurIPS), 2017, pp. 5998-6008.

H. Zhang, L. Wang, and J. Smith, "Real-time inventory forecasting and supply chain optimization using deep learning techniques," International Journal of Production Economics, vol. 255, p. 108682, 2023.

R. Wang, K. Li, and M. Zhou, "A hybrid XGBoost-LSTM model for time series forecasting with complex non-linear trends," Expert Systems with Applications, vol. 238, p. 122145, 2024.

S. Hochreiter and J. Schmidhuber, "Long short-term memory," Neural Computation, vol. 9, no. 8, pp. 1735–1780, 1997.

Y. Li, Z. Zhu, and D. E. Kouoksam, "A hybrid CNN-LSTM-Attention model for short-term demand forecasting in supply chain management," Computers & Industrial Engineering, vol. 171, p. 108420, 2022.

M. A. W. Pratama and D. Suhartono, "XGBoost and LSTM hybrid architecture for sales prediction in retail inventory management," Journal of Big Data, vol. 10, no. 1, p. 45, 2023.

S. A. Aloqaily et al., "Real-time data analytics and forecasting for smart supply chain management using IoT and edge computing," IEEE Transactions on Industrial Informatics, vol. 18, no. 8, pp. 5621–5630, 2022.