DEVELOPMENT AND COMPARATIVE ANALYSIS OF A HYBRID LSTM–XGBOOST MODEL FOR STOCK MARKET TIME SERIES FORECASTING

Abstract

Accurate stock market forecasting remains a challenging problem due to the nonlinear, noisy, and volatile nature of financial time series. This paper proposes a hybrid framework combining Long Short-Term Memory (LSTM) networks and eXtreme Gradient Boosting (XGBoost) for stock market prediction. LSTM serves as a temporal feature extractor to learn latent sequential representations, while XGBoost generates final predictions from the extracted features. The framework is evaluated on daily OHLCV stock data against standalone LSTM and XGBoost models using MAE, RMSE, MAPE, and R². Experimental results show that the standalone LSTM achieved the best performance (MAE = 15.4081, RMSE = 18.6627, MAPE = 6.1850%, R² = 0.501268), while both XGBoost and the hybrid model produced substantially larger errors with negative R² values. The findings demonstrate that, under the current experimental setting, model hybridization does not automatically improve forecasting accuracy and highlight the importance of compatibility between deep sequential feature extraction and tree-based regression.