Effective linkage of forecasting and optimization/rebalancing in asset management
We are proud to introduce our newest paper: «Stock Portfolio Management Based on AI Technology». This paper introduces a framework that integrates Long Short-Term Memory (LSTM) networks and other advanced AI techniques to forecast time series data—specifically, stock movements. These forecasts are then applied in a hybrid portfolio optimization proposal. The core objective is to enhance portfolio performance and diversification by selecting assets based on categories that go beyond traditional sectors, while incorporating advanced forecasting capabilities.
The AI contribution centers on the stock prediction subproblem, which is uncoupled from the portfolio optimization process. This separation allows for independent and robust estimation of stock closing prices at selected time horizons. The authors explore cutting-edge machine learning technologies for this prediction task, focusing notably on recurrent neural networks such as LSTM and attention mechanism-based Transformers.
Specifically, the paper details the implementation and evaluation of several AI models. LSTM-based models include pure LSTM, stacked-LSTM, bidirectional-LSTM, CNN-LSTM, Attention-LSTM, and multivariate LSTM. The research found that LSTM models generally demonstrate superior performance for short-term predictions, around 30 days. Transformer models, which use a self-attention mechanism, were implemented and evaluated with both one-dimensional and multi-dimensional inputs. The findings suggest that Transformer architectures, particularly those with a high number of attention heads, perform well and show greater stability over longer prediction periods.
These models are trained using historical daily open prices, trading volume, the RSI14d technical indicator, and—most notably—investor sentiment derived from multiple data sources, including Twitter/X and news outlets. Sentiment data, which includes positive and negative ratios for both Twitter/X and news, is pre-processed and incorporated into the models, treating the prediction as a multivariate time series problem.
The predicted stock performance produced by these AI models is integrated into a Hybrid Portfolio Optimization Algorithm. This algorithm aims to minimize variance while maximizing expected return. It incorporates asset diversification through user-defined categories and enforces a constraint on the minimum number of categories (N) that must be included in the portfolio. This innovative approach allows for stock selection based not only on expected performance but also on diversification across asset categories, such as everyday working companies, services and entertainment, and hardware-focused firms. Furthermore, the uncoupled design enables regular portfolio rebalancing by evaluating replacement stocks based on updated forecasts and transaction costs, making the management process dynamic and adaptable.
The concepts and methodologies presented in this paper have strong parallels with the objectives of the DIGEST project within the industrial asset management context. The connection lies primarily in the shared application of AI for forecasting, optimization, and dynamic decision-making under risk and specific constraints. A key technical objective of DIGEST is to «verify and validate statistical learning models adjusted to empirical data obtained from assets and operating processes.» The project also emphasizes advanced forecasting to deliver the most effective scenarios for asset managers.
The AI models (LSTM, Transformers) and the approach of separating forecasting from optimization and rebalancing, as demonstrated in the stock portfolio context, are directly transferable to industrial assets. This includes tasks like predictive maintenance (industrial asset prognosis) and anticipating machine failures.