Risk-Sensitive Q-Learning in Continuous Time with Application to Dynamic Portfolio Selection
By: Chuhan Xie
Potential Business Impact:
Helps computers make smarter money choices safely.
This paper studies the problem of risk-sensitive reinforcement learning (RSRL) in continuous time, where the environment is characterized by a controllable stochastic differential equation (SDE) and the objective is a potentially nonlinear functional of cumulative rewards. We prove that when the functional is an optimized certainty equivalent (OCE), the optimal policy is Markovian with respect to an augmented environment. We also propose \textit{CT-RS-q}, a risk-sensitive q-learning algorithm based on a novel martingale characterization approach. Finally, we run a simulation study on a dynamic portfolio selection problem and illustrate the effectiveness of our algorithm.
Similar Papers
Mean--Variance Portfolio Selection by Continuous-Time Reinforcement Learning: Algorithms, Regret Analysis, and Empirical Study
Portfolio Management
Helps investors pick winning stocks automatically.
Optimistic Reinforcement Learning with Quantile Objectives
Machine Learning (CS)
Teaches computers to make safer, smarter choices.
Sample-efficient and Scalable Exploration in Continuous-Time RL
Machine Learning (CS)
Teaches robots to learn from mistakes faster.