Graduate Seminar Financial and Actuarial Mathematics LMU and TUM

Reinforcement learning (RL) is a version of stochastic control in which the system dynamics are unknown (up to the type of dynamics such as Markov chains or diffusion processes). There has been an upsurge of interest in RL for (continuous-time) controlled diffusions in recent years. In this talk I will highlight the latest developments on theory and algorithms arising from this study, including entropy regularized exploratory formulation, policy evaluation, policy gradient, q-learning, and regret analysis. Time permitting, I will also discuss applications to mathematical finance and generative AI.

We discuss the limit of risk-based prices of European contingent claims in discrete-time financial markets under volatility uncertainty when the number of intermediate trading periods goes to infinity. The limiting dynamics are obtained using recently developed results for the construction of strongly continuous convex monotone semigroups. We connect the resulting dynamics to the semigroups associated to G-Brownian motion, showing in particular that the worst-case bounds always give rise to a larger bid-ask spread than the risk-based bounds. Moreover, the worst-case bounds are achieved as limit of the risk-based bounds as the agent’s risk aversion tends to infinity. The talk is based on joint work with Jonas Blessing and Alessandro Sgarabottolo.

We propose a continuous-time stochastic model to analyze the dynamics of impermanent loss in liquidity pools in decentralized finance (DeFi) protocols. We replicate the impermanent loss using option portfolios for the individual tokens. We estimate the risk-neutral joint distribution of the tokens by minimizing the Hansen–Jagannathan bound, which we then use for the valuation of options on relative prices and for the calculation of implied correlations. In our analyses, we investigate implied volatilities and implied correlations as possible drivers of the impermanent loss and show that they explain the cross-sectional returns of liquidity pools. We test our hypothesis on options data from a major centralized derivative exchange.

In this talk, we introduce a novel mesh-free and direct method for computing the shape derivative in PDE-constrained shape optimization problems. Our approach is based on a probabilistic representation of the shape derivative and is applicable for second-order semilinear elliptic PDEs with Dirichlet boundary conditions and a general class of target functions. The probabilistic representation derives from a boundary sensitivity result for diffusion processes due to Costantini, Gobet and El Karoui. Via so-called Taylor tests we verify the numerical accuracy of our methodology.

The theory we present aims at expanding the classical Arbitrage Pricing Theory to a setting where N agents invest in stochastic security markets while also engaging in zero-sum risk exchange mechanisms. We introduce in this setting the notions of Collective Arbitrage and of Collective Super-replication and accordingly establish versions of the fundamental theorem of asset pricing and of the pricing-hedging duality. When computing the Collective Super-replication price for a given vector of contingent claims, one for each agent in the system, allowing additional exchanges among the agents reduces the overall cost compared to classical individual super-replication. The positive difference between the aggregation (sum) of individual superhedging prices and the Collective Super-replication price represents the value of cooperation. Finally, we explain how these collective features can be associated with a broader class of risk measurement or cost assessment procedures beyond the superhedging framework. This leads to the notion of Collective Risk Measures, which generalize the idea of risk sharing and inf-convolution of risk measures.

Abstract We consider the fundamental problem of Linear Quadratic Gaussian Control on an infinite horizon with costly information acquisition. Specifically, we consider a two-dimensional coupled system, where one of the two states is observable, and the other is not. Additionally, to inference from the observable state, costly information is available via an additional, controlled observation process. Mathematically, the Kalman-Bucy filter is used to Markovianize the problem. Using an ansatz, the problem is then reduced to one of the control-dependent, conditional variance for which we show regularity of the value function. Using this regularity for the reduced problem together with the ansatz to solve the problem by dynamic programming and verification and construct the unique optimal control. We analyze the optimal control, the optimally controlled state and the value function and compare various properties to the literature of problems with costly information acquisition. Further, we show existence and uniqueness of an equilibrium for the controlled, conditional variance, and study sensitivity of the control problem at the equilibrium. At last, we compare the problem to the case of no costly information acquisition and fully observable states. Joint work with Christoph Knochenhauer and Yufei Zhang (Imperial College London).

The aim of the present talk is to discuss HJM cross currency models that can serve as the basis for the simulation of exposure profiles in the xVA context. Such models need to take into account the asymmetries that arise in the different currency denominations in view of the benchmark reform: for example, while in the EUR area Euribor is still the dominant interest rate benchmark, the situation in the US is much more complex due to the introduction of SOFR and alternative forward looking unsecured rates such as the Bloomberg BSBY or the Ameribor 90T. The impact of the Libor transition on the structure of cross currency swap is also an aspect we would like to address. In summary we would like to: - Provide, in a HJM setting, a unified treatment of forward looking and backward looking rates with and without a credit/liquidity component, i.e. consider a HJM setting for a general underlying index in each currency area. - Properly link such general single currency HJM models by means of cross currency processes that capture the cross currency basis. - Analyze cross currency swaps with arbitrary combinations of interest rate indexes and collateral rates in the different currency areas i.e. with and without Libor discontinuation. This is a joint work with Silvia Lavagnini (BI Oslo)

The goal of this work is to disentangle the roles of volume and participation rate in the price response of the market to a sequence of orders. To do so, we use an approach where price dynamics are derived from the order flow via no arbitrage constraints. We also introduce in the model sophisticated market participants having superior abilities to analyse market dynamics. Our results lead to two square root laws of market impact, with respect to executed volume and with respect to participation rate. This is joint work with Bruno Durin and Grégoire Szymanski.

Motivated by the tradeoff between exploitation and exploration in reinforcement learning, we study a continuous-time entropy-regularized mean variance portfolio selection problem in the presence of jumps. A first key step is to derive a suitable formulation of the continuous-time problem. In the existing literature for the diffusion case (e.g., Wang, Zariphopoulou and Zhou, Mach. Learn. Res. 2020), the conditional mean and the conditional covariance of the controlled dynamics are heuristically derived by a law of large numbers argument. In order to capture the influence of jumps, we first explicitly model distributional controls on discrete-time partitions and identify a family of discrete-time integrators which incorporate the additional exploration noise. Refining the time grid, we prove convergence in distribution of the discrete-time integrators to a multi-dimensional Levy process. This limit theorem gives rise to a natural continuous-time formulation of the exploratory control problem with entropy regularization. We solve this problem by adapting the classical Hamilton-Jacobi-Bellman approach. It turns out that the optimal feedback control distribution is Gaussian and that the optimal portfolio wealth process follows a linear stochastic differential equation, whose coefficients can be explicitly expressed in terms of the solution of a nonlinear partial integro-differential equation. We also provide a detailed comparison to the results derived by Wang and Zhou (Math. Finance, 2020) for the exploratory portfolio selection problem in the Black-Scholes model. The talk is based on joint work with Thuan Nguyen (Saarbrücken).