ECMath SE7: Optimizing strategies in energy and storage markets

This research is carried out in the framework of Matheon supported by Einstein Foundation Berlin.

Innovation area Mathematics for Sustainable Energies
Project heads Vladimir Spokoiny, John Schoenmakers
Staff Roland Hildebrand
Duration From June 1st, 2014 until May 31st, 2017
Institutes Weierstrass Institute

ECmath Cooperation A-CH3, A-CH4, C-SE13
External Cooperation Denis Belomestny, Rüdiger Kiesel (Duisburg-Essen Univ.)
Industrial Cooperation Deloitte and Touch


The last decades have seen an expanding competition and complexity in the emerging energy markets, mainly due to the increasing deregulation and privatization of these markets. As a consequence, the dynamics of energy prices have become highly volatile and so energy traders are calling for new instruments and techniques in order to protect their exposures against the risks entailed by the energy markets. A key issue in the energy markets is the fact that energy (in particular oil and gas) can be stored physically. Such storage facilities allow for anticipating and exploiting market fluctuations of energy prices according to the principle “sell high” and “buy low”. Initially, storage facilities were only accessible by major players in the respective industries, who had sufficient capital to build and maintain them. Meanwhile, due to the emerging liberalization of the energy markets all participants have the possibility to trade storage services via storage exchange platforms. As a consequence on the one hand, design of optimal strategies involving buying, storing, and selling energy over time are called for. On the other hand, this resulted in a high demand of proper statistical prediction algorithms based on adequate statistical modeling of energy prices and storage markets. This whole development has further led to the appearance of structured contracts, also called energy derivatives. In the gas and energy markets for example, so called “swing options” have become very popular. An example swing option gives the holder the right to buy or sell a certain amount of gas, electricity or storage capacity at a certain prescribed number of trading dates. Energy derivatives are typically difficult to treat from a quantitative point of view due to their complex structured exercise features and their highly path-dependent structure. As a general consequence, the rising complexity of the markets poses a bunch of challenging mathematical issues that are to be resolved. These issues may be categorized into the following main streams:

  • New numerical methods and algorithms for solving multidimensional problems in optimal (multiple) stopping and optimal control
  • Adequate modeling of various energy price processes including modern statistical forecast techniques

Particularly in electricity markets, the evaluation of (swing) options requires efficient algorithms for multiple stopping and more complicated control optimization problems. Initially, the dual martingale approach was extended to general optimal control problems by [BSS10]. A first breakthrough was achieved in [Sch12] where a new dual martingale representation for multiple stopping problems was developed. This representation has been extended to much more general pay-off profiles in [BSZ15] (see also [BMS13]). In this extended setting it is possible to treat more realistic energy derivatives by the Monte Carlo method. Such derivatives may involve several volume and exercise refraction constraints, as well as processes driving several underlying energy titles. A novel simulation based primal approach for (standard) multiple stopping, hence the pricing of (simply structured) swing options was developed in [BS06]. A main general goal is the highly non-trivial extension of the above mentioned methods to problems of general control optimization.

Mathematical approach

We consider problems of optimal control of stochastic processes. Specifically, models of energy storage and production systems coupled with models of energy trading platforms and energy derivatives are considered.
We explore a novel solution method, namely that of penalized dual martingales. This method solves the dual problem, which is posed as finding a martingale acting as Lagrange multiplier of the non-anticipativity constraint of the primal control problem. Any feasible solution of the dual problem yields an upper bound on the objective value of the primal (maximization) problem. The infinite-dimensional dual problem is condensed into a finite-dimensional problem by restricting the set of martingales to a finite-dimensional subspace and solved by Monte-Carlo simulations. In order for the method to converge to an almost surely optimal martingale, i.e., for which the upper bound of the objective value has vanishing variance (see [SZH13]), a penalty term is added to the objective function.
As a first step, a theory of convex penalizations for the dual martingale approach has been developed on the simpler example of an optimal stopping problem, and its convergence properties have been explored (see [BHS15]). As a next step, a more complicated model of a system of coupled water reservoirs for electricity production is being considered. This model is coupled to a model of electricity prices on the spot market, involving day-ahead bids of the energy producers (see [LWM13]). For this model, a dual martingale approach is being developed which is elaborated enough to yield good upper bounds on the objective value yet simple enough to be solvable by the envisaged Monte-Carlo simulations.
In this project we consider a combined model of a hydro-electric storage and production facility and of the day-by-day electricity bid market, involving day-ahead bids of the energy producers. This coupled model has been introduced in [LWM13]. The production facility consists of a number of interconnected water reservoirs. Electricity can be produced from these reservoirs by releasing water from a reservoir situated at higher elevation to one at lower elevation. It can also be stored in the form of potential energy by pumping water from a reservoir at lower elevation to one at higher elevation. The energy conversion efficiency of these processes is lower than 100%, however, and running the water in a cycle leads to a certain loss. The reservoirs are filled by rain and other natural processes, which are modeled as a stochastic process. This process is also coupled to the stochastic process modeling the electricity price. When exploiting the reservoirs, the operator has to meet certain constraints such as not exceeding the capacity of each turbine and not emptying a reservoir completely. Currently the reservoirs are assumed to be ordered linearly, as depicted in the figure below.


A cooperation has been started with the project A-CH3 "Multiview geometry for ophthalmic surgery simulation". This project aims at identifying the location of 3D objects from image data of multiple cameras in real time. Specific semi-definite programming techniques have been discussed to reduce the influence of noise on the reconstruction of the object locations from measurement data.

The project A-CH4 of M. von Kleist, C. Hartmann and M. Weber is concerned with optimal control of a stochastic dynamical system governed by a chemical master equation, which is modeling the evolution of microbial strains which are resistant to different drugs. The dual martingale methods recently developed in [HSZ16] for the optimal control of hydroelectric storage and production systems are to be adapted to the dynamical systems treated in the project A-CH4.

The cooperation with project C-SE13 of M. Eigel, R. Henrion, D. Hoemberg, and R. Schneider has been initiated, consultations on the application of robust stochastic optimization methods to chance-constrained problems are under way.

Publications related to the project

[BS14] Christian Bayer and John Schoenmakers. Simulation of forward-reverse stochastic representations for conditional diffusions. Ann. Appl. Probab., 24(5):1994-2032 (2014).
[BJS14] Denis Belomestny, Mark Joshi, and John Schoenmakers. Addendum to: Multilevel dual approach for pricing American style derivatives. Finance Stoch., 19(3):681-684 (2015).
[BLS15] Denis Belomestny, Marcel Ladkau, and John Schoenmakers. Multilevel simulation based policy iteration for optimal stopping - convergence and complexity. SIAM/ASA J. Uncertain. Quantif., 3(1):460-483 (2015).
[BSZ15] Christian Bender, John Schoenmakers, and Jianing Zhang. Dual representations for general multiple stopping problems. Math. Financ., 25(2):339-370 (2015).
[GPSS15] Zorana Grbac, Antonis Papapantoleon, John Schoenmakers, and David Skovmand. Affine LIBOR models with multiple curves: theory, examples and calibration. SIAM J. Financial Math., 6(1):984-1025 (2015).
[H15] Roland Hildebrand. Spectrahedral cones generated by rank 1 matrices. J. Global Optim., 64(2):349-397 (2016).
[MS15] Grigori N. Milstein and John Schoenmakers. Uniform approximation of the Cox-Ingersoll-Ross process. Adv. Appl. Prob., 47(4):1132-1156 (2015).
[MS16] Grigori N. Milstein and John Schoenmakers. Uniform approximation of the CIR process via exact simulation at random times. To appear in Adv. in Appl. Prob., Preprint, 2016.
[HGS15] Roland Hildebrand, Michel Gevers, and Gabriel E. Solari. Closed-loop optimal experiment design: Solution via moment extension. IEEE T. Automat. Contr., 60(7):1731-1744 (2015).
[BHS16] Denis Belomestny, Roland Hildebrand, and John Schoenmakers. Optimal stopping via pathwise dual empirical maximisation. WIAS Preprint 2043, Invited revision under review at SIAM J. Control Optim., 2016.
[HSZ16] Roland Hildebrand, John Schoenmakers, and Jianing Zhang. Regression based duality approach to optimal control with application to hydro electricity storage. Working paper, 2016.


[BMS13] Sven Balder, Antje Mahayni, and John Schoenmakers. Primal dual linear Monte Carlo algorithm for multiple stopping - an application to exible caps. Quant. Finance 13(7):1003--1013 (2013).
[BS06] Christian Bender and John Schoenmakers. An iterative method for multiple stopping: convergence and stability. Adv. Appl. Probab. 38(3):729--749, (2006).
[BSZ15] Christian Bender, John Schoenmakers, and Jianing Zhang. Dual representations for general multiple stopping problems. Math. Fin. 25(2):339--370 (2015).
[BSS10] David B. Brown, J. Smith, and P. Sun. Information relaxations and duality in stochastic dynamic programs. Oper. Res. 58(4):785--801 (2010).
[DK94] M.H.A. Davis and I. Karatzas. A deterministic approach to optimal stopping. In: Probability, Statistics and Optimization: A Tribute to Peter Whittle (F. Kelly, ed.), Chapter 33, pp. 455--466. Wiley and Sons, Chichester, (1994).
[LWM13] N. Lohndorf, D. Wozabal, and S. Minner. Optimizing trading decisions for hydro storage systems using approximate dual dynamic programming. Oper. Res. 61(4):810--823 (2013).
[Sch12] John Schoenmakers. A pure martingale dual for multiple stopping. Finance Stoch. 16(2):319--334 (2012).
[SZH13] John Schoenmakers, Jianing Zhang, and Junbo Huang. Optimal dual martingales, their analysis, and application to new algorithms for Bermudan products. SIAM J. Financial Math. 4:86--116 (2013).