WIAS Preprint No. 3077, (2023)

Approximating Langevin Monte Carlo with ResNet-like neural network architectures


Authors

  • Eigel, Martin
    ORCID: 0000-0003-2687-4497
  • Miranda, Charles
  • Schütte, Janina
    ORCID: 0009-0000-9924-3229
  • Sommer, David
    ORCID: 0000-0002-6797-8009

2020 Mathematics Subject Classification

  • 62F15 65N75 65C30 60H35 62H12 65C05 60H35 68T07

DOI

10.20347/WIAS.PREPRINT.3077

Abstract

We sample from a given target distribution by constructing a neural network which maps samples from a simple reference, e.g. the standard normal distribution, to samples from the target. To that end, we propose using a neural network architecture inspired by the Langevin Monte Carlo (LMC) algorithm. Based on LMC perturbation results, we show approximation rates of the proposed architecture for smooth, log-concave target distributions measured in the Wasserstein-2 distance. The analysis heavily relies on the notion of sub-Gaussianity of the intermediate measures of the perturbed LMC process. In particular, we derive bounds on the growth of the intermediate variance proxies under different assumptions on the perturbations. Moreover, we propose an architecture similar to deep residual neural networks and derive expressivity results for approximating the sample to target distribution map.

Download Documents