Probabilistic sampling for physics: finding needles in a field of high-dimensional haystacks

4 sept. 2023, 09:00 → 22 sept. 2023, 17:00 Europe/Paris

Institut Pascal

Manon Michel (Laboratoire de Mathématiques Blaise Pascal, CNRS, Université Clermont-Auvergne), Thomas Swinburne (Centre Interdisciplinaire de Nanoscience de Marseille, CNRS, Aix-Marseille Université)

This 3-week meeting aims at gathering physicists and mathematicians working on stochastic sampling. The goal is to foster new interdisciplinary collaborations to address challenges encountered while sampling probability distributions presenting high-energy barriers and/or a large number of metastable states. 

The challenges of metastability / slow mixing are faced by researchers working in a diverse range of fields in science, from fundamental problems in statistical mechanics to large-scale simulations of materials; in recent years significant progress has been made on several aspects, including :

• design of Markov processes which break free from a random-walk behavior
• numerical methods for dimensionality reduction of input data
• mode-hopping non-local moves by generative models,
• density region detection,
• analytical derivations of robustness guarantees.

By bringing together established experts and early-career researchers across a wide range of disciplines we aim to find new synergies to answer important and general questions, including:

• How to associate fastest local mixing strategies with mode-hopping moves?
• What physics can we preserve and extract along the dynamics?
• How to build reduced representations that can detect previously unseen events or regions
  
  A focus will be made on implementing and comparing algorithms via benchmark datasets and models. 
  
  Week 1: Plenary talks covering a diverse range of open problems in different theoretical and numerical aspects of stochastic sampling. Flash talks from all participants. Brainstorming sessions.
  Week 2: Presentations from leading and early-career researchers. Introduction of benchmark problems and implementation in Python code.
  Week 3: Presentations from leading and early-career researchers. Collaboration and discussion with benchmark problems.

The three week program will feature mornings of plenary and research talks, with afternoons largely allocated for discussion and collaboration. 
To promote long-term and interdisciplinary collaborations, a focus of the afternoon sessions will be both the design and, importantly, implementation of benchmark problems for stochastic sampling. This will allow the relative strengths of available methods to be assessed in detail, and provide a concrete framework for collaborations. The final week will focus on using these benchmark problems to guide discussion on combining advances from different fields. The achieved work (original results and review articles) will be featured as a publication in Cahiers de l'Institut Pascal, published by EDP Sciences.

Colloquia, open to a broader general public, will be given by:

• Stéphane Mallat, ENS Paris (Wednesday Sept, 6th - 10 AM)
• Freddy Bouchet, ENS Paris (Thursday Sept, 14th - 3.30 PM)
• Alberto Rosso, Uni. Paris-Saclay (Wednesday Sept, 20th - 3.30 PM)

 

Plenary talks will be given by:

• David Aristoff, Colorado State
• Mihai-Cosmin Marinica, CEA
• Alain Durmus, École Polytechnique
• Virginie Ehrlacher, Ecole des Ponts ParisTech
• Marylou Gabrie, École Polytechnique
• Tim Garoni, Monash Uni.
• Pierre Jacob, ESSEC
• Tony Lelievre, Ecole des Ponts ParisTech
• Manon Michel, CNRS - Uni. Clermont-Auvergne
• Pierre Monmarché, Sorbonne Université
• Feliks Nüske, Max Planck Institute DCTS Magdeburg
• Misaki Ozawa, CNRS & Univ. Grenoble Alpes
• Danny Perez, Los Alamos National Laboratory
• Grant Rotskoff, Stanford University
• Beatriz Seoane, LISN, Univ. Paris-Saclay
• Gabriel Stoltz, Ecole des Ponts ParisTech
• Thomas Swinburne, CNRS, Uni. Aix-Marseille
• Marija Vucelja, University of Virginia
• Martin Weigel, Chemnitz University

 

Research talks will be given by:

• Michael Albergo, New York Uni.
• Haochuan Chen, Uni. de Lorraine
• Aurélien Decelle, Complutense University of Madrid
• Jorge Fernandez de Cossio Diaz, ENS Paris
• Federico Ghimenti, Uni. de Paris Cité
• Tristan Guyon, CNRS, Uni. Clermont-Auvergne
• Léon Huet, Sorbonne Uni.
• Eiji Kawasaki, CEA
• Mouad Ramil, Seoul Uni.
• Vincent Souveton, CNRS, Uni. Clermont-Auvergne

 

 

We look forward to seeing you in Paris!          
Manon Michel, CNRS & Univ. Clermont Auvergne          
Thomas Swinburne, CNRS & Univ. Aix-Marseille          
 

Scientific committee: Alain Durmus (École Polytechnique), Virginie Ehrlacher (École des Ponts), Guilhem Lavaux (IAP), Mihai-Cosmin Marinica (CEA), Martin Weigel (Chemnitz Uni)

lundi 4 septembre

General address: Welcome Week 1 - Amphitheatre

1 Welcome address

Welcome.pdf

2 Flashtalks - Amphitheatre

11:00 Coffee/Tea break Cathedrale

Challenge and Perspective: Tony Lelièvre - Amphitheatre

3 Finding saddle points of energy landscapes: why and how?

One way to bridge the scale between full atomistic models and more coarse-grained descriptions is to use Markov State models parameterized by the Eyring Kramers formulas. These formulas give the hopping rates between local minima of the potential energy function. They require to identify the local minima and saddle points of the potential energy function. This approach is for example used in materials science (kinetic Monte Carlo models, accelerated dynamics à la A.F. Voter and D. Perez).

In this talk, I will first present a recent result obtained in collaboration with D. Le Peutrec (Université d'Orléans) and B. Nectoux (Université Clermont Auvergne) about the mathematical foundations of this approach, by deriving these Eyring-Kramers exit rates starting from the overdamped Langevin dynamics [1]. I will then introduce a recent algorithm we proposed together with P. Parpas (Imperial College London) in order to locate saddle points [2]. I will explain why these two works both rely on concentration properties of the eigenvectors of Witten Laplacians, in the small temperature regime.

References:
[1] TL, D. Le Peutrec and B. Nectoux, Eyring-Kramers exit rates for the overdamped Langevin dynamics: the case with saddle points on the boundary, https://arxiv.org/abs/2207.09284.
[2] TL, P. Parpas /Using Witten Laplacians to locate index-1 saddle points/, https://arxiv.org/abs/2212.10135.

Orateur: Tony LELIEVRE (Ecole des Ponts ParisTech)

12:30 Lunch - Buffet at IPa

Challenge and Perspective: Pierre Monmarché - Amphitheatre

4 Exploration with saddle point search

According to Large Deviation Principles, in a metastable regime, a local explorer is likely to exit from local modes through energy saddle points. Looking alternatively for minimizers or saddle points thus appears as a reasonable direction in order to find unknown modes in a high-dimensional non-convex landscape. The goal of this talk is to introduce the useful tools for that purpose, in particular saddle-point local search algorithms, free energy estimation, and switched diffusion processes.

Orateur: Pierre Monmarche

15:00 Coffee/Tea break Cathedrale

Challenge and Perspective: Thomas Swinburne - Amphitheatre

5 Predicting the atomic kinetics of materials with uncertainty

I will discuss two methods to coarse-grain and predict atomic kinetics generated by molecular dynamics, with application to diffusion and plasticity in metals. When the energy landscape is metastable, atomic kinetics can be mapped to a discrete Markov chain with robust Bayesian bounds on unseen transitions. These bounds are used to allocate resources in massively parallel computation and measure the convergence of coarse-grained properties such as transport coefficients.
However, in many important cases kinetics are driven and show no clear signature of metastabilty. I show how many-body basis functions used in atomic machine learning form a medium dimensional, metric latent space ideal for coarse-graining and analysis. A vector autoregressive model can resample and forecast latent space trajectories using a Mahalanobis distance to qualify forecast uncertainty. Finally, the descriptor manifold is analyzed in the context of the yielding transition.

Orateur: Thomas Swinburne (Centre Interdisciplinaire de Nanoscience de Marseille, CNRS, Aix-Marseille Université)

Brainstorming: Session 1 - Amphitheatre

6 Metastability

Orateur: Thomas Swinburne (Centre Interdisciplinaire de Nanoscience de Marseille, CNRS, Aix-Marseille Université)

mardi 5 septembre

Challenge and Perspective: Martin Weigel - Amphitheatre

7 Population Monte Carlo as a meta-algorithm

As a meta-algorithm, population annealing can be combined with a wide range of simulation methods, including Monte Carlo and molecular dynamics. In the past, we have analyzed the approach regarding the scaling of statistical and systematic errors, proposed improvements and implemented the method on highly-efficient graphics processing units. In the present talk I will discuss recent developments regarding the optimal choice of the resampling protocol, the combination of several population annealing simulations using weighted averages, applications relating to quantum Monte Carlo simulations, and the combination of population annealing with generalized ensembles such as for microcanonical and multicanonical simulations.

Orateur: Martin Weigel (Chemnitz University of Technology, Chemnitz, Germany)

11:00 Coffee/Tea break - Cathedrale

Challenge and Perspective: Misaki Ozawa - Amphitheatre

8 Sampling glassy disordered systems

In this talk, I will introduce sampling issues in glassy disordered systems, particularly glass-forming liquids, which consist of a long-standing problem in condensed matter physics. I will explain why this is important and difficult, and I will review various previous attempts.

Orateur: Misaki Ozawa (CNRS, Univ. Grenoble Alpes, France)

12:30 Lunch - IPa Canteen

Challenge and Perspective: Manon Michel - Amphitheatre

9 Upgrading MCMC dynamics by necessary symmetries and non-reversibility

In this talk, I will review the older and most recent developments regarding reversibility breaking in Markov-chain Monte Carlo (MCMC), from lifting to piecewise deterministic Markov processes. This will offer the opportunity to discuss the differences between necessary and sufficient symmetries for correctness in MCMC and how removing restrictive conditions can lead to more efficient algorithms. The discussed algorithms will be illustrated in statistical mechanics notorious examples, from particle to spin systems.

Orateur: Manon MICHEL (LMBP)

15:00 Coffee/Tea break - Cathedrale

Brainstorming: Session 2 - Amphitheatre

10 Beyond random-walk behavior

Orateur: Manon MICHEL (LMBP)

16:30 Social drinks at IPa - Terrace

mercredi 6 septembre

Colloquium: Stéphane Mallat - Amphitheatre

11 Multiscale Interaction Models of Physical Energies and Deep Networks - Stéphane Mallat (ENS Paris)

Building probabilistic models and sampling complex physical fields is an outstanding issue, despite remarkable numerical results of deep networks. Relying on the renormalization group approach, we show that the curse of dimensionality can be avoided by separating variabilites at multiple scales in wavelet bases. The main difficulty is to model interactions across scales. We show that these interactions are often spatailly local, with a probability distribution which is nearly log concave. They can thus be estimated and sampled with faster score matching algorithms. Models of multiscale interactions are introduced for cosmological and turbulence fields. Relations with deep neural networks are explained.

11:00 Coffee/Tea break - Cathedrale

Challenge and Perspective: Grant Rotskoff - Amphitheatre

12 Sampling with generative models

In probability theory, the notion of "weak convergence" is often used to describe two equivalent probability distributions. This metric requires equivalence of the average value of well-behaved functions under the two probability distributions being compared. In coarse-grained modeling, Noid and Voth developed a thermodynamic equivalence principle that has a similar requirement. Nevertheless, there are many functions of the fine-grained system that we simply cannot evaluate on the coarse-grained degrees of freedom. In this talk, I will describe an approach that combines accelerated sampling of a coarse-grained model with invertible neural networks to invert a coarse-graining map in a statistically precise fashion. I will show that for non-trivial biomolecular systems, we can quantitatively recover the fine-grained observables from coarse-grained sampling. Finally, I will discuss the general framework of using auxiliary models for mode discover when sampling with generative models.

Orateur: Grant Rotskoff (Stanford University)

12:30 Lunch - IPa Canteen

Challenge and Perspective: Beatriz Seoane - Amphitheatre

13 The effect of Monte Carlo Sampling in the training of Energy Based Models

Energy-based models (EBMs) are powerful generative machine learning models that are able to encode the complex distribution of a dataset in the Gibbs-Boltzmann distribution of a model energy function. This means that, if properly trained, they can be used to synthesize new patterns that resemble those of the data set as closely as possible, but also that this energy function can be used to "learn" something about the building mechanisms of the dataset under study. Indeed, EBMs can be considered a powerful modeling tool for arbitrary data if one were able to map complex energy functions defined in a neural network into spin-interaction Hamiltonians that can be studied using standard tools of statistical physics. Such an approach has long been used in physics for inverse Ising problems. The goal now is to extend this approach to more complex energy functions that can encode all higher order correlations in complex data. While this program is very encouraging, training good EBMs is particularly challenging, mainly because they rely on long Monte Carlo sampling processes to estimate the log-likelihood gradient. These sampling processes must be repeated once and over again each time parameters are updated, making the possibility of convergence to equilibrium at each step extremely difficult. In my talk, I will discuss how non-equilibrium effects can be exploited to train fast generators capable of accurately reproducing multimodal distributions of data, but at the cost of losing the connection between the Boltzmann distribution and the empirical distribution. I will also discuss optimized sampling algorithms or approximation schemes that aim to favor MCMC convergence throughout the learning process, as well as the possibility of using population annealing methods to mitigate the apperance of out-of-equilibrium effects in the trained models.

Orateur: Beatriz Seoane Bartolomé (LISN)

15:00 Coffee/Tea break - Cathedrale

Challenge and Perspective: Marylou Gabrié - Amphitheatre

14 Opportunities and challenges in enhancing sampling with generative models

Deep generative models parametrize very flexible families of distributions able to fit complicated datasets of images or text. These models provide independent samples from complex high-distributions at negligible costs. On the other hand, sampling exactly a target distribution, such a Bayesian posterior, is typically challenging: either because of dimensionality, multi-modality, ill-conditioning or a combination of the previous. In this talk, I will discuss recent works trying to enhance traditional inference and sampling algorithms with learning.

Orateur: Marylou Gabrié (École Polytechnique)

Brainstorming: Session 3 - Amphitheatre

15 Learning and Sampling

Orateur: Grant Rotskoff (Stanford University)

jeudi 7 septembre

Challenge and Perspective: Feliks Nüske - Amphitheatre

16 Dimensionality Reduction and Metastability Analysis using the Koopman Operator

The Koopman Operator presents a powerful framework for dimensionality reduction of (stochastic) dynamical systems. In addition, metastable sets and their rates of transition can be obtained by analysing its spectrum. In this talk, we first recap the basics of Koopman methods, and then move on to discuss recent advances and current challenges.

Orateur: Feliks Nüske (Max Planck Institute DCTS Magdeburg)

11:00 Coffee/Tea break - Cathedrale

Result Communication: - Amphitheatre

17 Stochastic models form enhanced sampling in Chemistry

The study of chemical reactions involving covalent links and breaks inherently presents stochastic complexities. To overcome the challenges posed by large energy barriers, researchers commonly use enhanced sampling techniques like transition path sampling and umbrella sampling to collect relevant data at the DFT level. However, the data acquired through these processes requires careful treatment to accurately infer any stochastic model. For transition path sampling, a realistic representation of the abort transitions is lacking, and for umbrella sampling, estimating the effect of external constant forces on the memory kernel is extremely challenging.

Within our team, we are developing innovative tools to address these issues. These tools aim to extract the true behavior of the reaction from the DFT data. The treatment of biases will then be employed to fit a quasi-Markovian Generalized Langevin Equation (GLE) model, enabling the extraction of meaningful insights beyond conventional DFT timescales.

Orateur: Léon Huet (IMPMC - Sorbonne Université)

12:30 Lunch - IPa Canteen

Challenge and Perspective: Virginie Ehrlacher - Amphitheatre

18 Sparse approximation of the Lieb functional in DFT with moment constraints

(joint work with Luca Nenna)
In this talk, we will present recent mathematical results about the Lieb functional in Density Functional Theory. More precisely, the Lieb functional, for a given electronic density, can be viewed as a generalized form of optimal transport problem for which the electronic density plays the role of a marginal. A numerical discretization of this problem can be obtained by imposing a finite number of moment constraints instead of this full marginal constraints. Using the so-called Tchakhaloff's theorem, it can be shown that a minimizer of the approximate Lieb problem can be obtained as a finite-rank projector, the rank of which is at most equal to the number of moment constraints of the problem. In other words, such minimizer has a very sparse structure which can be exploited for numerics. We will highlight some open questions related to the practical resolution of such problems, which require the use of advanced methods for manifold-constrained Monte-Carlo sampling.

Orateur: Virginie EHRLACHER (Ecole des Ponts ParisTech & INRIA)

15:00 Coffee/Tea break - Cathedrale

Challenge and Perspective: Alain Durmus - Amphitheatre

19 Barrier Hamiltonian Monte Carlo

This talk introduces Barrier Hamiltonian Monte Carlo (BHMC), a version of HMC which aims at sampling from a Gibbs distribution π on a manifold M, endowed with a Hessian metric g derived from a self-concordant barrier. Like Riemannian Manifold HMC, our method relies on Hamiltonian dynamics which comprise g. It incorporates the constraints defining M and is therefore able to exploit its underlying geometry. We first introduce c-BHMC (continuous BHMC), for which we assume that the Hamiltonian dynamics can be integrated exactly, and show that it generates a Markov chain for which π is invariant. Secondly, we design n-BHMC (numerical BHMC), a Metropolis-Hastings algorithm which combines an acceptance filter including a "reverse integration check" and numerical integrators of the Hamiltonian dynamics. Our main results establish that n-BHMC generates a reversible Markov chain with respect to π. This is in contrast to existing algorithms which extend the HMC method to Riemannian manifolds, as they do not deal with asymptotic bias. Our conclusions are supported by numerical experiments where we consider target distributions defined on polytopes.

Orateur: Alain Durmus (Ecole Polytechnique)

Brainstorming: Session 4 - Amphitheatre

20 Preserving physics in dynamics

Orateur: Pierre Monmarche

19:00 Social dinner

vendredi 8 septembre

Result Communication: - Amphitheatre

21 Discover committor-consistent path by neural networks

A significant challenge faced by atomistic simulations is the difficulty, and often impossibility, to sample the transitions between metastable states of the free-energy landscape associated to slow molecular processes. Importance-sampling schemes represent an appealing option to accelerate the underlying dynamics by smoothing out the relevant free-energy barriers, but require the definition of a suitable reaction-coordinate (RC) models expressed in terms of compact low-dimensional sets of collective variables (CVs). While most computational studies of slow molecular processes have traditionally relied on educated guesses based on human intuition to reduce the dimensionality of the problem at hand, a variety of machine-learning (ML) algorithms have recently emerged as powerful alternatives to discover a meaningful CV capable of capturing the dynamics of the slowest degrees of freedom. Considering a simple paradigmatic situation in which the long-time dynamics is dominated by the transition between two known metastable states, we compare two variational data-driven ML methods based on Siamese neural networks aimed at discovering a meaningful RC model---the slowest decorrelating CV of the molecular process, and the committor probability to first reach one of the two metastable states. One method is the state-free reversible variational approach for Markov processes networks (VAMPnets), or SRVs---the other, inspired by the transition path theory framework, is the variational committor-based neural networks, or VCNs. The relationship and the ability of these methodologies to discover the relevant descriptors of the slow molecular process of interest is illustrated with a series of simple model systems. We also show that both strategies are amenable to importance-sampling schemes through an appropriate reweighting algorithm that approximates the kinetic properties of the transition.

Orateur: Haochuan Chen (University of Lorraine)

22 Stochastic Interpolants: A unifying framework for flows and diffusions

I will discuss recent work on unifying flow-based and diffusion based methods through a
generative modeling paradigm we call stochastic interpolants. These models enable the use of a broad class of continuous-time stochastic processes called `stochastic interpolants' to bridge any two arbitrary probability density functions exactly in finite time. These interpolants are built by combining data from the two prescribed densities with an additional latent variable that shapes the bridge in a flexible way. The time-dependent probability density function of the stochastic interpolant is shown to satisfy a first-order transport equation as well as a family of forward and backward Fokker-Planck equations with tunable diffusion. Upon consideration of the time evolution of an individual sample, this viewpoint immediately leads to both deterministic and stochastic generative models based on probability flow equations or stochastic differential equations with an adjustable level of noise. The drift coefficients entering these models are time-dependent velocity fields characterized as the unique minimizers of simple quadratic objective functions, one of which is a new objective for the score of the interpolant density. Remarkably, we show that minimization of these quadratic objectives leads to control of the likelihood for any of our generative models built upon stochastic dynamics. By contrast, we establish that generative models based upon a deterministic dynamics must, in addition, control the Fisher divergence between the target and the model. We also construct estimators for the likelihood and the cross-entropy of interpolant-based generative models, discuss connections with other stochastic bridges, and demonstrate that such models recover the Schr\"odinger bridge between the two target densities when explicitly optimizing over the interpolant.

Orateur: Michael Albergo (New York University)

11:00 Coffee/Tea break - Cathedrale

Result Communication: - Amphitheatre

23 Beyond translational flows in non-reversible sampling

Breaking reversibility in Monte Carlo algorithms often leads to substantial accelerations in sampling complex systems. Event-Chain Monte Carlo (ECMC) has allowed to investigate the bidimensional hard-sphere phase transition, building on non-reversible continuous translational moves. However, more general systems require rotations of some sort to thermalize.
In this work, we build on the Piecewise Deterministic Markov Process framework to introduce more general deterministic flows in ECMC, along with explicit conditions for global balance. Purely non-reversible rotational flows are then applied to hard dimers and compared to the alternative of mixing translational ECMC with reversible Metropolis rotations.

Orateur: Tristan Guyon (Laboratoire de Mathématiques Blaise Pascal, Université Clermont-Auvergne)

24 Sampling efficiency of transverse forces in dense liquids

Sampling the Boltzmann distribution using forces that violate detailed balance can be faster than with the equilibrium evolution, but the acceleration depends on the nature of the nonequilibrium drive and the physical situation. Here, we study the efficiency of forces transverse to energy gradients in dense liquids through a combination of techniques: Brownian dynamics simulations, exact infinite-dimensional calculation and a mode-coupling approximation. We find that the sampling speedup varies non-monotonically with temperature, and decreases as the system becomes more glassy. We characterize the interplay between the distance to equilibrium and the efficiency of transverse forces by means of odd transport coefficients.

Orateur: Federico Ghimenti (Université Paris Cité)

25 Parallel Replica algorithm for Langevin dynamics and Adaptative Metadynamics

This talk will be divided into two independent parts. The first part shall focus on the extension of the formalization of an algorithm (Parallel Replica) to the case of the Langevin dynamics. Parallel Replica is used in material science to sample rare-events and consists in a parallelization in time of the sampling. It can be formalized using the notion of quasi-stationary distributions which can be seen as the local equilibrium reached by a dynamics when it remains trapped in a state for a long time. Due to the degeneracy of the Langevin dynamics, its existence was an open question until recently. In this talk we will provide an explicit control of the density of the Langevin process which allows us to obtain the existence of a QSD using well-known spectral results.
The second part of this talk shall focus on the sampling of the minimum energy path (MEP) between metastable states in a case where the MEP is locally orthogonal to the dynamics. In order to do that we will introduce Adaptative Metadynamics method with an augmented loss which takes into account the initial, final configuration as well as a penalization term for high energetic configurations.

Orateur: Mouad Ramil

13:00 Lunch - IPa Canteen

General address: Conclusion Week 1 - Cathedrale

lundi 11 septembre

General address: Welcome Week 2 - Amphitheatre

Challenge and Perspective: Gabriel Stoltz - Amphitheatre

26 Optimizing the diffusion for sampling with overdamped Langevin dynamics Gabriel Stoltz (CERMICS ,Ecole des Ponts and Matherials, Inria Paris)

Overdamped Langevin dynamics are stochastic differential equations, where gradient dynamics are perturbed by noise in order to sample high dimensional probability measures such as the ones appearing in computational statistical physics and Bayesian inference. By varying the diffusion coefficient, there are in fact infinitely many overdamped Langevin dynamics which preserve the target probability measure at hand. This suggests to optimize the diffusion coefficient in order to increase the convergence rate of the dynamics, as measured by the spectral gap of the generator associated with the stochastic differential equation. We analytically study this problem here, obtaining in particular necessary conditions on the optimal diffusion coefficient. We also derive an explicit expression of the optimal diffusion in some homogenized limit. Numerical results, both on discretizations of the spectral gap problem and Monte Carlo simulations of the stochastic dynamics, demonstrate the increased quality of the sampling arising from an appropriate choice of the diffusion coefficient.

Orateur: Gabriel STOLTZ (Ecole des Ponts & Inria Paris)

11:00 Coffee/Tea break - Cathedrale

Challenge and Perspective: Danny Perez - Amphitheatre

27 Dynamically-accurate biasing using Accelerated Molecular Dynamics

Modifying or biasing the dynamics of atomistic systems can result in faster mixing and convergence of thermodynamic observables, but it generally yields non-physical kinetics. I will introduce a family of so called "Accelerated Molecular Dynamics" methods that are specifically designed to produce statistically accurate "state-to-state" dynamics for metastable systems at a much reduced computational cost compared to brute force molecular dynamics. I will discuss the basics of the methodology, as well as implementation on parallel computers.

Orateur: Dr Danny Perez (Los Alamos National Laboratory)

12:30 Lunch - Buffet at IPa

Challenge and Perspective: Mihai Cosmin Marinica - Amphitheatre

28 Sampling Complex Energy Landscapes in Material Science Using Data-Driven Force Fields - Mihai Cosmin Marinica

A. Zhong (1), C. Lapointe (1), A.M. Goryaeva (1), J. Wrobel (3), T. D. Swinburne (2),
A. Allera (1), M. Athènes (1), M.-C. Marinica (1)

(1)DES - Service de Recherches de Métallurgie Physique, CEA, Université Paris-Saclay, F-91191 Gif-sur-Yvette, France
(2)CNRS, Centre Interdisciplinaire de Nanoscience de Marseille (CINaM), Université Aix-Marseille, France
(3) Faculty of Materials Science and Engineering, University of Technology, Wołoska 141, 02-507
Warsaw, Poland

Statistical methods, such as Machine Learning (ML) trained on physical data, can be invaluable when traditional approaches are limited or when their direct application is hampered by challenges like high computational costs. In materials science the interaction and transformation of crystal defects networks give rise to an extraordinarily diverse range of defect morphologies [1]. Additionally, accounting for chemical disorder presents another layer of complexity that is often overlooked in the emerging field of machine learning approaches. By utilizing the recently open-sourced MiLaDy (Machine Learning Dynamics) package [2] combined with accelerated Molecular Dynamics based on the Bayesian adaptive biasing force method [2], we aim to sample the intricate energy landscapes of defects by: (i) using methods that can identify complex networks of minima and saddle points at zero K; (ii) offering reliable force fields that handle intricate defects such as interstitials and dislocation loops; (iii) probing the atomistic free energy landscape of metals with ab initio accuracy up to the melting temperature [3]; (iv) examining chemical disorder in high entropy alloys (HEA); and, finally, (v) proposing surrogate models that sidestep traditional approaches to access challenging properties, like vibrational entropies [4].

[1] A. M. Goryaeva et al. Nature Commun. 14, 3003 (2023); A. M. Goryaeva et al. Nature Commun. 11, 4691 (2020);
[2] M.-C. Marinica, A. M. Goryaeva, T. D. Swinburne et al, MiLaDy - Machine Learning Dynamics, CEA Saclay, 2015-2023: https://ai-atoms.github.io/milady/ ;
[3] A. Zhong, C. Lapointe, A. M. Goryaeva, J. Baima, M. Athènes, and M.-C. Marinica,, Phys. Rev. Mater. 7, 023802 (2023); C. Lapointe et al. (to be submitted);
[4] C. Lapointe, et al. , Phys. Rev. Materials 4, 063802 (2020); C. Lapointe, et al, Phys. Rev. Materials 6, 113803 (2022).

Brainstorming: Session 5 - Amphitheatre

29 Enhanced sampling

Orateur: Gabriel STOLTZ (Ecole des Ponts & Inria Paris)

16:00 Coffee/Tea break - Cathedrale

General address: Hackathon preparation and work organisation - Amphitheatre

mardi 12 septembre

Hackathon / Free discussion - L107, L203, L217, Cathedrale, Terrace

Challenge and Perspective: Tim Garoni - Amphitheatre

30 Limiting distributions of Glauber coupling times

Coupling from the past is a method for obtaining perfect samples from Markov chain Monte Carlo algorithms. The price paid is that the running time becomes random. We will present some recent results concerning the limit behaviour of this random time, and discuss a number of open conjectures.

Orateur: Tim Garoni (Monash University)

11:00 Coffee/Tea break - Cathedrale

Challenge and Perspective: Pierre Jacob - Amphitheatre

31 Coupling methods to de-bias and assess MCMC algorithms

How to parallelize computation and how to diagnose convergence remain largely open questions regarding MCMC. Since Glynn & Rhee (Journal of Applied Probability, Vol. 51A, 2014), various advances based on couplings of MCMC algorithms have been proposed. The key is the design of coupled chains that, if properly constructed, can be employed to construct estimators that do not suffer from the usual "burn-in" or initialization bias. These methods are distinct from coupling from the past, and appear much more widely applicable, but do not yield perfect samples. This presentation provides an overview of this family of techniques and recent developments.

Orateur: Pierre JACOB (ESSEC Business School)

12:30 Lunch - IPa Canteen

Challenge and Perspective: Frédéric Cazals - Amphithéâtre

32 Sampling protein structure

Setting aside the problem of designing force fields, sampling protein conformations to estimate their thermodynamic and kinetic properties remains a challenge. In this talk, I will review recent work on two connected problems in this realm.

The first one is the calculation of high dimensional volumes of polytopes, using random walks (hit-and-run, HMC, PDMP).

The second one is the sampling of protein conformations in internal coordinates, using the same kind of random walks.

Orateur: Frederic Cazals (Inria)

15:00 Coffee break

Brainstorming: Session 6 - Amphitheatre

33 Mixing and convergence

Orateur: Tim Garoni (Monash University)

Feedback: Hackathon Launch - Amphitheatre

17:00 Social drinks at IPa - Terrace

mercredi 13 septembre

Hackathon / Free discussion - L107, L203, L217, Cathedrale, Terrace

Result Communication: Marija Vulceja - Amphitheatre

34 Anomalous thermal relaxations of physical systems

Rapid cooling or heating of a physical system can lead to unusual thermal relaxation phenomena. A prime example of anomalous thermal relaxation is the Mpemba effect. The phenomenon occurs when a system prepared at a hot temperature overtakes an identical system prepared at a warm temperature and equilibrates faster to the cold environment. A similar effect exists in heating. Comparing two identical physical systems in their equilibration, we would expect that the system with a smaller mismatch between its and the environment's temperature will thermalize faster -- yet it is not always the case.

I will present theoretical results on the Mpemba effect in over-damped Langevin dynamics and Markov jump processes. I will link the Mpemba effect's occurrence with the physical systems' properties and dynamics. In particular, I will derive the necessary conditions for the Mpemba effect in the small diffusion limit of one-dimensional over-damped Langevin dynamics on a double-well potential. Next, I will provide analytical results and insights on when the Mpemba effect happens in Markov jump processes as a function of the dynamics. I will explore the connections between the Mpemba effect and optimal transport. I will also discuss potential applications of anomalous thermal relaxations in MCMC sampling.

This material is based upon work supported by the National Science Foundation under Grant No. DMR-1944539.

Orateur: Marija Vucelja (University of Virginia)

11:00 Coffee/Tea break - Cathedrale

Hackathon / Free discussion - L107, L203, L217, Cathedrale, Terrace

12:30 Lunch - IPa canteen

Hackathon / Free discussion - L107, L203, L217, Cathedrale, Terrace

15:00 Coffee/Tea break - Cathedrale

Feedback: Cathedrale

Hackathon / Free discussion - L107, L203, L217, Cathedrale, Terrace

jeudi 14 septembre

Hackathon / Free discussion - L107, L203, L217, Cathedrale, Terrace

11:00 Coffee/Tea break - Cathedrale

Feedback: Cathedrale

Hackathon / Free discussion - L107, L203, L217, Cathedrale, Terrace

12:30 Lunch - IPa canteen

Faraday discussion: - Amphitheatre

35 Approximate sampling and estimation of partition functions using neural networks

Discussion about the arXiv preprint 2209.10423

"We consider the closely related problems of sampling from a distribution known
up to a normalizing constant, and estimating said normalizing constant. We show
how variational autoencoders (VAEs) can be applied to this task. In their standard
applications, VAEs are trained to fit data drawn from an unknown and intractable
distribution. We invert the logic and train the VAE to fit a simple and tractable
distribution, on the assumption of a complex and intractable latent distribution,
specified up to normalization. This procedure constructs approximations without
the use of training data or Markov chain Monte Carlo sampling. We illustrate our
method on three examples: the Ising model, graph clustering, and ranking."

Orateur: Eiji Kawasaki (CEA)

2209.10423.pdf

15:00 Coffee/Tea break - Cathedrale

Colloquium: Freddy Bouchet - Amphitheatre

36 Probabilistic forecast of extreme heat waves and climate tipping points using neural networks and rare event simulations

Rare events are of primarily importance for understanding the impact of climate change. The first class are extreme events which have devastating impacts; the second are rare trajectories which lead to bifurcations and drastic changes of the climate system configurations and tipping points. However, because those events are too rare and realistic models are too complex, they cannot be computed directly. We must complement traditional approaches using direct numerical simulations with tools issued from statistical physics, stochastic processes, and applied mathematics, to enhance the sampling of these rare events.

Rare event simulation is a very efficient tool to oversample drastically the statistics of rare events. As a complementary tool, machine learning is a way to build statistical models of the probability to observe rare events, based on long datasets. We will demonstrate the efficiencies of both approaches for climate application and discuss their coupling to mutually enhance their efficiency, solving the lack of data issue on the one hand and the sampling difficulty on the other hand.

We will discuss applications to a few key problems related to the emergency related to the climate crisis and the needed transitions. First, we will study the probability of extreme heat waves, the deadliest phenomena related to climate change. Second, we will discuss apply those tools to climate tipping points. Finally, we will study the probability of rare periods of imbalance between renewable electricity production and demand, a key factor for the design of future electricity systems.

Orateur: Freddy BOUCHET (ENS, LMD, IPSL)

Hackathon / Free discussion - L107, L203, L217, Cathedrale, Terrace

19:00 Social dinner

vendredi 15 septembre

Hackathon / Free discussion - L107, L203, L217, Cathedrale, Terrace

Result Communication: Amphitheatre

37 Cooperative learning: improving inference through sampling with coupled systems

Constraint Satisfaction problems (CSPs) deal with finding a solution to a set of variables that satisfy a set of constraints. In the last decade, it has been found that many CSPs can have different levels of computational hardness when the number of constraints is changed. The same issue arises in inference problems in the so-called planted setting, where a planted configuration always exists by construction, and the number of constrains plays the role of a signal-to-noise ratio. In this presentation, I will focus on a particular model, namely the binary perceptron in the teacher-student scenario. In this system, it is known when it is easy or hard to find the teacher's solution, based on a statistical mechanics analysis of the landscape of solutions. Nevertheless, in practice, it remains very difficult to find the teacher's solutions in some parts of the easy region. We will show that by replicating the same system and adding a cooperative coupling between the different replicas, the phase diagram of the model is changed and it becomes much easier to find the teacher's solution using polynomial-time optimization algorithms based on MonteCarlo sampling, such as simulated annealing.

Orateur: Giovanni CATANIA (Universidad Complutense Madrid)

38 Anharmonic thermo-elasticity of tungsten from accelerated Bayesian adaptive biasing force calculations with data-driven force fields

The elastic properties of tungsten, a ubiquitous material in future energy systems, are investigated up to its melting temperature by means of a data-driven approach. The proposed workflow combines machine learning of the force field and enhanced sampling of the crystalline structure. While the machine learning force field achieves the accuracy of ab initio calculations, its implementation in sampling methods is often limited due to its high computational cost, which is commonly a few orders of magnitude larger than that of traditional potentials. To overcome this limitation, we propose a fast and robust Bayesian sampling scheme aiming at estimating the fully anharmonic free energy of crystalline solids with the help of an improved adaptive biasing force method. This method performs a thermodynamic integration from a harmonic reference system, wherein zero frequencies associated with the periodic boundaries are screened off. The proposed sampling method drastically improves the speed of convergence and overall accuracy. We demonstrate the efficiency of the improved method by calculating the second order derivatives of the free energy, such as the elastic constants, which are performed almost 100 times faster than with the standard methods. The proposed method enables the prediction of the elastic properties of tungsten in the range of temperatures that cannot be investigated experimentally, from 2100 K to the melting temperature. The accuracy and numerical efficiency of the proposed strategy open up many avenues for the reliable prediction of finite-temperature properties of materials, such as the relative stability of structural defects and elastic constants.

Orateur: Anruo Zhong (Université Paris-Saclay, CEA, Service de Recherche en Corrosion et Comportement des Matériaux, SRMP, 91191 Gif-sur-Yvette, France)

11:00 Coffee/Tea break - Cathedrale

Feedback: Amphitheatre

Hackathon / Free discussion - L107, L203, L217, Cathedrale, Terrace

12:30 Lunch - IPa canteen

General address: Conclusion week 2 - Amphitheatre

15:00 Coffee/Tea break - Cathedrale

Hackathon / Free discussion - L107, L203, L217, Cathedrale, Terrace

lundi 18 septembre

Hackathon / Free discussion - L107, L203, L217, Cathedrale, Terrace

Result Communication: Amphitheatre

39 Sampling with flows, diffusion and autoregressive neural networks: A spin-glass perspective

Recent years witnessed the development of powerful generative models based on flows, diffusion or autoregressive neural networks, achieving remarkable success in generating data from examples with applications in a broad range of areas. A theoretical analysis of the performance and understanding of the limitations of these methods remain, however, challenging.

In this talk, I present our recent work undertaking a step in this direction, by analysing the efficiency of sampling by these methods on a class of problems with a known probability distribution and comparing it with the sampling performance of more traditional methods such as the Monte Carlo Markov chain and Langevin dynamics.

We focus on a class of probability distribution widely studied in the statistical physics of disordered systems that relate to spin glasses, statistical inference and constraint satisfaction problems. We leverage the fact that sampling via flow-based, diffusion-based or autoregressive networks methods can be equivalently mapped to the analysis of a Bayes optimal denoising of a modified probability measure. Our findings demonstrate that these methods encounter difficulties in sampling stemming from the presence of a first-order phase transition along the algorithm's denoising path.

Our conclusions go both ways: we identify regions of parameters where these methods are unable to sample efficiently, while that is possible using standard Monte Carlo or Langevin approaches. We also identify regions where the opposite happens: standard approaches are inefficient while the discussed generative methods work well.

Orateur: Davide Ghio (EPFL)

40 Disentangling representations in RBMs without adversaries

A goal of unsupervised machine learning is to build representations of complex high-dimensional data, with simple relations to their properties. Such disentangled representations make it easier to interpret the significant latent factors of variation in the data, as well as to generate new data with desirable features. The methods for disentangling representations often rely on an adversarial scheme, in which representations are tuned to avoid discriminators from being able to reconstruct information about the data properties (labels). Unfortunately, adversarial training is generally difficult to implement in practice. Here we propose a simple, effective way of disentangling representations without any need to train adversarial discriminators and apply our approach to Restricted Boltzmann Machines, one of the simplest representation-based generative models. Our approach relies on the introduction of adequate constraints on the weights during training, which allows us to concentrate information about labels on a small subset of latent variables. The effectiveness of the approach is illustrated with four examples: the CelebA dataset of facial images, the two-dimensional Ising model, the MNIST dataset of handwritten digits, and the taxonomy of protein families. In addition, we show how our framework allows for analytically computing the cost, in terms of the log-likelihood of the data, associated with the disentanglement of their representations.

Orateur: Jorge Fernández de Cossío Díaz (PSL)

11:00 Coffee/Tea break - Cathedrale

Feedback

Hackathon / Free discussion - L107, L203, L217, Cathedrale, Terrace

12:30 Lunch - Buffet at IPa

Hackathon / Free discussion - L107, L203, L217, Cathedrale, Terrace

15:00 Coffee/Tea break - Cathedrale

Hackathon / Free discussion - L107, L203, L217, Cathedrale, Terrace

mardi 19 septembre

Hackathon / Free discussion - L107, L203, L217, Cathedrale, Terrace

Faraday discussion: - Amphitheatre

41 Learning in Restricted Boltzmann Machines: The Critical Role of Sampling Processes

Generative models aim to learn the empirical distribution of a given data set in order to build a probabilistic model capable of generating new samples that are statistically similar to the data set. One can also assume that one can obtain an approximately tractable analytical description of this distribution.

In this presentation, I will specifically consider the case of the so-called Restricted Boltzmann Machine (RBM), a bipartite generative neural network where the learning process crucially depends on sampling: The gradient is computed using a Monte Carlo estimate of the correlation between the variables of the model. When we deal with multimodal datasets, accurate sampling during the learning process becomes increasingly challenging. This complexity arises when different modes begin to manifest in the model during training, resulting in the rapid divergence of chain mixing times to impractical values.

First, I will show how by means of a bias Monte Carlo method, it is feasible to drastically speed up the mixing time for structured datasets living in a low-dimensional space. Second, I will explore a mean-field method, in which we can expand the system into a low-dimensional subspace where we can efficiently approximate the free energy landscape and design a convex learning algorithm - at the cost of neglecting fluctuations that lie outside the subspace.

I will conclude with an open discussion on the limitation surrounding these techniques.

References by importance order regarding discussion time:
1a - https://scipost.org/SciPostPhys.14.3.032
1b - https://journals.aps.org/prl/abstract/10.1103/PhysRevLett.127.158303 (arXiv:2103.10755v2)
2 - https://journals.aps.org/prl/pdf/10.1103/PhysRevLett.108.165701?casa_token=0YgEmURTLA0AAAAA%3A5sVciafIrVvzUZsjL-B8P7kFEHBFQFHQ4nyaJ2cQOShzN1tXE2mo5jlUedKizf0tbTNmZ2x4wb2PJg (arXiv:1103.2599v3)
3 - https://proceedings.neurips.cc/paper_files/paper/2021/hash/2aedcba61ca55ceb62d785c6b7f10a83-Abstract.html

Orateur: Aurélien DECELLE (Universidad Complutense de Madrid)

1103.2599.pdf

2103.10755.pdf

Neurips.pdf

SciPostPhys_14_3_032.pdf

11:00 Coffee/Tea break - Cathedrale

Feedback: Cathedrale

Hackathon / Free discussion - L107, L203, L217, Cathedrale, Terrace

12:30 Lunch - IPa canteen

Hackathon / Free discussion - L107, L203, L217, Cathedrale, Terrace

15:00 Coffee/Tea break - Cathedrale

Hackathon / Free discussion - L107, L203, L217, Cathedrale, Terrace

16:30 Social drinks at IPa - Terrace

mercredi 20 septembre

Hackathon / Free discussion - L107, L203, L217, Cathedrale, Terrace

Result Communication: - Amphitheatre

42 Renormalization Group Approach for Machine Learning Hamiltonian

Reconstructing, or generating, high dimensional probability distributions starting from data is a central problem in machine learning and data sciences.
We will present a method —The Wavelet Conditional Renormalization Group —that combines ideas from physics (renormalization group theory) and computer science (wavelets, Monte-Carlo sampling, etc.). The Wavelet Conditional Renormalization Group allows reconstructing in a very efficient way classes of high dimensional distributions and the associated Hamiltonians hierarchically from large to small length scales. We will present the method and then show its applications to data from statistical physics and cosmology.

Orateur: Misaki Ozawa (CNRS, Univ. Grenoble Alpes, France)

43 Sampling with Neural Hamiltonian Flows

Normalizing Flows (NF) are Generative models which transform a simple prior distribution into the desired target. They however require the design of an invertible mapping whose Jacobian determinant has to be computable. Recently introduced, Neural Hamiltonian Flows (NHF) are Hamiltonian dynamics-based flows, which are continuous, volume-preserving and invertible and thus make for natural candidates for robust NF architectures. In particular, their similarity to classical Mechanics could lead to easier interpretability of the learned mapping. In this presentation, I will detail the NHF architecture and show that they may still pose a challenge to interpretability. For this reason, I will introduce a fixed kinetic energy version of the model. Inspired by physics, this approach improves interpretability and requires less parameters than the original model. I will talk about the robustness of the NHF architecture, especially its fixed-kinetic version, on a simple 2D problem and present first results in higher dimension. Finally, I will show how to adapt NHF to the context of Bayesian inference and illustrate the method on an example from cosmology.

Orateur: Vincent Souveton (LMBP - Université Clermont Auvergne)

11:00 Coffee/Tea break - Cathedrale

Feedback: Cathedrale

Hackathon / Free discussion - L107, L203, L217, Cathedrale, Terrace

12:30 Lunch - IPa canteen

General address: Perspective brainstorming - Cathedrale

15:00 Coffee break

Colloquium: Alberto Rosso - Amphitheatre

44 Rare events, large deviation and sampling

In many systems exceptional events can have a crucial impact, while the routine is peaceful and with no consequences. Well known examples are the earthquakes in the Earth's lithosphere or the events of extreme weather. Predicting their magnitude or their occurrence rate is a major challenge for human security and economy. Large deviation theory is the branch of probabilities that adresses this issue. In this talk, I introduce the rate function, the central object of the theory, and study it for several dynamical models. A special focus is dedicated to the Monte Carlo approaches able to compute the rate function and designed to sample rare events. In the last part of the talk I will also discuss the issue of rare events in the context of learning. How detect exceptional rare events from examples that are almost always non relevant?

Orateur: Alberto ROSSO (LPTMS-CNRS)

Hackathon / Free discussion - L107, L203, L217, Cathedrale, Terrace

jeudi 21 septembre

Hackathon / Free discussion - L107, L203, L217, Cathedrale, Terrace

Result Communication: - Amphitheatre

45 Conditioning and variance reduction schemes for more accurate estimation of thermodynamic and transport properties of metal alloys - Manuel Athènes

The Monte Carlo method is a stochastic simulation approach mainly used to estimate multi-dimensional integrals. This is traditionally done by generating a Markov chain of states and then implementing an estimator. In this context, conditioning is a trick consisting in doing part of the job in closed form, through numerical quadrature, so as to reduce the statistical variance associated with the estimator. In this talk, we present several applications of this variance reduction technique in Monte Carlo simulations of condensed-matter systems. In particular, we show how to sample multi-particle systems more ergodically with the help of adaptive biasing forces and how to estimate thermodynamic quantities more accurately through conditioning and the law of total variance. We then show that conditioning can also be used to improve the measurement of linear mass transport coefficients through a law of total diffusion and by solving the first-passage problems associated with the transport problem. Illustrative applications are chosen in the field of materials science: structural transitions in metallic clusters, phase diagram of FeCr alloy, migration barriers associated with point defects in α-Iron, thermo-elasticity of tungsten with data-driven force fields and elasto-diffusion in aluminium.

11:00 Coffee/Tea break - Cathedrale

Feedback: Cathedrale

Hackathon / Free discussion - L107, L203, L217, Cathedrale, Terrace

12:30 Lunch - IPa canteen

Hackathon / Free discussion - L107, L203, L217, Cathedrale, Terrace

15:00 Coffee/Tea break - Cathedrale

Hackathon / Free discussion - L107, L203, L217, Cathedrale, Terrace

19:00 Social dinner

vendredi 22 septembre

Hackathon / Free discussion - L107, L203, L217, Cathedrale, Terrace

Result Communication: - Amphitheatre

46 Active particle systems through the lens of piecewise deterministic Markov processes

Run-and-tumble particles are a paradigmatic model in out-of-equilibrium physics that exhibits interesting phenomena not found in their passive counterparts such as motility-induced phase separation. I will present the long-time behavior of a pair of such particles with hard-core interactions on a unidimensional torus and on a line by casting them as a piecewise deterministic Markov process. I will discuss the non-equilibrium steady states of theses systems and the universality classes they pertain to as well as the speed of convergence towards the invariant measures.

Orateur: Léo Hahn (Université Clermont Auvergne)

47 Splitting schemes for second order approximations of piecewise-deterministic Markov processes

Piecewise deterministic Markov processes (PDMPs) received substantial interest in recent years as an alternative to classical Markov chain Monte Carlo algorithms. While theoretical properties of PDMPs have been studied extensively, their practical implementation remains limited to specific applications in which bounds on the gradient of the negative log-target can be derived. In order to address this problem, we propose to approximate PDMPs using splitting schemes, that means simulating the deterministic dynamics and the random jumps in two different stages. We show that as expected basic symmetric splittings of PDMPs are of second order. Then we focus on the Zig-Zag sampler (ZZS) and illustrate with numerical simulations the advantages of our proposed scheme over competitors.

Orateur: Andrea Bertazzi (CMAP École polytechnique)

11:00 Coffee/Tea break - Cathedrale

General address: Conclusion - Full program - Cathedrale

12:30 Lunch - IPa canteen

Hackathon / Free discussion - L107, L203, L217, Cathedrale, Terrace