Orateur
Description
Energy-based models (EBMs) are powerful generative machine learning models that are able to encode the complex distribution of a dataset in the Gibbs-Boltzmann distribution of a model energy function. This means that, if properly trained, they can be used to synthesize new patterns that resemble those of the data set as closely as possible, but also that this energy function can be used to "learn" something about the building mechanisms of the dataset under study. Indeed, EBMs can be considered a powerful modeling tool for arbitrary data if one were able to map complex energy functions defined in a neural network into spin-interaction Hamiltonians that can be studied using standard tools of statistical physics. Such an approach has long been used in physics for inverse Ising problems. The goal now is to extend this approach to more complex energy functions that can encode all higher order correlations in complex data. While this program is very encouraging, training good EBMs is particularly challenging, mainly because they rely on long Monte Carlo sampling processes to estimate the log-likelihood gradient. These sampling processes must be repeated once and over again each time parameters are updated, making the possibility of convergence to equilibrium at each step extremely difficult. In my talk, I will discuss how non-equilibrium effects can be exploited to train fast generators capable of accurately reproducing multimodal distributions of data, but at the cost of losing the connection between the Boltzmann distribution and the empirical distribution. I will also discuss optimized sampling algorithms or approximation schemes that aim to favor MCMC convergence throughout the learning process, as well as the possibility of using population annealing methods to mitigate the apperance of out-of-equilibrium effects in the trained models.
