Recent seminars

Europe/Lisbon — Online

Leong Chuan Kwek

Leong Chuan Kwek, Nanyang Technological University, Singapore
Machine Learning and Quantum Technology

The rise of machine learning in recent times has remarkably transformed science and society. The goal of machine learning is to get computers to act without being explicitly programmed. Machine learning with deep reinforcement learning (RL) was recently recognized as a powerful tool to engineer dynamics in quantum system. Also, recently there has been some interest to exploit and leverage the limited available quantum resources for performing classically challenging tasks with noisy intermediate-scale quantum (NISQ) computers. Here, we discuss some of our recent work on the applications of machine learning to quantum systems.

Video

Europe/Lisbon — Online

J. Nathan Kutz

J. Nathan Kutz, University of Washington
Deep learning for the discovery of parsimonious physics models

A major challenge in the study of dynamical systems is that of model discovery: turning data into reduced order models that are not just predictive, but provide insight into the nature of the underlying dynamical system that generated the data. We introduce a number of data-driven strategies for discovering nonlinear multiscale dynamical systems and their embeddings from data. We consider two canonical cases: (i) systems for which we have full measurements of the governing variables, and (ii) systems for which we have incomplete measurements. For systems with full state measurements, we show that the recent sparse identification of nonlinear dynamical systems (SINDy) method can discover governing equations with relatively little data and introduce a sampling method that allows SINDy to scale efficiently to problems with multiple time scales, noise and parametric dependencies. For systems with incomplete observations, we show that the Hankel alternative view of Koopman (HAVOK) method, based on time-delay embedding coordinates and the dynamic mode decomposition, can be used to obtain a linear models and Koopman invariant measurement systems that nearly perfectly captures the dynamics of nonlinear quasiperiodic systems. Neural networks are used in targeted ways to aid in the model reduction process. Together, these approaches provide a suite of mathematical strategies for reducing the data required to discover and model nonlinear multiscale systems.

Video

Additional file

Kutz slides.pdf

Europe/Lisbon — Online

Simon Du

Simon Du, University of Washington
Provable Representation Learning

Representation learning has been widely used in many applications. In this talk, I will present our work, which uncovers when and why representation learning provably improves the sample efficiency, from a statistical learning point of view. I will show

  1. the existence of a good representation among all tasks, and
  2. the diversity of tasks are key conditions that permit improved statistical efficiency via multi-task representation learning.

These conditions provably improve the sample efficiency for functions with certain complexity measures as the representation. If time permits, I will also talk about leveraging the theoretical insights to improve practical performance.

Video

Additional file

Du slides.pdf

Europe/Lisbon — Online

Usman Khan

Usman Khan, Tufts University
Distributed ML: Optimal algorithms for distributed stochastic non-convex optimization

In many emerging applications, it is of paramount interest to learn hidden parameters from data. For example, self-driving cars may use onboard cameras to identify pedestrians, highway lanes, or traffic signs in various light and weather conditions. Problems such as these can be framed as classification, regression, or risk minimization in general, at the heart of which lies stochastic optimization and machine learning. In many practical scenarios, distributed and decentralized learning methods are preferable as they benefit from a divide-and-conquer approach towards data at the expense of local (short-range) communication. In this talk, I will present our recent work that develops a novel algorithmic framework to address various aspects of decentralized stochastic first-order optimization methods for non-convex problems. A major focus will be to characterize regimes where decentralized solutions outperform their centralized counterparts and lead to optimal convergence guarantees. Moreover, I will characterize certain desirable attributes of decentralized methods in the context of linear speedup and network independent convergence rates. Throughout the talk, I will demonstrate such key aspects of the proposed methods with the help of provable theoretical results and numerical experiments on real data.

Video

Additional file

Khan slides.pdf

Europe/Lisbon — Online

Ard Louis

Ard Louis, University of Oxford
Deep neural networks have an inbuilt Occam's razor

One of the most surprising properties of deep neural networks (DNNs) is that they perform best in the overparameterized regime. We are taught early on that having more parameters than data points is a terrible idea. So why do DNNs work so well in a regime where classical learning theory predicts they should heavily overfit? By adapting the coding theorem from algorithmic information theory (which every physicist should learn about!) we show that DNNs are exponentially biased at initialisation to functions that have low descriptional (Kolmogorov) complexity. In other words, DNNs have an inbuilt Occam's razor, a bias towards simple functions. We next show that stochastic gradient descent (SGD), the most popular optimisation method for DNNs, follows the same bias, and so does not itself explain the good generalisation of DNNs. Our approach naturally leads to a marginal-likelihood PAC-Bayes generalisation bound which performs better than any other bounds on the market. Finally, we discuss why this bias towards simplicity allows DNNs to perform so well, and speculate on what this may tell us about the natural world.

Video

Additional file

Louis slides.pdf