Planned seminars

Europe/Lisbon
Instituto Superior Técnicohttps://tecnico.ulisboa.pt

Anja Butter

Anja Butter, ITP, University of Heidelberg

First-principle simulations are at the heart of the high-energy physics research program. They link the vast data output of multi-purpose detectors with fundamental theory predictions and interpretation. In the coming LHC runs, these simulations will face unprecedented precision requirements to match the experimental accuracy. New ideas and tools based on neural networks have been developed at the interface of particle physics and machine learning. They can improve the speed and precision of forward simulations and handle the complexity of collision data. Such networks can be employed within established simulation tools or as part of a new framework. Since neural networks can be inverted, they open new avenues in LHC analyses.

Europe/Lisbon
Instituto Superior Técnicohttps://tecnico.ulisboa.pt

Paulo Tabuada

Paulo Tabuada, University of California, Los Angeles

Deep neural networks have drastically changed the landscape of several engineering areas such as computer vision and natural language processing. Notwithstanding the widespread success of deep networks in these, and many other areas, it is still not well understood why deep neural networks work so well. In particular, the question of which functions can be learned by deep neural networks has remained unanswered.

In this talk we give an answer to this question for deep residual neural networks, a class of deep networks that can be interpreted as the time discretization of nonlinear control systems. We will show that the ability of these networks to memorize training data can be expressed through the control theoretic notion of controllability which can be proved using geometric control techniques. We then add an additional ingredient, monotonicity, to conclude that deep residual networks can approximate, to arbitrary accuracy with respect to the uniform norm, any continuous function on a compact subset of $n$-dimensional Euclidean space by using at most $n+1$ neurons per layer. We will conclude the talk by showing how these results pave the way for the use of deep networks in the perception pipeline of autonomous systems while providing formal (and probability free) guarantees of stability and robustness.

Europe/Lisbon
Instituto Superior Técnicohttps://tecnico.ulisboa.pt

John Baez

John Baez, U.C. Riverside

Shannon entropy is a powerful concept. But what properties single out Shannon entropy as special? Instead of focusing on the entropy of a probability measure on a finite set, it can help to focus on the "information loss", or change in entropy, associated with a measure-preserving function. Shannon entropy then gives the only concept of information loss that is functorial, convex-linear and continuous.

This is joint work with Tom Leinster and Tobias Fritz.

Europe/Lisbon
Instituto Superior Técnicohttps://tecnico.ulisboa.pt

Petar Veličković

Petar Veličković, DeepMind and University of Cambridge

The last decade has witnessed an experimental revolution in data science and machine learning, epitomised by deep learning methods. Indeed, many high-dimensional learning tasks previously thought to be beyond reach — such as computer vision, playing Go, or protein folding — are in fact feasible with appropriate computational scale. Remarkably, the essence of deep learning is built from two simple algorithmic principles: first, the notion of representation or feature learning, whereby adapted, often hierarchical, features capture the appropriate notion of regularity for each task, and second, learning by local gradient-descent type methods, typically implemented as backpropagation.

While learning generic functions in high dimensions is a cursed estimation problem, most tasks of interest are not generic, and come with essential pre-defined regularities arising from the underlying low-dimensionality and structure of the physical world. This talk is concerned with exposing these regularities through unified geometric principles that can be applied throughout a wide spectrum of applications.

Such a 'geometric unification' endeavour in the spirit of Felix Klein's Erlangen Program serves a dual purpose: on one hand, it provides a common mathematical framework to study the most successful neural network architectures, such as CNNs, RNNs, GNNs, and Transformers. On the other hand, it gives a constructive procedure to incorporate prior physical knowledge into neural architectures and provide principled way to build future architectures yet to be invented.

Europe/Lisbon
Instituto Superior Técnicohttps://tecnico.ulisboa.pt

Dario Izzo

Dario Izzo, European Space Agency

The problem of determining the density distribution of celestial bodies from the induced gravitational pull is of great importance in astrophysics as well as space engineering (thinking of situations where spacecraft need to perform orbital and surface proximity operations). Knowledge of a body density distribution provides also great insights on the body's origin and composition. In practice, the state-of-the-art approaches for modelling the gravity field of extended bodies are spherical harmonics models, mascon models and polyhedral gravity models. All of these, however, while being widely studied and developed since the early works from Laplace, introduce requirements such as knowledge of a shape model, assumption of a homogeneous internal density, being outside the Brillouin sphere, etc...


In this talk, we introduce and explain Neural Density Fields, a new approach to represent the density of extended bodies and learn its accurate form inverting data from gravitational accelerations, orbits or the gravity potential. The resulting deep learning model, called geodesyNets is able to compete with classical approaches while solving most of their limitations. We also introduce eclipseNets, a deep learning model based on related ideas and able to learn the eclipse shadow cones of irregular bodies, thus allowing highly precise propagation and stability studies.

Europe/Lisbon
Instituto Superior Técnicohttps://tecnico.ulisboa.pt

Audrey Durand

Audrey Durand, IID, Université Laval, Canada

Learning a behaviour to conduct a given task can be achieved by interacting with the the environment. This is the crux of reinforcement learning (RL), where an (automated) agent learns to solve a problem through an iterative trial-and-error process. More specifically, an RL agent can interact with the environment and learn from these interactions by observing a feedback on the goal task. Therefore, these methods typically require to be able to intervene on the environment and make (possibly a very large number of) mistakes. Although this can be a limiting factor in some applications, simple RL settings, such as bandit settings, can still host a variety of problems for interactively learning behaviours. In other situations, simulation might be the key.

In this talk, we will show that RL can be used to formulate and tackle data acquisition (imaging) problems in neurosciences. We will see how bandit methods can be used to optimize super-resolution imaging by learning on real devices through an actual empirical process. We will also see how simulation can be leveraged to learn more sequential decision making strategies. These applications highlight the potential of RL to support expert users on difficult task and enable new discoveries.

Europe/Lisbon
Instituto Superior Técnicohttps://tecnico.ulisboa.pt

Joseph Bakarji

Joseph Bakarji, University of Washington

Dimensional analysis is a robust technique for extracting insights and finding symmetries in physical systems, especially when the governing equations are not known. The Buckingham Pi theorem provides a procedure for finding a set of dimensionless groups from given measurements, although this set is not unique. We propose an automated approach using the symmetric and self-similar structure of available measurement data to discover the dimensionless groups that best collapse this data to a lower dimensional space according to an optimal fit. We develop three data-driven techniques that use the Buckingham Pi theorem as a constraint: (i) a constrained optimization problem with a nonparametric function, (ii) a deep learning algorithm (BuckiNet) that projects the input parameter space to a lower dimension in the first layer, and (iii) a sparse identification of nonlinear dynamics (SINDy) to discover dimensionless equations whose coefficients parameterize the dynamics. I discuss the accuracy and robustness of these methods when applied to known nonlinear systems.

Europe/Lisbon
Instituto Superior Técnicohttps://tecnico.ulisboa.pt

Inês Hipólito

Inês Hipólito, Humboldt-Universität

Living beings do an extraordinary thing. By being alive they are resisting the second law of thermodynamics. This law stipulates that open, living systems tend to dissipation by the increase of entropy or chaos. From minimal cognitive organisms like plants to more complex organisms equipped with nervous systems, all living systems adjust and adapt to their environments, thereby resisting the second law. Impressively, while all animals cognitively enact and survive their local environments, more complex systems do so also by actively constructing their local environments, thereby not only defying the second law, but also (evolution) selective properties. Because all living beings defy the second law by adjusting and engaging with the environment, a prominent question is how do living organisms persist while engaging in adaptive exchanges with their complex environments? In this talk I will offer an overview of how the Free Energy Principle (FEP) offers a principled solution to this problem. The FEP prescribes that living systems maintain themselves by remaining in non-equilibrium steady states by restricting themselves to a limited number of states; it has been widely applied to explain neurocognitive function and embodied action, develop artificial intelligence and inspire psychopathology models.