NUmerical methods for Compression and LEarning

11 May 2022, 10:00 → 13 May 2022, 19:00 Europe/Rome

Gran Sasso Science Institute

Gran Sasso mountain from Collemaggio Park, L'Aquila

As the amount of available data is growing very fast, the importance of being able to handle and exploit very-large-scale data in an efficient and robust manner is becoming increasingly more relevant. This workshop aims at bringing together experts from signal processing, compressed sensing, low rank methods and machine learning with the goal of highlighting modern approaches as well as challenges in computational mathematics arising in all these areas and at their intersection.

The workshop will feature lectures from invited speakers as well as poster sessions open to all participants.

Online participation will be possible via the zoom link:
https://us02web.zoom.us/j/83830006962?pwd=SmI1MTVKRTllU3dBR01Ybko5bzBJdz09

If you are attending via zoom, please raise your hand using the specific zoom feature at the end of the talks to ask your questions.

Workshop flyer

Speakers:

• Evrim Acar Ataman (SIMULA Labs, Norway)
• Matthew Colbrook (Univ Cambridge, UK, and ENS Paris, France)
• Marcello Carioni (Univ Twente, Netherlands)
• Bruno Iannazzo (Univ Perugia, Italy)
• Christian Lubich (Univ Tuebingen, Germany)
• Carlo Marcati (University of Pavia, Italy)
• Daniel Peterseim (Univ Augsburg, Germany)
• Leonardo Robol (Univ Pisa, Italy)
• Karsten Urban (Ulm University, Germany)
• Andre Uschmajew  (MPI Leipzig, Germany)
• Konstantin Usevich (CNRS Nancy, France)

Organizers:

• Nicola Guglielmi
• Francesco Tudisco

.  . 

Wednesday, 11 May

Poster: Welcome and Poster Session

Thursday, 12 May

Welcome

Lecture talk

1 The Reduced Basis Method in Space and Time: Challenges, Limits and Perspectives — Part 1

In many engineering applications, a partial differential equation (PDE) has to be solved very often (“multi-query”) and/or extremely fast (“realtime”) and/or using restricted memory/CPU (“cold computing”). Moreover, the mathematical modeling yields complex systems in the sense that:
(i) each simulation is extremely costly, its CPU time may be in the order of several weeks;
(ii) we are confronted with evolutionary, time-dependent processes with long time horizons or time-periodic behaviors (which often requires long-time horizons in order to
find the time-periodic solution). All problems rely on time-dependent parameterized partial differential equations (PPDEs);
(iii) the processes often involve transport and wave-type phenomena as well as complex coupling and nonlinearities.

Without significant model reduction, one will not be able to tackle such problems. Moreover, there is a requirement in each of the above problems to ensure that the reduced simulations are certified in the sense that a reduced output comes along with a computable indicator which is a sharp upper bound of the error.

The Reduced Basis Method (RBM) is a well-established method for Model Order Reduction of PPDEs. We recall the classical framework for well-posed linear problems and extend this setting towards time-dependent problems of heat, transport, wave and Schrödinger type. The question of optimal approximation rates is discussed and possible benefits of ultraweak variational space-time methods are described.

Speaker: Karsten Urban (University of Ulm)

GSSI-Slides-Urban.pdf

2 Computing Means of SPD matrices — Part 1

One of the most fruitful tasks in data processing is to identify structures in the set where data lie and exploit them to design better models and reliable algorithms.

As a paradigm of this process we show how the cone of positive definite matrices can be endowed with Riemannian geometries alternative to the customary Euclidean geometry. This can provide new tools for data scientists, in terms of averaging and clustering techniques.

These geometries have been used to give a definition of the geometric and the power mean of positive definite matrices. We describe the way in which these objects have been understood and the matrix analytic, geometric and computational tools needed to describe and compute them.
In particular, we will use computational techniques related to primary matrix functions, rational Krylov subspaces and Riemannian optimization.

Speaker: Bruno Iannazzo (University of Perugia)

11:00 Coffee break

Lecture talk

3 Compression of partial differential operators by numerical homogenization

Numerical homogenization is a methodology for the computational solution of multiscale partial differential equations. It aims at the compression of the corresponding partial differential operators to finite-dimensional sparse surrogate models. The surrogates are valid on a given target scale of interest, thereby accounting for the impact of features on under-resolved scales. This talk shows how to construct such surrogates by localized orthogonal decompositions and discusses the underlying mathematics as well as applications to random diffusion and Schrödinger operators.

Speaker: Daniel Peterseim (University of Augsburg)

12:30 Lunch

Lecture talk

4 The Reduced Basis Method in Space and Time: Challenges, Limits and Perspectives — Part 2

In many engineering applications, a partial differential equation (PDE) has to be solved very often (“multi-query”) and/or extremely fast (“realtime”) and/or using restricted memory/CPU (“cold computing”). Moreover, the mathematical modeling yields complex systems in the sense that:
(i) each simulation is extremely costly, its CPU time may be in the order of several weeks;
(ii) we are confronted with evolutionary, time-dependent processes with long time horizons or time-periodic behaviors (which often requires long-time horizons in order to
find the time-periodic solution). All problems rely on time-dependent parameterized partial differential equations (PPDEs);
(iii) the processes often involve transport and wave-type phenomena as well as complex coupling and nonlinearities.

Without significant model reduction, one will not be able to tackle such problems. Moreover, there is a requirement in each of the above problems to ensure that the reduced simulations are certified in the sense that a reduced output comes along with a computable indicator which is a sharp upper bound of the error.

The Reduced Basis Method (RBM) is a well-established method for Model Order Reduction of PPDEs. We recall the classical framework for well-posed linear problems and extend this setting towards time-dependent problems of heat, transport, wave and Schrödinger type. The question of optimal approximation rates is discussed and possible benefits of ultraweak variational space-time methods are described.

Speaker: Karsten Urban (University of Ulm)

GSSI-Slides-Urban.pdf

5 Computing Means of SPD matrices — Part 2

One of the most fruitful tasks in data processing is to identify structures in the set where data lie and exploit them to design better models and reliable algorithms.

As a paradigm of this process we show how the cone of positive definite matrices can be endowed with Riemannian geometries alternative to the customary Euclidean geometry. This can provide new tools for data scientists, in terms of averaging and clustering techniques.

These geometries have been used to give a definition of the geometric and the power mean of positive definite matrices. We describe the way in which these objects have been understood and the matrix analytic, geometric and computational tools needed to describe and compute them.
In particular, we will use computational techniques related to primary matrix functions, rational Krylov subspaces and Riemannian optimization.

Speaker: Bruno Iannazzo (University of Perugia)

16:00 Coffee break and poster session

Lecture talk

6 Hierarchical adaptive low-rank format with applications to discretized PDEs

When solving PDEs over tensorized 2D domains, the regularity in the solution often appears in form of an approximate low-rank
structure in the solution vector, if properly reshaped in matrix
form. This enables the use of low-rank methods such as Sylvester solvers (namely, Rational Krylov methods and/or ADI) which allow to treat separable differential operators. We consider the setting where this global smoothness is absent, but still locally exists almost everywhere. We show that the solution can still be efficiently stored by replacing low-rank matrices with appropriate hierarhical low-rank structures, and Sylvester solvers can be generalized to this setting.
The structure can be determined with a black-box approximation scheme, that finds it adaptively. In addition, theoretical results that guarantee the structure preservation hold for these more general structure as well, and the computational complexities of the proposed method nicely interpolate between the low-rank and the completely unstructured case. We discuss how to effectively evolve the structure in time when approximating the solution of the PDE at different time steps, in the hypothesis of moving (but isolated) singularities.

Speaker: Leonardo Robol (University of Pisa)

robol.pdf

7 Neural networks, flexible activation functions and tensor decompositions

Neural networks are a fundamental tool for solving various machine learning tasks, such as supervised and unsupervised classification.
Despite this success, they still have a number of drawbacks, including lack of interpretability and large number of parameters.
In this work, we are particularly interested in learning neural network architectures with flexible activation functions (contrary to fixed activation functions commonly used).
Our approach relies on a tensor-based framework for decomposition of multivariate maps, developed in the context of nonlinear system identification.
We propose a new compression algorithm which is based on a constrained coupled matrix-tensor factorization (CMTF) of the Jacobian tensor and the matrix of function evaluations.

Speaker: Konstantin Usevich (CNRS Nancy)

20:30 Social dinner

Friday, 13 May

Lecture talk

8 An SDP approach for tensor product approximation of linear operators on matrix spaces

Tensor structured linear operators play an important role in matrix equations and low-rank modelling. Motivated by this we consider the problem of approximating a matrix by a sum of Kronecker products. It is known that an optimal approximation in Frobenius norm can be obtained from the singular value decomposition of a rearranged matrix, but when the goal is to approximate the matrix as a linear map, an operator norm would be a more appropriate error measure. We present an alternating optimization approach for the corresponding approximation problem in spectral norm that is based on semidefinite programming, and report on its practical performance for small examples. This is joint work with Venkat Chandrasekaran and Mareike Dressler.

Speaker: André Uschmajew (MPI Leipzig)

9 Koopman operators and a programme on the foundations of infinite-dimensional spectral computations

Koopman operators are infinite-dimensional operators that globally linearise nonlinear dynamical systems, making their spectral information valuable for understanding dynamics. Their increasing popularity, dubbed “Koopmania”, includes 10,000s of articles over the last decade. However, Koopman operators can have continuous spectra and lack finite-dimensional invariant subspaces, making computing their spectral properties a considerable challenge. This talk describes data-driven algorithms with rigorous convergence guarantees for computing spectral properties of Koopman operators from trajectory data. We introduce residual dynamic mode decomposition (ResDMD), the first scheme for computing the spectra and pseudospectra of general Koopman operators from trajectory data without spectral pollution. By combining ResDMD and the resolvent, we compute smoothed approximations of spectral measures associated with measure-preserving dynamical systems. When computing the continuous and discrete spectrum, explicit convergence theorems provide high-order convergence, even for chaotic systems. Kernelized variants of our algorithms allow for dynamical systems with a high-dimensional state-space. We compute the spectral measures of a protein molecule (20,046-dimensional state-space) and computing nonlinear Koopman modes with error bounds for chaotic turbulent flow past aerofoils with Reynolds number greater than 100,000 (295,122-dimensional state-space). Finally, these algorithms are placed within a broader programme on the foundations of infinite-dimensional spectral computations, which extends beyond spectra to numerical solutions of PDEs, optimisation, computer-assisted proofs, and the foundations of AI.

Speaker: Matthew Colbrook

10:30 Coffee break

Lecture talk

10 Neural and operator network approximations for elliptic PDEs

The application of neural networks (NNs) to the numerical solution of PDEs has seen growing popularity in the last five years: NNs have been used as an ansatz space for the solutions, with different training approaches (PINNs, deep Ritz methods, etc.); they have also been used to infer discretization parameters and strategies.

In this talk, I will focus on deep ReLU NN approximation theory. I will first show how NNs accurately approximate functions with isolated singularities, for example the solutions to elliptic problems in polygons and polyhedra, or eigenfunctions of problems with singular potentials that arise in quantum chemistry. I will then introduce operator networks, which approximate the solution operator of PDEs. I will, in particular, consider operator networks that, given a fixed right-hand side, map sets of diffusion-reaction coefficients into the space of solutions (coefficient-to-solution map). When the coefficients are smooth, the size of the networks can then be bounded with respect to the H^1 norm of the error, uniformly over the parameter set. The proofs of our approximation rates combine elliptic regularity, classical and recent results in numerical analysis, and tools from NN approximation theory.

Speaker: Carlo Marcati (University of Pavia)

Marcati_GSSI.pdf

11 Sparse optimization methods for infinite-dimensional variational problems

In this talk, I will review several recent results about the sparse optimization of infinite-dimensional variational problems. First, I will focus on the so-called representer theorems that allow to prove, in the case of finite-dimensional data, the existence of a solution given by the linear combination of suitably chosen atoms. In particular, I will try to convey the importance of such statements in understanding the sparsity in infinite-dimensional settings, describing several possible applications for various relevant problems.
In the second part of the talk, I will focus on a sparse optimization algorithm, named generalized conditional gradient method, that is built on the characterization of sparse objects for infinite-dimensional variational problems. This algorithm is a variant of the classical Frank-Wolfe algorithm, and it does not require an a priori discretization of the domain. I will show convergence results under general assumptions on the variational problem and finally, I will present some numerical examples in the context of dynamic inverse problems regularized with optimal transport energies.

Speaker: Marcello Carioni (University of Twente)

iNuclei 13:5:2022.pdf

13:00 Lunch

Poster