Advanced topics in numerical analysis

by Francesco Tudisco (GSSI), Nicola Guglielmi (GSSI)

Tuesday 1 Nov 2022, 00:00 → Tuesday 31 Jan 2023, 23:00 Europe/Rome

Ex-ISEF/Building-Room D (GSSI)

Lecturers
Nicola Guglielmi, nicola.guglielmi@gssi.it
Francesco Tudisco, francesco.tudisco@gssi.it

Timetable and workload
Lectures: 60 hours
Final project assignment: 20 hours

Course description and outcomes
This course is an introduction to modern numerical analysis and numerical optimization. The primary objective of the course is to develop a graduate-level understanding of computational mathematics and skills to solve a range of real-world mathematical problems on a computer by implementing advanced numerical algorithms using a scientific computing language (such as MATLAB or Julia). 

Course requirements
Calculus and basic linear algebra and numerical analysis. Previous programming experience in any language may help.

Course content
The course will cover the following topics

PRELIMINARIES (NG)

• Quadrature
  • I.1. Quadrature formulas, order conditions.
  • I.2. Error analysis
  • I.3. Superconvergence and orthogonal polynomials 
  • I.4. Gauss quadrature.
• Initial Value Problems for ODEs
  • II.1. One-step (Runge-Kutta) methods 
  • II.2. Stiff problems
  • II.3. Collocation methods.

 

PRELIMINARIES (FT)

• Eigenvalues and variational characterization(s)
• (Nonlinear) Cheeger inequality and projection operators
• Power method and subspace iterations
• Basic iterative solvers for linear systems

 

MAIN TOPICS (NG)

• Elements of numerical optimization
  • IV.1. The main methods of convex optimization. 
  • IV.2. Univariate optimization. Unimodular functions.
• Gradient methods
  • V.1. Gradient systems.
  • V.2. Classical gradient methods. 
  • V.3. Conjugate direction methods.
• Constrained optimization
  • VI.1. Penalty methods. 
  • VI.2. Projection methods.
• VII. Eigenvalue optimization
  • VII.1. Variational properties of eigenvalues. 
  • VII.2. Structured spectral optimization.

 

MAIN TOPICS (FT)

• GMRES and Krylov subspace methods for eigenvalues
• Quasi-Newton methods for nonlinear systems and optimization
• Linear Multistep and Nesterov acceleration
• Nonlinear fixed point iterations and monotone operators
• Stochastic gradient descent

 

Books of reference

• E. Hairer, G. Wanner, S. P. Nørsett; Solving Ordinary Differential Equations I
• R. Fletcher, Practical methods of optimization. Second edition. 
• W. Gautschi, Numerical analysis. An introduction.
• J. Stoer and R. Bulirsch, Introduction to numerical analysis. Third edition
• Y. Saad; Iterative methods for Sparse Linear Systems (Free Online Version)
• Y. Saad;  Numerical methods for Large Eigenvalue Problems (Free Online Version)
• E.K. Ryu, S. Boyd; A premier on monotone operator methods (Free Oline Version)
• J. Nocedal, S. J. Wright; Numerical optimization 

Examination and grading
Students will be evaluated on the basis of a written exam and computational assessment to be taken at the end of the course. Both tests are graded based on the ECTS grading scale. 

Lecture notes - Francesco Tudisco