Teacher: Bjarne Grimstad

Deep neural networks are being widely deployed in the industry to solve complex tasks which conventional methods have struggled to solve. There are several reasons for this breakthrough: more data, more compute, and improved algorithms and software. A perhaps equally important factor to the success is the flexibility of neural networks as a modeling framework. Without explicitly programming the logic of an algorithm, neural networks can learn to solve complex tasks from data. In this paradigm, the job of the engineer has moved from programming logic, to designing meaningful machine learning tasks.

This specialization course gives an introduction to neural networks. We will see how neural networks, being computational graphs, can be used to build very generic hypothesis spaces of nonlinear models. With these models we can solve regression, classification or other types of learning tasks by simply selecting an appropriate cost function. The course ends with a couple of guest lectures on topics from ML research and/or an industrial applications of neural networks.

The goal with the course is to add neural networks to your problem solving toolkit. After completing the course, you should: understand the basic concepts of neural networks; be able to develop and apply a neural network; understand the advantages and limitations of neural networks; be able to digest new ML research and continue developing your skillset by self-study.

• Part I – Machine learning basics (7 hours)
  • Course introduction and motivation – why learn from data?
  • Learning algorithms – how to learn from data?
  • Generalization and test sets
  • Capacity and overfitting
  • Hyper-parameters and validation sets
  • Maximum likelihood estimation – Linear and logistic regression
• Part II – Deep learning (8 hours)
  • Deep feed-forward neural networks
  • Back-propagation
  • Stochastic gradient descent
  • Regularization
  • Implementation and practice
  • Popular architectures
• Part III – Programming and case study (5 hours)
  • Programming in PyTorch (Python)
  • Case study: industrial application of neural networks
  • Orientation about the project and exam

The first lecture is scheduled to Monday, September 1, 10:15-12:00 AM.

Most, if not all, of the lectures will be held digitally on Zoom. Information about the lectures will be shared on Blackboard.

The lectures will be given in Norwegian, with slides written in English.

Students will be given a voluntary project assignment or topic to study.

The exam will consist of two parts: 1) an oral presentation of the assignment, and 2) questions from the syllabus.

Exam dates: TBD.

• Course in matrix algebra and calculus (e.g. TMA4115 - Calculus 3, or a similar course)
• Introductory course in statistics (e.g. TMA4240 – Statistics, or a similar course)
• Introductory course in mathematical optimization (e.g. TTK4135 - Optimization and Control, or a similar course)

• Lecture notes
• Deep learning by Goodfellow et al. (freely available at https://www.deeplearningbook.org/)
• Selection of papers (TBD)