Instructors: Anastasios Lekkas (NTNU) and Francesco Scibilia (Equinor/NTNU)

Updated Info: The course starts on 9 October 2019, 10:15-13:00, Room R52 (previously R53). Note: Lecture 7 is in room S21, Monday Nov. 18.

This specialization course will present an introduction to autonomous robots from both the academic and industrial viewpoints. For the academic part, emphasis will be given to recent advances in deep reinforcement learning, which combines deep neural networks with reinforcement learning to provide a framework for discovering suitable control actions (policies) and addressing complex tasks without explicit programming. For the industry‐focused lectures, aspects of artificial intelligence and autonomous robotics systems will be considered from industrial domain perspectives as inspection and maintenance.

Lecture 1 (09/10/19, 10:15-13:00, R52):

• Course introduction: Objectives, overview of lectures, assignments, exam info.
• Autonomy in the scope of the course. An introduction to reinforcement learning and its connection with dynamic programming.
• Markov Decision Processes and problem formulation

Lecture 2 (16/10/19, 10:15-13:00, R52):

• Optimal policies in discrete environments (Part 1): Value iteration and policy iteration
• Optimal policies in discrete environments (Part 2): Q-learning and SARSA

Lecture 3 (23/10/19, 10:15-13:00, R52):

• Deep learning and deep reinforcement learning.
• The DQN algorithm.
• Introduction to policy gradient algorithms for continuous action and state spaces. REINFORCE algorithm

Lecture 4 (30/10/19, 10:15-12:00, R52):

• Deep deterministic policy gradients (DDPG)
• Model predictive control and reinforcement learning (by Prof. Sebastien Gros)

Lecture 5 (06/11/19, 10:15-12:00, R52): Artificial intelligence in autonomous robotics systems: what is an actionable definition in an industrial setting. Different levels of autonomy. Hierarchical architecture (Sense‐plan‐act and behaviorbased substrates) and autonomy layers. A control theory approach to planning and action – Model Predictive Control.

Lecture 6 (13/11/19, 10:15-12:00, R52): Autonomous mobile robotics systems – environment perception vs mission sensors, localization and mapping, navigation and sense & avoid. Considerations on implementing autonomous systems in industrial applications as: system integration, ethics concerns, solutions fit to existing organization and systems, data management and intellectual property aspects.

Lecture 7 (18/11/19, 10:15-12:00, S21): More info to be announced soon.

28 & 29 November