Introduction to Automatic Differentiation
- Wednesday, March 15th 2023, 18:00 - 19:30
On March 15th, we're gathering for an introductory discussion to automatic differentiation where Bogdan will be talking about what automatic differentiation is, provide some examples and implememntation details.
Machine learning has become an indispensable part of modern society and continues to have a significant societal, economic and scientific impact.
One of the pillars of machine learning is mathematical optimization, which involves the numerical computations of parameters for a system designed to make decisions based on as yet unseen data. More specifically, gradient-based optimization plays a critical role in many machine learning tasks. For example, training a neural network is an optimization problem with respect to its set of weights, while the backpropagation algorithm is a special case of reverse-mode automatic differentiation.
Modern deep learning frameworks provide differentiation capability in one way or another, but the underlying mechanism is not always made clear and the term "autodiff" seems to be surrounded by a certain air of mystery. This talk offers a tutorial introduction to automatic differentiation focusing on its history, mathematical formulation and basic building blocks.