Advanced Topics

This chapter strings the conclusions of the previous chapters into a more complete theory, and extends them to practical optimisation problems.

The Hessian matrix and the theory of extrema handle higher-order expansions and the classification of critical points systematically; the implicit function theorem shows that, under suitable conditions, the implicit function determined by an equation is also differentiable, which provides the theoretical foundation for implicit curves and constrained optimisation; and gradient descent is what actually puts these theories to work — it explains the choice of step size, the rate of convergence, and why one can get stuck in a local minimum rather than reaching the global optimum.

If your goal is to apply this course to machine learning or optimisation problems, this is the section most worth returning to.

Chapters

Found an issue with this content?

Tell us what could be improved.