Advanced Topics
This chapter strings the conclusions of the previous chapters into a more complete theory, and extends them to practical optimisation problems.
The Hessian matrix and the theory of extrema handle higher-order expansions and the classification of critical points systematically; the implicit function theorem shows that, under suitable conditions, the implicit function determined by an equation is also differentiable, which provides the theoretical foundation for implicit curves and constrained optimisation; and gradient descent is what actually puts these theories to work — it explains the choice of step size, the rate of convergence, and why one can get stuck in a local minimum rather than reaching the global optimum.
If your goal is to apply this course to machine learning or optimisation problems, this is the section most worth returning to.
