Math for ML

math_for_ml.md

Calculus

Matrix derivatives

• A good collection of links on matrix derivatives https://gist.github.com/yig/c1959ce997f1d2fd6f3f982cb482e060

• A gentle intro. This series gives an overview of gradient, Jacobian and importance of chain rule for NN

  • https://www.lesswrong.com/posts/9L9XuXhLYBm47yYkf/a-primer-on-matrix-calculus-part-1-basic-review
  • https://www.lesswrong.com/s/nMGrhBYXWjPhZoyNL/p/KKwv9kcQz29vqPLAD
  • https://www.lesswrong.com/s/nMGrhBYXWjPhZoyNL/p/u4DWp72GyLsQn24oR

• "Matrix calculus you need"

  • https://arxiv.org/pdf/1802.01528.pdf or https://explained.ai/matrix-calculus/

Linear algebra

• Main facts and a lot of examples, but miss some concepts like eigendecomposition
  • http://vmls-book.stanford.edu
• Focus on methods from NLA, so some advanced topics and detailed descriptions can be skipped in first reading
  • http://www.stat.uchicago.edu/~lekheng/courses/309/books/Trefethen-Bau.pdf
• Matrix cookbook - These pages are a collection of facts (identities, approxima- tions, inequalities, relations, ...) about matrices and matters relating to them.
  • https://www.math.uwaterloo.ca/~hwolkowi/matrixcookbook.pdf