-
Gradient descent variants on one loss surface: momentum, RMSProp, Adam visualized
Imagine four hikers trying to reach the bottom of the same mountain valley. Each can measure the slope beneath their feet, but none has a complete map of the terrain.
-
Derivatives and gradients: the one calculus lesson ML actually uses
Machine learning can look like a huge collection of algorithms, architectures, and optimization tricks. Underneath most of them, however, sits one compact idea: