What is Gradient Descent?
ML Fundamentals
Gradient Descent — An optimization algorithm that iteratively adjusts model parameters in the direction that reduces the loss function. Variants include SGD, Adam, and AdamW.
FAQ
What is gradient descent?
An algorithm that finds the minimum of a function by repeatedly taking steps in the direction of steepest decrease (negative gradient).
What is Adam optimizer?
An adaptive gradient descent variant that maintains per-parameter learning rates. The default choice for most deep learning training.
Related Terms
Learn Gradient Descent in depth
Free hands-on course with code examples and Google Colab notebooks.
Start Course →