What is Attention?
Deep Learning
Attention Mechanism — A neural network mechanism that allows models to focus on relevant parts of the input when producing each output. Self-attention is the core of transformer architecture.
FAQ
What is attention in AI?
A mechanism that lets the model weigh the importance of different input tokens when generating each output token.
What is self-attention?
When a sequence attends to itself — each token looks at all other tokens to understand context. This is the core of transformers.
Related Terms
Learn Attention in depth
Free hands-on course with code examples and Google Colab notebooks.
Start Course →