AI Glossary

Autonomous AI systems that can reason, plan, use tools, and take actions to accomplish goals. They combine LLMs with too...

A neural network mechanism that allows models to focus on relevant parts of the input when producing each output. Self-a...

A GPU-optimized quantization method that identifies and preserves the most important weights based on activation pattern...

The algorithm used to train neural networks. It calculates gradients of the loss function with respect to each weight by...

Processing multiple inference requests simultaneously to maximize GPU utilization. Continuous batching (used by vLLM) dy...

A fully managed AWS service for accessing foundation models (Claude, Llama, Titan) via API. Includes RAG (Knowledge Base...

Bidirectional Encoder Representations from Transformers. A pre-trained language model by Google that understands context...

A prompting technique that asks the model to show its reasoning step by step before giving a final answer. Significantly...

The process of splitting documents into smaller pieces (chunks) for storage in a vector database. Chunk size and strateg...

Continuous Integration and Continuous Deployment adapted for machine learning. Includes automated testing of data, model...

A neural network architecture designed for processing grid-like data (images, video). Uses convolutional filters to dete...

A platform for packaging applications into containers — lightweight, portable, isolated environments that include all de...

A simpler alternative to RLHF for aligning LLMs with human preferences. Directly optimizes the model using preference pa...

Dense numerical vector representations of text, images, or other data that capture semantic meaning. Similar items have ...

A centralized repository for storing, managing, and serving ML features. Ensures consistency between training and servin...

Providing a small number of examples (2-5) in the prompt to guide the model output format and behavior. A key prompt eng...

The process of further training a pre-trained model on domain-specific data to adapt it for a particular task, style, or...

A file format for storing quantized LLM weights that enables running large language models on consumer hardware (CPUs an...

A post-training quantization method for LLMs that uses approximate second-order information to minimize quantization err...

An optimization algorithm that iteratively adjusts model parameters in the direction that reduces the loss function. Var...

When an LLM generates confident-sounding but factually incorrect information. A fundamental limitation of generative mod...

The largest open-source AI platform. Hosts models, datasets, and spaces. Provides the Transformers library for using pre...

The process of running a trained model to generate predictions or outputs. In LLMs, inference means generating text toke...

An open-source container orchestration platform that automates deployment, scaling, and management of containerized appl...

A memory optimization for transformer inference that stores previously computed key-value pairs so they do not need to b...

A framework for building applications with LLMs. Provides abstractions for chains, agents, retrieval, memory, and tool u...

A C/C++ implementation for running LLM inference on consumer hardware. Created by Georgi Gerganov, it enables running mo...

A neural network with billions of parameters trained on massive text datasets to understand and generate human language....

A parameter-efficient fine-tuning technique that freezes the original model weights and trains small rank-decomposition ...

An open protocol that standardizes how AI models connect to external tools and data sources. Created by Anthropic, it pr...

The practice of deploying, monitoring, and maintaining machine learning models in production. Combines ML engineering, D...

When a deployed model performance degrades over time because the real-world data distribution changes from what the mode...

An open-source tool for running LLMs locally on your machine with a single command. Supports Llama, Mistral, Qwen, and o...

A family of techniques that fine-tune only a small subset of model parameters instead of all weights. Includes LoRA, QLo...

A managed vector database service for similarity search. Commonly used in RAG systems to store and retrieve document emb...

The practice of designing and optimizing input prompts to get desired outputs from LLMs. Techniques include zero-shot, f...

An open-source deep learning framework by Meta. The most popular framework for AI research and increasingly for producti...

A technique that combines 4-bit quantization with LoRA fine-tuning. Enables fine-tuning a 65B parameter model on a singl...

The process of reducing the precision of model weights (e.g., from 16-bit to 4-bit) to decrease model size and increase ...

A technique that enhances LLM responses by retrieving relevant documents from an external knowledge base and including t...

A second-stage retrieval step that re-scores and reorders initial search results using a more powerful model. Improves R...

A technique for aligning LLMs with human preferences by training a reward model on human comparisons, then using reinfor...

A fully managed AWS service for building, training, and deploying ML models. Provides notebooks, training jobs, model ho...

A parameter that controls randomness in LLM outputs. Temperature 0 gives deterministic (most likely) outputs. Higher val...

The process of splitting text into tokens (subwords, words, or characters) that a model can process. Common tokenizers i...

Using a model pre-trained on a large dataset as a starting point for a new task. Instead of training from scratch, you f...

A neural network architecture based on self-attention mechanisms. The foundation of all modern LLMs (GPT, BERT, Llama, M...

A database optimized for storing and querying high-dimensional vectors (embeddings). Used in RAG systems to find semanti...

An open-source library for high-throughput LLM inference and serving. Uses PagedAttention to manage GPU memory efficient...

Using a model to perform a task it was not explicitly trained for, without any examples. LLMs can do zero-shot classific...