Foundation Model
A large model trained on broad data that can be adapted to many downstream tasks.
Full Definition
Foundation model is a term coined by Stanford's CRFM in 2021 to describe large models trained on diverse, massive datasets that serve as a general base — a foundation — for a wide range of downstream applications through fine-tuning or prompting. GPT-4, Claude, Gemini, and Llama are all foundation models. The concept captures the paradigm shift from training task-specific models from scratch to adapting a single powerful general model. Foundation models exhibit emergent capabilities (abilities not present in smaller models) and enable rapid prototyping of AI applications without task-specific training data, but they also concentrate risk: flaws in the foundation propagate to all downstream uses.
Examples
Using GPT-4 as a foundation and fine-tuning it on medical records to build a clinical note summarisation tool.
Anthropic's Claude serving as a foundation for hundreds of third-party chatbots and automation tools via API.
Apply this in your prompts
PromptItIn automatically uses techniques like Foundation Model to build better prompts for you.
You might also like
Related Terms
Large Language Model
A neural network with billions of parameters trained on text to understand and g…
View →Pretraining
The initial phase of training a model on massive text data to learn general lang…
View →Fine-Tuned Model
A pretrained model whose weights have been updated on a specific dataset for a t…
View →