Base Model
A model trained only on next-token prediction over a large corpus, before any instruction tuning.
PromptItIn team

What it means
A base model (also called a pretrained model) has learned the statistical structure of language from massive text corpora but has not been fine-tuned to follow instructions or adopt a helpful persona.
It continues text rather than answering questions — given 'The capital of France is', it predicts 'Paris' because that is the statistically expected continuation, not because it is trying to be helpful. Base models are the foundation on which instruction-tuned and RLHF-trained models are built.
They are useful for researchers studying model capabilities and for practitioners who want maximum flexibility before layering application-specific fine-tuning.
Examples
- GPT-3's original release was a base model; it required careful few-shot prompting rather than natural instruction-following.
- Meta's Llama 3 base weights are released openly so developers can fine-tune them for their own use cases.
Related terms
Pretraining
The initial phase of training a model on massive text data to learn general language representations.
Instruction-Tuned Model
A model fine-tuned on instruction-response pairs to follow natural-language directives reliably.
Fine-Tuned Model
A pretrained model whose weights have been updated on a specific dataset for a targeted task.
Put this into practice
PromptItIn applies techniques like base model for you: describe what you want and it builds the prompt.
You might also like