PromptItIn logoPromptItIn
AI glossary · Training

Unsupervised Learning

Training a model on raw data without human-provided labels, discovering structure autonomously.

PromptItIn team

A bookshelf with one card pulled out

What it means

Unsupervised learning trains models on raw, unlabelled data, letting the model discover patterns, clusters, or representations without explicit guidance.

In language modelling, the dominant unsupervised approach is self-supervised learning: the model creates its own labels from the data (predict the next token; reconstruct a masked token) and learns by minimising prediction error. This enables training on effectively unlimited text data scraped from the internet.

The representations learned through unsupervised pretraining encode rich semantic and syntactic information that transfers to downstream supervised tasks. Autoencoding (BERT), causal language modelling (GPT), and contrastive learning (CLIP) are all self-supervised unsupervised techniques.

Examples

  • Word2Vec learning that 'king' − 'man' + 'woman' ≈ 'queen' purely from co-occurrence statistics in unlabelled text.
  • GPT-3's pretraining objective: predict the next token in a sequence, learning from 300 billion tokens with no human labelling.

Pretraining

The initial phase of training a model on massive text data to learn general language representations.

Supervised Learning

Training a model on input-output pairs where the correct output is provided as a label.

Embedding

A dense numerical vector that represents a token, sentence, or document in a continuous semantic space.

Put this into practice

PromptItIn applies techniques like unsupervised learning for you: describe what you want and it builds the prompt.

Try PromptItIn freeAdd to Chrome, free

You might also like

How to Use AI for Data Analysis →Prompts for Data Analysts →Data Analysis Prompts →Online Course Outline →Startup Financial Model →How to Use Role in AI Prompts →
← Browse the AI prompt glossary