See tokenization
Observe how training text becomes token IDs, sequences and model input.
Browser-native transformer lab
Train a tiny GPT inside the browser.
BabyGPT is an educational training engine that makes tokenization, transformer structure, optimization and text generation visible in one browser-native experiment.
Runs in the browser
Visual transformer learning
Designed for experimentation
Browser transformer
Text → tokens → attention → next token
Training locally
Training text
Context window: 8 tokens
the small model learns patterns one token at a time
the
small
model
learns
patterns
one
token
?
Token embeddings
8 × 32 vectors
Causal attention
2 attention heads
Block 1
Each token can attend only to itself and previous tokens.
Next-token prediction
at
probability 0.64
at
64%
by
21%
in
9%
Epoch 18 / 40
Loss 2.184
18K parameters
Runs entirely in the browser
Why BabyGPT exists
Large models hide the mechanics. BabyGPT exposes them.
Modern language models are too large to inspect directly. BabyGPT reduces the architecture to a scale where the complete learning process can be understood, changed and observed.
See the architecture
Understand embeddings, attention, feed-forward layers and output projection.
See learning happen
Follow loss, epochs and generated samples as the model learns statistical patterns.
Training flow
From text input to generated continuation.
The experiment follows the same conceptual stages as a larger autoregressive transformer, but at a scale suitable for learning and browser execution.
01
Load text
Provide a small training corpus directly in the browser.
02
Tokenize
Convert the text into a compact vocabulary and numerical sequence.
03
Forward pass
Produce next-token predictions through the transformer stack.
04
Optimise
Calculate loss, update weights and repeat across training batches.
05
Generate
Sample new text one token at a time from the trained model.
What you can change
Small parameters make cause and effect visible.
BabyGPT is designed for experimentation. Change one architectural or training variable and observe how the learning behaviour changes.
Context
Sequence length
Explore how much prior text the model can use for each prediction.
Architecture
Layers, heads and embeddings
Compare model capacity, training speed and generated output.
Training
Learning rate and epochs
Observe undertraining, instability, memorisation and gradual convergence.
Generation
Temperature and sampling
Change how conservative or diverse the generated continuation becomes.
What the lab demonstrates
The complete transformer loop in one inspectable environment.
Token embeddings
Learn how discrete token IDs become trainable vector representations.
Causal attention
See how each position reads only the tokens that came before it.
Transformer blocks
Follow attention, residual paths, normalization and feed-forward processing.
Next-token objective
Understand how the model learns by predicting the following token.
Loss trajectory
Observe whether training is converging, unstable or simply memorising.
Autoregressive generation
Watch the trained model build a sequence one prediction at a time.
Who it is for
A learning environment, not a production model.
BabyGPT is useful when the goal is to understand and experiment with transformer mechanics rather than deploy a capable general-purpose language model.
Students
Learn transformer fundamentals through direct interaction.
Educators
Demonstrate training and generation without large infrastructure.
Developers
Experiment with architecture and training choices at a manageable scale.
AI researchers
Prototype small educational or browser-native learning experiments.
Veritiana Labs
Understand the model by training one yourself.
BabyGPT reduces transformer learning to an inspectable browser experiment where architecture, training and generation remain visible.
Explore Veritiana projects