GPT-2 Saved-Weights Example

This is the load-and-sample half of the byte-level GPT example.

1. Train and save parameters:

lake build -R -K cuda=true torchlean:exe
lake exe -K cuda=true torchlean gpt2 --cuda --fast-kernels --tiny-shakespeare --steps 1 --windows 1 \
  --prompt "First Citizen:" --generate 0 \
  --save-params data/model_zoo/gpt2_shakespeare.params.json

2. Load the saved weights and sample text (no training loop, no optimizer state):

lake exe -K cuda=true torchlean gpt2_saved --cuda --fast-kernels \
  --params data/model_zoo/gpt2_shakespeare.params.json \
  --prompt "First Citizen:" --generate 0

What A Checkpoint Is Here

This example uses the simplest TorchLean checkpoint format:

• a shape-indexed pack of model parameters,
• stored as exact Float.toBits values in JSON, and
• checked against the model's parameter shapes before inference starts.

So save/load is model-agnostic: if we can name the model, TorchLean can compute the expected parameter shapes and reject stale or mismatched checkpoint files.

Why This Is A Separate Example

The inference-only workflow is direct: load a checkpoint, convert it into runtime parameter handles, and sample text without building a training loop.

def NN.Examples.Models.Sequence.Gpt2Saved.exeName : String

CLI subcommand name used in terminal banners and error messages.

def NN.Examples.Models.Sequence.Gpt2Saved.usage : String

Help text for checkpoint-only GPT-2 sampling.

def NN.Examples.Models.Sequence.Gpt2Saved.sampleWithSavedParams (load : API.text.SavedParamsGenerationOptions) : IO String

Load parameters from disk and run sampling with the fixed byte-level GPT architecture.

Important: the checkpoint must match Gpt2.model's parameter shapes. If the model configuration in Gpt2.lean changes (heads, width, layers, etc.), mismatched checkpoints fail the shape check before sampling starts.

def NN.Examples.Models.Sequence.Gpt2Saved.main (args : List String) : IO UInt32

CLI entrypoint for saved-parameter sampling.