Char-GPT (minGPT-style) Example

This example mirrors the classic "character-level GPT on a single text file" walkthrough popularized by Andrej Karpathy's minGPT/nanoGPT teaching material:

• build an alphabet (itos) from the training text,
• build a stoi tokenizer from that alphabet,
• train a compact causal Transformer to predict the next character,
• sample text continuations from a prompt.

It uses TorchLean's one-hot token interface (batch × seqLen × vocab) so the whole example stays in the same typed tensor world as the rest of the codebase.

Implementation note: training draws a fresh deterministic random window each step, following the minGPT/nanoGPT batching pattern. The --windows flag is accepted as a corpus-scale hint for shared scripts, but this command does not precompute a fixed window table.

Quick run:

lake build -R -K cuda=true torchlean:exe
lake exe -K cuda=true torchlean chargpt --cuda --tiny-shakespeare --steps 1 --batch 1 --seq-len 1 --generate 0

chargpt is the character-tokenizer teaching path. It rebuilds deterministic training windows from the corpus, so it is not part of the 10-step CUDA check tier. Use gpt2 or text_gpt2 for the compact GPT-style 10-step checks.

def NN.Examples.Models.Sequence.CharGpt.exeName : String

CLI subcommand name used in terminal banners and error messages.

def NN.Examples.Models.Sequence.CharGpt.takeInputText (args : List String) : IO (String × List String)

Parse corpus flags and return the UTF-8 training text plus remaining CLI arguments.

def NN.Examples.Models.Sequence.CharGpt.buildAlphabet (s : String) : Array Char

Build a deterministic character alphabet from the corpus.

def NN.Examples.Models.Sequence.CharGpt.defaultLogJson : System.FilePath

Default JSON loss-curve path for this command.

def NN.Examples.Models.Sequence.CharGpt.usage : String

Help text for character-level GPT training.

def NN.Examples.Models.Sequence.CharGpt.escapeCharIdsForDisplay (t : API.text.Tokenizer) (ids : List ℕ) : String

Decode token ids for terminal output with control characters escaped.

def NN.Examples.Models.Sequence.CharGpt.asciiAllowed (c : Char) : Bool

Printable-ASCII generation filter used by --ascii-only.

@[reducible, inline]

abbrev NN.Examples.Models.Sequence.CharGpt.Predictor (batch seqLen vocab : ℕ) : Type

Fitted predictor for a runtime-sized character GPT model.

def NN.Examples.Models.Sequence.CharGpt.generateSampledFromIds (batch seqLen vocab : ℕ) (predict : Predictor batch seqLen vocab) (promptIds : List ℕ) (steps : ℕ) (temperature : Float) (topK seed repeatWindow : ℕ) (repeatPenalty : Float) (allowId : ℕ → Bool := fun (x : ℕ) => true) (padId : ℕ := 0) : IO (List ℕ)

Autoregressively extend character token ids using a trained CharGPT model.

def NN.Examples.Models.Sequence.CharGpt.main (args : List String) : IO UInt32

CLI entrypoint for character-level GPT training and sampling.