LSTM Text Example

Runnable torchlean lstm example. It reads a local text corpus, takes a short byte window from the front, and trains an LSTM plus a time-distributed linear head.

The model constructor is exposed as TorchLean.nn.models.lstm. The local code names the architecture, builds the text dataset, and trains through the public Trainer surface.

Scope

This is the gated recurrent baseline. It keeps the text window short so the example stays focused on the LSTM cell, the time-distributed head, and the public Trainer API. For generation and longer-context language-model behavior, use one of:

• torchlean chargpt (Karpathy-style, single-file char-level GPT),
• torchlean gpt2 (byte-level GPT-2-style model + save/reload),
• torchlean text_gpt2 (CUDA corpus trainer).

python3 scripts/datasets/download_example_data.py --tiny-shakespeare
lake -R -K cuda=true exe torchlean lstm --device cuda --tiny-shakespeare --steps 1

def NN.Examples.Models.Sequence.Lstm.exeName : String

CLI subcommand name used in terminal banners and error messages.

def NN.Examples.Models.Sequence.Lstm.defaultLogPath : System.FilePath

Default JSON loss-curve path for this command.

def NN.Examples.Models.Sequence.Lstm.contextLength : ℕ

Number of byte-level timesteps in the training window.

def NN.Examples.Models.Sequence.Lstm.vocabularySize : ℕ

Tiny one-hot token width for the example dataset.

def NN.Examples.Models.Sequence.Lstm.defaultWindows : ℕ

Default number of distinct corpus windows exposed to the trainer.

theorem NN.Examples.Models.Sequence.Lstm.instNeZeroNatVocabularySize : NeZero vocabularySize

def NN.Examples.Models.Sequence.Lstm.byteBucket (id : ℕ) : Fin vocabularySize

Compact byte vocabulary: encode byte id b as b % 8; collisions are intentional.

def NN.Examples.Models.Sequence.Lstm.hiddenWidth : ℕ

Hidden state width of the LSTM cell.

@[reducible, inline]

abbrev NN.Examples.Models.Sequence.Lstm.modelConfig : TorchLean.nn.models.Recurrent.Config

Shared shape/config record for the reusable LSTM-with-head constructor.

@[reducible, inline]

abbrev NN.Examples.Models.Sequence.Lstm.input : Spec.Shape

Input shape: one token vector per timestep.

@[reducible, inline]

abbrev NN.Examples.Models.Sequence.Lstm.output : Spec.Shape

Output shape: one prediction row per timestep.

def NN.Examples.Models.Sequence.Lstm.model : TorchLean.nn.Builder (TorchLean.nn.Sequential input output)

LSTM followed by a time-distributed linear output head.

def NN.Examples.Models.Sequence.Lstm.samples (corpus : String) (windows : ℕ) : TorchLean.Data.SampleStream (TorchLean.Sample.Supervised Float input output)

Build a finite next-token dataset from evenly spaced corpus windows.

def NN.Examples.Models.Sequence.Lstm.train (runtime : TorchLean.Runtime.Config) (data : RealData.TextWindowFlags) (flags : TorchLean.CLI.Training.OptimizerOptions) : IO Unit

Train the LSTM with the public Trainer surface.

def NN.Examples.Models.Sequence.Lstm.main (args : List String) : IO UInt32

CLI entrypoint for the LSTM text command.