RNN Text Example

Runnable torchlean rnn example. It reads a local text corpus, takes a short byte window from the front, and trains a vanilla RNN plus a time-distributed linear head.

The model constructor is exposed as TorchLean.nn.models.rnn. The local code names the architecture, builds the text dataset, and trains through the public Trainer surface.

Scope

This is the plain recurrent baseline. It keeps the text window short so the example stays focused on the recurrent cell, the time-distributed head, and the public Trainer API. For generation and longer contexts, use chargpt, gpt2, or text_gpt2.

python3 scripts/datasets/download_example_data.py --tiny-shakespeare
lake -R -K cuda=true exe torchlean rnn --device cuda --tiny-shakespeare --steps 1

def NN.Examples.Models.Sequence.Rnn.exeName : String

CLI subcommand name used in terminal banners and error messages.

def NN.Examples.Models.Sequence.Rnn.defaultLogPath : System.FilePath

Default JSON loss-curve path for this command.

def NN.Examples.Models.Sequence.Rnn.contextLength : ℕ

Number of byte-level timesteps in the training window.

def NN.Examples.Models.Sequence.Rnn.vocabularySize : ℕ

Tiny one-hot token width for the example dataset.

def NN.Examples.Models.Sequence.Rnn.defaultWindows : ℕ

Default number of distinct corpus windows exposed to the trainer.

theorem NN.Examples.Models.Sequence.Rnn.instNeZeroNatVocabularySize : NeZero vocabularySize

def NN.Examples.Models.Sequence.Rnn.byteBucket (id : ℕ) : Fin vocabularySize

Compact byte vocabulary: encode byte id b as b % 8; collisions are intentional.

def NN.Examples.Models.Sequence.Rnn.hiddenWidth : ℕ

Hidden state width of the vanilla recurrent cell.

@[reducible, inline]

abbrev NN.Examples.Models.Sequence.Rnn.modelConfig : TorchLean.nn.models.Recurrent.Config

Shared shape/config record for the reusable RNN-with-head constructor.

@[reducible, inline]

abbrev NN.Examples.Models.Sequence.Rnn.input : Spec.Shape

Input shape: one token vector per timestep.

@[reducible, inline]

abbrev NN.Examples.Models.Sequence.Rnn.output : Spec.Shape

Output shape: one prediction row per timestep.

def NN.Examples.Models.Sequence.Rnn.model : TorchLean.nn.Builder (TorchLean.nn.Sequential input output)

Vanilla RNN followed by a time-distributed linear output head.

def NN.Examples.Models.Sequence.Rnn.samples (corpus : String) (windows : ℕ) : TorchLean.Data.SampleStream (TorchLean.Sample.Supervised Float input output)

Build a finite next-token dataset from evenly spaced corpus windows.

def NN.Examples.Models.Sequence.Rnn.train (runtime : TorchLean.Runtime.Config) (data : RealData.TextWindowFlags) (flags : TorchLean.CLI.Training.OptimizerOptions) : IO Unit

Train the vanilla RNN with the public Trainer surface.

def NN.Examples.Models.Sequence.Rnn.main (args : List String) : IO UInt32

CLI entrypoint for the vanilla RNN text command.