RNN Text Example

Runnable torchlean rnn example. It reads a local text corpus, creates a byte-level causal-language-model window, and trains a vanilla RNN plus time-distributed linear head.

The model constructor lives in NN.API.Models.SimpleSeq so other examples can reuse it. This file keeps only the architecture-specific declarations; the shared corpus loading, CLI parsing, logging, and train loop live in NN.Examples.Models.Sequence.SimpleText.

What This Example Is (And Is Not)

This is a small layer smoke test for the vanilla RNN cell plus the TorchLean training loop. It uses a single fixed text window and a simple MSE-on-one-hot objective so it can run on CPU or CUDA quickly.

For an actual language-model walkthrough (longer context, sampling, etc.), use chargpt, gpt2, or text_gpt2.

python3 scripts/datasets/download_example_data.py --tiny-shakespeare
lake build -R -K cuda=true && lake exe torchlean rnn --cuda --tiny-shakespeare --steps 1

def NN.Examples.Models.Sequence.Rnn.exeName : String

def NN.Examples.Models.Sequence.Rnn.defaultLogJson : System.FilePath

def NN.Examples.Models.Sequence.Rnn.seqLen : ℕ

Short byte-window length used for a quick recurrent-model smoke test.

def NN.Examples.Models.Sequence.Rnn.inputSize : ℕ

Byte vocabulary size.

We use full byte tokens (0..255) so the sample is easy to interpret. If you want to make CPU runs even faster, you can lower this to 32, but you will be training on hashed bytes (byte % vocab).

def NN.Examples.Models.Sequence.Rnn.hiddenSize : ℕ

Hidden state width of the vanilla recurrent cell.

def NN.Examples.Models.Sequence.Rnn.cfg : API.nn.models.SeqRnnHeadConfig

Shared shape/config record consumed by the reusable API constructor.

@[reducible, inline]

abbrev NN.Examples.Models.Sequence.Rnn.σ : Shape

@[reducible, inline]

abbrev NN.Examples.Models.Sequence.Rnn.τ : Shape

def NN.Examples.Models.Sequence.Rnn.mkModel : API.nn.M (API.nn.Sequential σ τ)

def NN.Examples.Models.Sequence.Rnn.mkSample {α : Type} [API.Semantics.Scalar α] [API.Runtime.Scalar α] (input : String) : API.sample.Supervised α σ τ

Convert corpus text into one supervised causal sequence window.

def NN.Examples.Models.Sequence.Rnn.runner : SimpleText.RunnerConfig σ τ

Shared runner configuration for torchlean rnn.

def NN.Examples.Models.Sequence.Rnn.main (args : List String) : IO UInt32