ViT-Style Real-Data Example

Runnable torchlean vit example. It trains a compact ViT-style image classifier on a prepared CIFAR-10 minibatch: patch embedding by convolution, token reshape, transformer block, and linear head.

The reusable model wiring lives behind the public TorchLean.nn.models.vit constructor. The command adds CIFAR loader construction and the step-limited training loop.

python3 scripts/datasets/download_example_data.py --cifar10
lake -R -K cuda=true exe torchlean vit --device cuda --n-total 1 --steps 1

This command is a small runtime check. Larger image-token runs belong in runtime profiling work, not the default quick path.

def NN.Examples.Models.Vision.Vit.exeName : String

CLI subcommand name used in terminal banners and parser errors.

def NN.Examples.Models.Vision.Vit.defaultLogPath : System.FilePath

Default JSON loss-curve path for this command.

def NN.Examples.Models.Vision.Vit.batchSize : ℕ

Static minibatch size for the ViT example.

The batch axis is part of the checked model type, so changing this value changes the input and output shapes at compile time.

def NN.Examples.Models.Vision.Vit.inputChannels : ℕ

CIFAR image channels.

def NN.Examples.Models.Vision.Vit.cropHeight : ℕ

Height of the CIFAR crop used by this runnable ViT command.

def NN.Examples.Models.Vision.Vit.cropWidth : ℕ

Width of the CIFAR crop used by this runnable ViT command.

def NN.Examples.Models.Vision.Vit.patchHeight : ℕ

Patch height used by the convolutional patch embedding.

def NN.Examples.Models.Vision.Vit.patchWidth : ℕ

Patch width used by the convolutional patch embedding.

def NN.Examples.Models.Vision.Vit.stride : ℕ

Patch stride; equal to patch size here, so patches do not overlap.

def NN.Examples.Models.Vision.Vit.padding : ℕ

No zero-padding for the patch embedding.

def NN.Examples.Models.Vision.Vit.modelWidth : ℕ

Transformer feature width. The 4×4 crop yields four 2×2 image patches, plus the learned class token used by the classifier.

def NN.Examples.Models.Vision.Vit.classCount : ℕ

CIFAR class count, hence the output-logit width.

def NN.Examples.Models.Vision.Vit.attentionHeads : ℕ

Number of attention heads in each encoder block.

def NN.Examples.Models.Vision.Vit.attentionHeadWidth : ℕ

Per-head feature width; attentionHeads * attentionHeadWidth = modelWidth.

def NN.Examples.Models.Vision.Vit.feedForwardWidth : ℕ

Feed-forward hidden width inside the encoder block.

def NN.Examples.Models.Vision.Vit.transformerLayers : ℕ

Number of Transformer encoder blocks.

@[reducible, inline]

abbrev NN.Examples.Models.Vision.Vit.modelConfig : TorchLean.nn.models.ViT.Config 2

Shared ViT configuration used by shapes and the reusable public model constructor.

@[reducible, inline]

abbrev NN.Examples.Models.Vision.Vit.batch : Spec.Shape

Batch shape used by this training example.

@[reducible, inline]

abbrev NN.Examples.Models.Vision.Vit.input : Spec.Shape

Batched image shape derived from modelConfig.

@[reducible, inline]

abbrev NN.Examples.Models.Vision.Vit.output : Spec.Shape

Batched classifier output derived from modelConfig.

def NN.Examples.Models.Vision.Vit.model : TorchLean.nn.Builder (TorchLean.nn.Sequential input output)

Compact ViT-style classifier from the public model API.

The constructor builds patch embedding, token reshape, positional embeddings, the configured encoder stack, token pooling, and the classifier head.

def NN.Examples.Models.Vision.Vit.train (runtime : TorchLean.Runtime.Config) (flags : RealData.CifarModelTrainFlags) : IO TorchLean.Trainer.Report

Train the CIFAR ViT with the public Trainer surface.

def NN.Examples.Models.Vision.Vit.main (args : List String) : IO UInt32

CLI entrypoint for CIFAR ViT training on the selected runtime device.