Positional Encodings

Configuration records for learned positions, fixed sinusoidal encodings, and rotary positional embeddings. The seeded constructors live in NN.API.Seeded.

structure TorchLean.nn.LearnedPositionalEmbedding.Config : Type

Learned positional embedding configuration.

This is a trainable parameter tensor of shape (sequenceLength × embeddingWidth) that is broadcast across batchShape and added to the input.

• initialization : Init.Scheme

  Initialization scheme for the positional embedding table.

def TorchLean.nn.LearnedPositionalEmbedding.Config.validate (config : Config) (sequenceLength embeddingWidth : ℕ) : Except String Unit

Validate table dimensions and initialization.

structure TorchLean.nn.SinusoidalPositionalEncoding.Config : Type

Sinusoidal positional encoding configuration.

Classic non-trainable Transformer sinusoidal encoding, added to token embeddings. startPosition is an absolute-position offset for KV-cache decoding.

• startPosition : ℕ

  Absolute position offset for the first row of the encoding table.

def TorchLean.nn.SinusoidalPositionalEncoding.Config.validate (_config : Config) (sequenceLength embeddingWidth : ℕ) : Except String Unit

Validate the tensor dimensions before materializing the fixed encoding buffer.

structure TorchLean.nn.RotaryEmbedding.Config : Type

Rotary positional embedding (RoPE) configuration.

startPosition is an absolute-position offset for KV-cache decoding.

• startPosition : ℕ

  Absolute position offset for the first row of RoPE angles.

def TorchLean.nn.RotaryEmbedding.Config.validate (_config : Config) (sequenceLength headWidth : ℕ) : Except String Unit

Validate the tensor dimensions before materializing the fixed rotation buffers.