Deterministic parameter initialization

Pure, reproducible initializers for TorchLean.Tensor Float. These definitions are used when building model parameters before they enter a runtime session. Large runtime backends may provide more specialized allocation paths, but they should implement the same initialization scheme.

The formulas follow the corresponding PyTorch initializers:

• Scheme.xavierUniform uses the Glorot bound sqrt (6 / (fanIn + fanOut));
• Scheme.kaimingUniform uses the ReLU-oriented He bound sqrt (6 / fanIn).

References:

• Glorot and Bengio, Understanding the difficulty of training deep feedforward neural networks, AISTATS 2010.
• He et al., Delving Deep into Rectifiers, ICCV 2015.
• PyTorch initialization reference: https://pytorch.org/docs/stable/nn.init.html

inductive Runtime.Autograd.Torch.Init.Scheme : Type

A deterministic scheme for initializing a tensor of Float values.

• zeros : Scheme
• ones : Scheme
• uniform (lo hi : Float) : Scheme
• normal (mean std : Float) : Scheme
• xavierUniform (fanIn fanOut : ℕ) : Scheme
• kaimingUniform (fanIn : ℕ) : Scheme

def Runtime.Autograd.Torch.Init.instReprScheme.repr : Scheme → ℕ → Std.Format

@[instance_reducible]

instance Runtime.Autograd.Torch.Init.instReprScheme : Repr Scheme

def Runtime.Autograd.Torch.Init.Scheme.validate : Scheme → Except String Unit

Check the parameter domains of a declared initialization scheme.

def Runtime.Autograd.Torch.Init.Internal.uniformLimit (fan : ℕ) (denominator : Float) : Float

Preserve the ordinary bound unless converting the fan count erased a nonzero root.

The denominator retains each scheme's original casting and addition order. On range loss, FloatLib rounds the square root of the exact rational ratio without casting the count first.

def Runtime.Autograd.Torch.Init.xavierUniformLimit (fanIn fanOut : ℕ) : Float

Xavier/Glorot uniform bound.

The zero-fan case has no meaningful random interval. Returning zero keeps direct scheme use finite and agrees with the empty-tensor behavior expected when a weight has no elements. A count that overflows binary64 is handled before rounding the root, which may still be representable.

def Runtime.Autograd.Torch.Init.kaimingUniformLimit (fanIn : ℕ) : Float

Kaiming/He uniform bound.

A zero fan-in uses the degenerate interval [0, 0], avoiding infinities and NaN in host and runtime initialization paths. Overflowing counts use the same exact-root recovery as Xavier.

def Runtime.Autograd.Torch.Init.sampleAt (sch : Scheme) (seed idx : ℕ) : Float

Return sample idx from sch, using the counter-based stream determined by seed.

Uniform sampling keeps the ordinary affine formula unless the span of two finite bounds overflows. Interpolating the endpoints separately then avoids materializing that span.

def Runtime.Autograd.Torch.Init.tensor (sch : Scheme) (seed : ℕ := 0) {s : Spec.Shape} : TorchLean.Tensor Float s

Initialize the row-major buffer directly, avoiding recursive subtensor construction. Flat index i still receives sampleAt sch seed i, preserving the seed and sample order.

def Runtime.Autograd.Torch.Init.xavierUniform (outDim inDim : ℕ) (seed : ℕ := 0) : TorchLean.Tensor Float [outDim, inDim]

Initialize a matrix with the Xavier/Glorot uniform distribution and gain 1.