Deterministic parameter initialization

Pure, reproducible initializers for TorchLean.Tensor Float. These definitions are used when building model parameters before they enter a runtime session. Large runtime backends may provide more specialized allocation paths, but they should implement the same initialization scheme.

The formulas follow the corresponding PyTorch initializers:

• Scheme.xavierUniform uses the Glorot bound sqrt (6 / (fanIn + fanOut));
• Scheme.kaimingUniform uses the ReLU-oriented He bound sqrt (6 / fanIn).

References:

• Glorot and Bengio, Understanding the difficulty of training deep feedforward neural networks, AISTATS 2010.
• He et al., Delving Deep into Rectifiers, ICCV 2015.
• PyTorch initialization reference: https://pytorch.org/docs/stable/nn.init.html

inductive Runtime.Autograd.Torch.Init.Scheme : Type

A deterministic scheme for initializing a tensor of Float values.

• zeros : Scheme
• ones : Scheme
• uniform (lo hi : Float) : Scheme
• normal (mean std : Float) : Scheme
• xavierUniform (fanIn fanOut : ℕ) : Scheme
• kaimingUniform (fanIn : ℕ) : Scheme

@[instance_reducible]

instance Runtime.Autograd.Torch.Init.instReprScheme : Repr Scheme

def Runtime.Autograd.Torch.Init.instReprScheme.repr : Scheme → ℕ → Std.Format

def Runtime.Autograd.Torch.Init.Scheme.validate : Scheme → Except String Unit

Reject initializer parameters that would produce invalid floating-point samples.

def Runtime.Autograd.Torch.Init.xavierUniformLimit (fanIn fanOut : ℕ) : Float

Xavier/Glorot uniform bound.

The zero-fan case has no meaningful random interval. Returning zero keeps direct scheme use finite and agrees with the empty-tensor behavior expected when a weight has no elements.

def Runtime.Autograd.Torch.Init.kaimingUniformLimit (fanIn : ℕ) : Float

Kaiming/He uniform bound.

A zero fan-in uses the degenerate interval [0, 0], avoiding infinities and NaN in host and runtime initialization paths.

def Runtime.Autograd.Torch.Init.sampleAt (sch : Scheme) (seed idx : ℕ) : Float

Return sample idx from sch, using the counter-based stream determined by seed.

def Runtime.Autograd.Torch.Init.tensor (sch : Scheme) (seed : ℕ := 0) {s : Spec.Shape} : TorchLean.Tensor Float s

Initialize the row-major buffer directly, avoiding recursive subtensor construction. Flat index i still receives sampleAt sch seed i, preserving the seed and sample order.

def Runtime.Autograd.Torch.Init.xavierUniform (outDim inDim : ℕ) (seed : ℕ := 0) : TorchLean.Tensor Float [outDim, inDim]

Initialize a matrix with the Xavier/Glorot uniform distribution and gain 1.

def Runtime.Autograd.Torch.Init.kaimingUniform (outDim inDim : ℕ) (seed : ℕ := 0) : TorchLean.Tensor Float [outDim, inDim]

Initialize a matrix with the Kaiming/He uniform distribution for ReLU networks.