Normalization module wrappers #
This file wraps selected normalization specs as NNModuleSpecs for composition/export.
For LayerNormModuleSpec, we require (defaulted) proofs that dimensions are positive. This matches
the spec-level intent: normalization divides by the number of features and uses variance/standard
deviation, so degenerate "zero-width" cases are excluded when we want clean theorems.
PyTorch mental picture: nn.LayerNorm(embedDim) applied at each timestep, with weight=gamma and
bias=beta.
def
Spec.LayerNormModuleSpec
{α : Type}
[Context α]
(seqLen embedDim : ℕ)
(gamma beta : Tensor α (Shape.dim embedDim Shape.scalar))
(h_seq_pos : seqLen > 0)
(h_embed_pos : embedDim > 0)
:
ModSpec.NNModuleSpec α (Shape.dim seqLen (Shape.dim embedDim Shape.scalar))
(Shape.dim seqLen (Shape.dim embedDim Shape.scalar))
LayerNorm over the last dimension, wrapped as an NNModuleSpec.