TorchLean API

Docs HomeInstallationGuideExamplesGraphsTools

NN.Runtime.Autograd.Torch.Core.Ops.Layers

Eager Tensor Operations #

PyTorch-style tensor operations backed by the eager CPU/CUDA tapes. These wrappers record runtime nodes, dispatch CUDA kernels when requested, and preserve the typed TensorRef surface.

Neural-network layers #

source
def Runtime.Autograd.Torch.Internal.EagerSession.linear {α : Type} (s : EagerSession α) [Inhabited α] [Add α] [Mul α] [Zero α] [DecidableEq Spec.Shape] {inDim outDim : ℕ} (w : TensorRef α (Spec.Shape.dim outDim (Spec.Shape.dim inDim Spec.Shape.scalar))) (b : TensorRef α (Spec.Shape.dim outDim Spec.Shape.scalar)) (x : TensorRef α (Spec.Shape.dim inDim Spec.Shape.scalar)) :
IO (TensorRef α (Spec.Shape.dim outDim Spec.Shape.scalar))

Fully-connected linear layer y = w x + b. PyTorch: torch.nn.functional.linear.

Instances For
    source
    def Runtime.Autograd.Torch.Internal.EagerSession.mseLoss {α : Type} [CudaBridge.TensorConv α] (s : EagerSession α) [Inhabited α] [Add α] [Sub α] [Mul α] [Div α] [Zero α] [One α] [Coe ℕ α] [DecidableEq Spec.Shape] {sh : Spec.Shape} (yhat target : TensorRef α sh) :
    IO (TensorRef α Spec.Shape.scalar)

    Mean-squared-error loss returning a scalar. PyTorch: torch.nn.functional.mse_loss.

    Instances For
      source
      def Runtime.Autograd.Torch.Internal.EagerSession.layerNorm {α : Type} (s : EagerSession α) [Context α] [DecidableRel fun (x1 x2 : α) => x1 > x2] [DecidableEq Spec.Shape] {seqLen embedDim : ℕ} (h_seq_pos : seqLen > 0) (h_embed_pos : embedDim > 0) (x : TensorRef α (Spec.Shape.dim seqLen (Spec.Shape.dim embedDim Spec.Shape.scalar))) (gamma beta : TensorRef α (Spec.Shape.dim embedDim Spec.Shape.scalar)) :
      IO (TensorRef α (Spec.Shape.dim seqLen (Spec.Shape.dim embedDim Spec.Shape.scalar)))

      Layer normalization over embedding dimension. PyTorch: nn.LayerNorm / functional.layer_norm.

      Instances For
        source
        def Runtime.Autograd.Torch.Internal.EagerSession.batchnormChannelFirst {α : Type} (s : EagerSession α) [Context α] [DecidableRel fun (x1 x2 : α) => x1 > x2] [DecidableEq Spec.Shape] {channels height width : ℕ} (h_c : channels > 0) (h_h : height > 0) (h_w : width > 0) (x : TensorRef α (Spec.Shape.dim channels (Spec.Shape.dim height (Spec.Shape.dim width Spec.Shape.scalar)))) (gamma beta : TensorRef α (Spec.Shape.dim channels Spec.Shape.scalar)) :
        IO (TensorRef α (Spec.Shape.dim channels (Spec.Shape.dim height (Spec.Shape.dim width Spec.Shape.scalar))))

        BatchNorm for channel-first images (C,H,W) (no batch axis). PyTorch: nn.BatchNorm2d (conceptually).

        Instances For
          source
          def Runtime.Autograd.Torch.Internal.EagerSession.multiHeadAttention {α : Type} (s : EagerSession α) [Context α] [DecidableRel fun (x1 x2 : α) => x1 > x2] [DecidableEq Spec.Shape] {n numHeads dModel headDim : ℕ} (h1 : n ≠ 0) (wq wk wv : TensorRef α (Spec.Shape.dim dModel (Spec.Shape.dim (numHeads * headDim) Spec.Shape.scalar))) (wo : TensorRef α (Spec.Shape.dim (numHeads * headDim) (Spec.Shape.dim dModel Spec.Shape.scalar))) (x : TensorRef α (Spec.Shape.dim n (Spec.Shape.dim dModel Spec.Shape.scalar))) (mask : Option (Spec.Tensor Bool (Spec.Shape.dim n (Spec.Shape.dim n Spec.Shape.scalar))) := none) :
          IO (TensorRef α (Spec.Shape.dim n (Spec.Shape.dim dModel Spec.Shape.scalar)))

          Multi-head self-attention (typed, proof-friendly). PyTorch: nn.MultiheadAttention (conceptually).

          Instances For