TorchLean API

Docs HomeGuideExamplesGraphs

NN.API.Models.Transformer

Transformer Model Helpers (API) #

Small config-style Transformer constructors for runnable examples.

source
structure NN.API.nn.models.TransformerEncoderConfig :
Type

Configuration for a single Transformer encoder block over batched token embeddings.

Instances For
    source
    @[implicit_reducible]
    instance NN.API.nn.models.instReprTransformerEncoderConfig :
    Repr TransformerEncoderConfig
    source
    def NN.API.nn.models.instReprTransformerEncoderConfig.repr :
    TransformerEncoderConfig → ℕ → Std.Format
    Instances For
      source
      @[reducible, inline]
      abbrev NN.API.nn.models.transformerEncoderShape (cfg : TransformerEncoderConfig) :
      Shape
      Instances For
        source
        def NN.API.nn.models.transformerEncoder (cfg : TransformerEncoderConfig) (h_seqLen : cfg.seqLen ≠ 0 := by decide) (h_dModel : cfg.dModel ≠ 0 := by decide) :
        M (Sequential (transformerEncoderShape cfg) (transformerEncoderShape cfg))

        Build one Transformer encoder block with input/output shape (batch × seqLen × dModel).

        Instances For