TorchLean API

Docs HomeGuideExamplesGraphs

NN.API.Public.Facade.Runtime.Adapters

TorchLean Adapter Facade #

Parameter-efficient adapter operations exposed by the NN umbrella.

source
@[reducible, inline]
abbrev TorchLean.Adapters.LoRA.Params (α : Type) (inDim rank outDim : ℕ) :
Type

LoRA adapter parameters for a linear weight matrix of shape inDim × outDim.

The usual LoRA scaling is alpha / rank; TorchLean keeps the final scalar as an explicit scale argument so callers can choose that convention, a schedule, or a test value.

Instances For
    source
    def TorchLean.Adapters.LoRA.delta {α : Type} [Add α] [Mul α] [Zero α] {inDim rank outDim : ℕ} (p : Params α inDim rank outDim) (scale : α) :
    Spec.Tensor α (Spec.Shape.dim inDim (Spec.Shape.dim outDim Spec.Shape.scalar))

    The low-rank matrix scale * (A * B) added to a base linear weight.

    Instances For
      source
      def TorchLean.Adapters.LoRA.effectiveWeight {α : Type} [Add α] [Mul α] [Sub α] [Zero α] {inDim rank outDim : ℕ} (base : Spec.Tensor α (Spec.Shape.dim inDim (Spec.Shape.dim outDim Spec.Shape.scalar))) (p : Params α inDim rank outDim) (scale : α) :
      Spec.Tensor α (Spec.Shape.dim inDim (Spec.Shape.dim outDim Spec.Shape.scalar))

      Apply a LoRA adapter to a base linear weight matrix.

      Instances For
        source
        def TorchLean.Adapters.LoRA.linear {α : Type} [Add α] [Mul α] [Sub α] [Zero α] {batch inDim rank outDim : ℕ} (x : Spec.Tensor α (Spec.Shape.dim batch (Spec.Shape.dim inDim Spec.Shape.scalar))) (base : Spec.Tensor α (Spec.Shape.dim inDim (Spec.Shape.dim outDim Spec.Shape.scalar))) (p : Params α inDim rank outDim) (scale : α) :
        Spec.Tensor α (Spec.Shape.dim batch (Spec.Shape.dim outDim Spec.Shape.scalar))

        Run a batched linear projection using the base weight plus the LoRA delta.

        Instances For