TorchLean API

NN.Runtime.Autograd.Engine.Cuda.Ops.SelectiveScan

Native diagonal scan tape nodes #

Both coefficient layouts return all states after their updates. Their backward closures retain the forward states and accumulate into all four parents. The variable-coefficient kernel returns one gradient per token; the shared-coefficient kernel sums those contributions across time. Each native thread owns one state channel and traverses time in order.

def Runtime.Autograd.Cuda.Tape.Internal.diagonalScan (variableCoefficients : Bool) {seqLen state : } (t : Tape) (aId bId xId initialId : ) :

Common recording and shape checks for the two diagonal-scan coefficient layouts.

Instances For
    def Runtime.Autograd.Cuda.Tape.selectiveScanDiag {seqLen state : } (t : Tape) (aId bId xId initialId : ) :

    Record the recurrence with coefficient vectors shared across time.

    Instances For
      def Runtime.Autograd.Cuda.Tape.selectiveScanDiagVar {seqLen state : } (t : Tape) (aId bId xId initialId : ) :

      Record the recurrence with independent coefficient rows for every token.

      Instances For