TorchLean API

Docs HomeGuideExamplesGraphs

NN.Runtime.Autograd.Engine.Cuda.Ops.Elementwise

CUDA Tape Operations: Elementwise Nodes #

Elementwise ops #

The backward closures below return newly allocated gradient buffers. When a derivative uses intermediate CUDA buffers, it releases those intermediates before returning the final gradient. The returned buffers are owned by the tape/gradient accumulator; workspace buffers are owned locally.

source
def Runtime.Autograd.Cuda.Tape.add {s : Spec.Shape} (t : Tape) (aId bId : ℕ) :
Result (Tape × ℕ)

Pointwise addition node for two tensors with the same shape.

Instances For
    source
    def Runtime.Autograd.Cuda.Tape.sub {s : Spec.Shape} (t : Tape) (aId bId : ℕ) :
    Result (Tape × ℕ)

    Pointwise subtraction node for two tensors with the same shape.

    Instances For
      source
      def Runtime.Autograd.Cuda.Tape.mul {s : Spec.Shape} (t : Tape) (aId bId : ℕ) :
      Result (Tape × ℕ)

      Pointwise multiplication node for two tensors with the same shape.

      Instances For
        source
        def Runtime.Autograd.Cuda.Tape.scale {s : Spec.Shape} (t : Tape) (xId : ℕ) (c : Float) :
        Result (Tape × ℕ)

        Multiply by a scalar constant.

        Instances For
          source
          def Runtime.Autograd.Cuda.Tape.abs {s : Spec.Shape} (t : Tape) (xId : ℕ) :
          Result (Tape × ℕ)

          Pointwise absolute-value node.

          Instances For
            source
            def Runtime.Autograd.Cuda.Tape.sqrt {s : Spec.Shape} (t : Tape) (xId : ℕ) :
            Result (Tape × ℕ)

            Pointwise square-root node using the CUDA buffer derivative convention.

            Instances For
              source
              def Runtime.Autograd.Cuda.Tape.clamp {s : Spec.Shape} (t : Tape) (xId : ℕ) (lo hi : Float) :
              Result (Tape × ℕ)

              Clamp each element to [lo, hi].

              Instances For
                source
                def Runtime.Autograd.Cuda.Tape.max {s : Spec.Shape} (t : Tape) (aId bId : ℕ) :
                Result (Tape × ℕ)

                Pointwise maximum node; the backward rule splits ties according to Buffer.maxBwd.

                Instances For
                  source
                  def Runtime.Autograd.Cuda.Tape.min {s : Spec.Shape} (t : Tape) (aId bId : ℕ) :
                  Result (Tape × ℕ)

                  Pointwise minimum node; the backward rule splits ties according to Buffer.minBwd.

                  Instances For
                    source
                    def Runtime.Autograd.Cuda.Tape.div {s : Spec.Shape} (t : Tape) (aId bId : ℕ) :
                    Result (Tape × ℕ)

                    Pointwise division node with the usual quotient-rule backward closure.

                    Instances For
                      source
                      def Runtime.Autograd.Cuda.Tape.relu {s : Spec.Shape} (t : Tape) (xId : ℕ) :
                      Result (Tape × ℕ)

                      Pointwise ReLU node with zero derivative on the nonpositive branch.

                      Instances For
                        source
                        def Runtime.Autograd.Cuda.Tape.exp {s : Spec.Shape} (t : Tape) (xId : ℕ) :
                        Result (Tape × ℕ)

                        Pointwise exponential node; backward recomputes exp x as local workspace.

                        Instances For
                          source
                          def Runtime.Autograd.Cuda.Tape.log {s : Spec.Shape} (t : Tape) (xId : ℕ) :
                          Result (Tape × ℕ)

                          Pointwise natural-log node; callers are responsible for the positive-domain convention.

                          Instances For
                            source
                            def Runtime.Autograd.Cuda.Tape.inv {s : Spec.Shape} (t : Tape) (xId : ℕ) :
                            Result (Tape × ℕ)

                            Elementwise reciprocal 1/x.

                            Instances For
                              source
                              def Runtime.Autograd.Cuda.Tape.safeLog {s : Spec.Shape} (t : Tape) (xId : ℕ) (ε : Float) :
                              Result (Tape × ℕ)

                              Elementwise "safe log" that protects against log(0) by adding a small ε internally.

                              Spec semantics: log(softplus(x) + ε).

                              Instances For
                                source
                                def Runtime.Autograd.Cuda.Tape.sigmoid {s : Spec.Shape} (t : Tape) (xId : ℕ) :
                                Result (Tape × ℕ)

                                Elementwise sigmoid (logistic).

                                Instances For
                                  source
                                  def Runtime.Autograd.Cuda.Tape.tanh {s : Spec.Shape} (t : Tape) (xId : ℕ) :
                                  Result (Tape × ℕ)

                                  Pointwise hyperbolic tangent node.

                                  Instances For
                                    source
                                    def Runtime.Autograd.Cuda.Tape.softplus {s : Spec.Shape} (t : Tape) (xId : ℕ) :
                                    Result (Tape × ℕ)

                                    Pointwise softplus node with sigmoid derivative.

                                    Instances For