Native CUDA Backend Capsules

Capsule metadata for TorchLean's native CUDA runtime provider.

These capsules describe kernels that currently live under csrc/cuda/** and are exposed to Lean through NN.Runtime.Autograd.Engine.Cuda.*. The C/CUDA source still owns the implementation; this module gives the planner a typed, inspectable contract layer for those implementation choices.

def NN.Backend.NativeCUDA.nativeCapsule (name : String) (op : BackendOp) (valueSummary vjpSummary : String) (vjpMode : VJPMode := VJPMode.backendVJP) : KernelCapsule

Build a checked native-CUDA capsule with explicit FFI, value, VJP, and layout contracts.

def NN.Backend.NativeCUDA.nativePointwiseCapsule (op : BackendOp) : KernelCapsule

Build the standard native-CUDA capsule for a pointwise operation.

def NN.Backend.NativeCUDA.nativeReductionCapsule (op : BackendOp) : KernelCapsule

Build a native-CUDA reduction capsule with implementation-selected parallel reduction order.

def NN.Backend.NativeCUDA.nativeAccumulationCapsule (name : String) (op : BackendOp) (valueSummary vjpSummary : String) (vjpMode : VJPMode := VJPMode.backendVJP) : KernelCapsule

Native CUDA kernel with an accumulation whose tree/order is selected by the implementation.

This covers matrix products, affine layers, convolutions, losses, and average pooling. CUDA may use parallel trees, fused multiply-add, cuBLAS/cuDNN algorithms, or architecture-specific schedules; the capsule therefore records the reduction as implementation-defined instead of pretending it is the reference left fold.

def NN.Backend.NativeCUDA.nativeViewCapsule (op : BackendOp) : KernelCapsule

Build the standard native-CUDA capsule for a shape or layout transformation.

def NN.Backend.NativeCUDA.nativeForwardOnlyCapsule (op : BackendOp) (valueSummary : String) : KernelCapsule

Build a native-CUDA forward-only capsule with no registered reverse derivative.

def NN.Backend.NativeCUDA.nativeConvPoolCapsule (op : BackendOp) : KernelCapsule

Build a native-CUDA capsule for channel-first convolution or pooling.

def NN.Backend.NativeCUDA.nativeSelectionCapsule (op : BackendOp) : KernelCapsule

Native window selection whose traversal and tie winner are chosen by the CUDA implementation.

def NN.Backend.NativeCUDA.matmul : KernelCapsule

Native CUDA batched/matrix multiplication, backed by CUDA/cuBLAS paths.

def NN.Backend.NativeCUDA.relu : KernelCapsule

Native CUDA ReLU activation.

def NN.Backend.NativeCUDA.gelu : KernelCapsule

Native CUDA GELU activation.

def NN.Backend.NativeCUDA.add : KernelCapsule

Native-CUDA pointwise addition.

def NN.Backend.NativeCUDA.sub : KernelCapsule

Native-CUDA pointwise subtraction.

def NN.Backend.NativeCUDA.mul : KernelCapsule

Native-CUDA pointwise multiplication.

def NN.Backend.NativeCUDA.scale : KernelCapsule

Native-CUDA scalar multiplication.

def NN.Backend.NativeCUDA.abs : KernelCapsule

Native-CUDA pointwise absolute value.

def NN.Backend.NativeCUDA.sqrt : KernelCapsule

Native-CUDA pointwise square root.

def NN.Backend.NativeCUDA.clamp : KernelCapsule

Native-CUDA pointwise interval clamp.

def NN.Backend.NativeCUDA.max : KernelCapsule

Native-CUDA pointwise maximum.

def NN.Backend.NativeCUDA.min : KernelCapsule

Native-CUDA pointwise minimum.

def NN.Backend.NativeCUDA.sigmoid : KernelCapsule

Native-CUDA pointwise sigmoid.

def NN.Backend.NativeCUDA.tanh : KernelCapsule

Native-CUDA pointwise hyperbolic tangent.

def NN.Backend.NativeCUDA.softplus : KernelCapsule

Native-CUDA pointwise softplus.

def NN.Backend.NativeCUDA.exp : KernelCapsule

Native-CUDA pointwise exponential.

def NN.Backend.NativeCUDA.sin : KernelCapsule

Native CUDA sine, with angles measured in radians.

def NN.Backend.NativeCUDA.cos : KernelCapsule

Native CUDA cosine, with angles measured in radians.

def NN.Backend.NativeCUDA.log : KernelCapsule

Native-CUDA pointwise natural logarithm.

def NN.Backend.NativeCUDA.inv : KernelCapsule

Native-CUDA pointwise reciprocal.

def NN.Backend.NativeCUDA.safeLog : KernelCapsule

Native-CUDA guarded logarithm used by numerically defensive programs.

def NN.Backend.NativeCUDA.logSoftmax : KernelCapsule

Native-CUDA log-softmax reduction and normalization.

def NN.Backend.NativeCUDA.softmax : KernelCapsule

Native CUDA row/axis softmax kernels.

def NN.Backend.NativeCUDA.hardMaskedSoftmax : KernelCapsule

Native CUDA hard-masked row softmax.

def NN.Backend.NativeCUDA.reduceSum : KernelCapsule

Native-CUDA sum reduction.

def NN.Backend.NativeCUDA.reduceMean : KernelCapsule

Native-CUDA arithmetic-mean reduction.

def NN.Backend.NativeCUDA.reshape : KernelCapsule

Native-CUDA shape-preserving reshape view.

def NN.Backend.NativeCUDA.permute : KernelCapsule

Native-CUDA axis permutation.

def NN.Backend.NativeCUDA.broadcast : KernelCapsule

Native-CUDA tensor broadcasting.

def NN.Backend.NativeCUDA.concat : KernelCapsule

Native-CUDA tensor concatenation.

def NN.Backend.NativeCUDA.slice : KernelCapsule

Native-CUDA contiguous tensor slice.

def NN.Backend.NativeCUDA.gather : KernelCapsule

Native-CUDA indexed gather.

def NN.Backend.NativeCUDA.scatterAdd : KernelCapsule

Native-CUDA indexed scatter-add.

def NN.Backend.NativeCUDA.randUniform : KernelCapsule

Native-CUDA seeded uniform-random tensor generation.

def NN.Backend.NativeCUDA.bernoulliMask : KernelCapsule

Native-CUDA seeded Bernoulli-mask generation.

def NN.Backend.NativeCUDA.layerNorm : KernelCapsule

Native CUDA layer normalization.

def NN.Backend.NativeCUDA.batchNorm : KernelCapsule

Native CUDA batch normalization.

def NN.Backend.NativeCUDA.conv : KernelCapsule

Native CUDA generic channel-first convolution.

def NN.Backend.NativeCUDA.convTranspose : KernelCapsule

Native CUDA generic channel-first transpose convolution.

def NN.Backend.NativeCUDA.maxPool : KernelCapsule

Native CUDA max pooling.

def NN.Backend.NativeCUDA.smoothMaxPool : KernelCapsule

Native CUDA smooth max pooling.

def NN.Backend.NativeCUDA.avgPool : KernelCapsule

Native CUDA average pooling.

def NN.Backend.NativeCUDA.linear : KernelCapsule

Native CUDA linear layer.

def NN.Backend.NativeCUDA.mseLoss : KernelCapsule

Native CUDA mean-squared-error loss.

def NN.Backend.NativeCUDA.fftFno : KernelCapsule

Native CUDA FFT/FNO kernels.

def NN.Backend.NativeCUDA.selectiveScan : KernelCapsule

Native CUDA selective scan kernels.

def NN.Backend.NativeCUDA.capsules : Array KernelCapsule

Native CUDA capsules, excluding attention which has a dedicated semantic split.