Residual Linear Block

A small DAG model for x ↦ ReLU(Wx + b + x). The weight and bias have shapes [d, d] and [d]; both start at zero, so the initial model computes ReLU(x).

The input variable appears in both the linear branch and the skip branch. A let1 binds the linear result before the addition. Reusing an environment variable does not recompute a preceding input expression.

Read this alongside NN.GraphSpec.DAG.Core for typed variables and let1 semantics.

@[reducible, inline]

abbrev NN.GraphSpec.Models.ResidualLinearParams (d : ℕ) : List Spec.Shape

Parameter ABI for the residual block.

The layout is exactly:

• W : Tensor α [d, d]
• b : Tensor α [d]

The parameter-free skip path reuses the input x.

def NN.GraphSpec.Models.residualLinear (d : ℕ) : DAG.Model (ResidualLinearParams d) [[d]] [d]

Residual linear block in DAG form.

In ordinary math notation, this is

$$ x\mapsto\operatorname{ReLU}(Wx+b+x). $$

The same input variable is used by the linear operation and the residual addition.