Shared definitions for the graph CROWN engine.
This file contains the rank-one tensor representation, parameter stores, interval boxes, shape permutation helpers, and tensor casts used by the IBP, derivative, affine, CROWN, and backward objective passes.
An existentially sized rank-one tensor used by the flat LiRPA engine.
Graph nodes carry dimensions discovered while lowering, so the dimension cannot always appear in
the surrounding static type. The field n is the hidden tensor dimension, not duplicate runtime
storage.
- n : ℕ
Number of scalar entries.
- v : TorchLean.Tensor α [self.n]
Rank-one tensor payload (shape
.dim n .scalar).
Instances For
Coordinate on axis corresponding to a row-major flat index.
Instances For
Eval BatchNorm scale for one channel.
Instances For
Eval-mode BatchNorm bias after folding one channel's running statistics into an affine map.
Instances For
Build the exact diagonal affine form for eval-mode BatchNorm on an arbitrary channel axis.
The graph records the channel axis while the payload stores one scale and bias per channel. A malformed axis or mismatched channel extent has no verifier transfer rule.
Instances For
Parameters keyed by node id (weights, biases, constants, and seeded input boxes).
This is kept compact: it is the graph interpreter used to run IBP/CROWN on a pure Graph
without pulling in a heavyweight runtime.
- inputBoxes : Std.HashMap ℕ (FlatBox α)
Seed boxes for designated input nodes (
id -> FlatBox). - constVals : Std.HashMap ℕ (FlatTensor α)
Constants (
id -> FlatTensor). - linearWB : Std.HashMap ℕ (LinParams α)
Linear layer params (
id -> (W,b)). - matmulW : Std.HashMap ℕ (MatParams α)
Matmul params (
id -> W) for bias-free multiplication. - convCfg : Std.HashMap ℕ (IR.ConvParams α)
Convolution specs (
id -> convolution configuration). - batchNormEval : Std.HashMap ℕ (IR.BatchNormEvalParams α)
Eval-mode BatchNorm parameters (
id -> gamma/beta/running stats). - layerNorm : Std.HashMap ℕ (IR.LayerNormParams α)
Affine LayerNorm parameters keyed by node id.
Last-axis value bounds use the stored scale, bias, and epsilon. An absent payload selects unit scale, zero bias, and default epsilon. Payload derivative bounds remain unresolved until a rule accounts for all three parameters.
Instances For
Insert an input interval box for a graph node.
Instances For
Seed a graph input with a uniform ℓ∞ box around a shaped tensor.
Instances For
Whether the current dense convolution transfer exactly matches the corresponding IR semantics.
The flattened CROWN rule implements channel-first convolution without leading batch axes, channel groups, dilation, or asymmetric padding. The payload must also agree with the configuration stored in the graph node; otherwise executable IR evaluation and bound propagation would denote different operators.
Instances For
Check the graph-level semantic restrictions imposed by the current CROWN engine.
Unsupported convolutions, non-leading-axis concatenation, and LayerNorm over more than the last axis are left without bounds. LayerNorm payloads must match that axis's shape exactly. This predicate checks the common shape contract; individual transfers also check their arithmetic and derivative requirements.
Instances For
Whether every node in a graph is interpreted exactly by the current CROWN engine.
Instances For
Default inhabitant for FlatBox (a 0-dimensional box at 0).
Elementwise product of two FlatBoxes (interval product per component). Requires equal dims.
Instances For
Chain-rule multiplication for derivative intervals. Returns none on dimension mismatch.
Instances For
Convert a dependent Box of shape .dim n .scalar into a FlatBox with dim := n.
Instances For
Convert a FlatBox to a dependent Box at shape .dim B.dim .scalar.
Instances For
Add two flat interval boxes coordinatewise; dimension mismatches preserve the left box.
Instances For
Interval subtraction on FlatBox endpoints (sound enclosure).
Instances For
Sum all coordinates of a flat box with directed accumulation.
Instances For
Average all coordinates of a nonempty flat box with directed division.
Instances For
Apply ReLU to both endpoints of a FlatBox (monotone activation, so endpoints suffice).
Instances For
Componentwise absolute value bounds. Soundly encloses abs over each interval component.
Instances For
traverseFin is plumbing, so it lives in Internal like the rest of the codebase's plumbing. This
namespace used to be called boxUnaryEnclosure, after the function below that is its only caller,
which was misleading twice over: that caller is actually spelled boxUnaryEnclosure?, and
traverseFin knows nothing about boxes or enclosures.
traverseFin succeeds exactly when every component does, and then returns those components.
This is the only fact the box operations need about it: it lets a coordinatewise enclosure argument
be read off from the aggregate Option without ever mentioning the dif in the definition.
Apply a scalar interval enclosure coordinatewise to a flat box.
Instances For
Componentwise square-root bounds, failing when a coordinate interval reaches below zero.
Instances For
Softplus bounds, applied independently at every tensor coordinate.
Instances For
SafeLog bounds with one shared scalar epsilon interval.
The epsilon parent must contain one scalar. Keeping this check here makes direct graph construction follow the same contract as the typed builder and IR shape inference.
Instances For
Componentwise reciprocal bounds, failing when an input coordinate interval crosses zero.
Instances For
Derivative range for exp; exp' = exp.
Instances For
Derivative range for log; log' x = 1/x on a strictly positive interval.
Instances For
Second-derivative range for log; log'' x = -1/x² on a positive interval.
Instances For
Negate an interval box by swapping and negating its endpoints.
Instances For
Apply a full axis permutation to a shape-tagged tensor when the permutation is valid.
Instances For
Return the flat-coordinate permutation induced by an axis permutation.
perm follows the tensor convention used by Shape.permute?: output axis i is read from input
axis perm[i]. The resulting function therefore maps each output flat coordinate to the input
flat coordinate from which its value is taken. Invalid permutations and inconsistent dimensions
are rejected.
Instances For
Componentwise max bounds: max(x,y) over interval boxes.
Instances For
Componentwise min bounds: min(x,y) over interval boxes.
Instances For
Componentwise square of an interval box: for each component [l,u] produce [min (l^2,u^2), max (l^2,u^2)], with 0 as the minimum when the interval crosses 0.
The body is exposed because the proof layer theorem module unfolds this executable rule when proving dimension preservation and pointwise enclosure.
Instances For
Interval multiplication for scalar endpoints: given [aLo,aHi] and [bLo,bHi], return bounds
on the product.
Instances For
Length of the last axis of a shape; scalars are treated as length one.
Instances For
Reinterpret a flattened tensor as shape s when the element counts agree.
Instances For
IBP rule for broadcasting a flattened input box to a target shape.
Instances For
IBP rule for reducing a shaped box by summing along one axis.
Instances For
IBP rule for reducing a shaped box by averaging along one axis.
Instances For
Format-independent softmax enclosure on a flattened tensor.
A singleton row is exactly one. Every coordinate of a longer row lies in [0,1]. This deliberately
forgoes the tighter exponential formula above so executable checking does not assume a directed
transcendental implementation that its scalar backend has not supplied.
Instances For
Hard-masked softmax IBP (last axis) #
Blocked coordinates have weight zero. An allowed coordinate lies in [0,1], and it has weight one
when it is the only allowed coordinate in its row. These bounds do not evaluate exp or division,
so they remain valid for executable endpoint types whose BoundOps instance covers only directed
arithmetic. A tighter transfer rule requires separately certified directed bounds for
transcendental operations.
Conservative interval bounds for hard-masked softmax along the last tensor axis.
Instances For
LayerNorm IBP (last axis) #
Layer normalization (Ba et al.) computes, per vector, something like:
y = (x - mean(x)) / sqrt(var(x) + eps).
We implement a conservative enclosure by:
- Bounding mean using sums of endpoints.
- Bounding variance using a max-deviation upper bound.
- Bounding the per-component ratio by checking endpoint combinations against a positive denominator interval.
This is intended as a simple checker-side transfer rule. It is conservative and is not an optimized relaxation.
References:
- Ba, Kiros, Hinton, "Layer Normalization", 2016: https://arxiv.org/abs/1607.06450
- Bound propagation context: Xu et al., 2020 (auto_LiRPA): https://arxiv.org/abs/2002.12920
Ideal-arithmetic upper bound on the variance term used by analytic LayerNorm rules.
Given endpoint bounds for a vector and bounds on its mean, each coordinate is at most
max |x_i - μ| away from the bounded mean interval. Squaring and summing those coordinate radii
gives a conservative variance upper bound. The implementation uses ordinary scalar arithmetic and
is called only from exact-arithmetic branches; executable endpoint propagation uses
ibpLayerNormRange? instead.
Instances For
Ideal-arithmetic mean bounds for a nonempty vector with bounded coordinates.
For n = 0, the mathematical mean is undefined; this total helper returns (0,0) so callers do
not accidentally divide by zero while they reject or totalize the empty case.
Instances For
Ideal-arithmetic bounds for x - μ when x and μ are bounded by intervals.
LayerNorm transfer rules repeatedly need this centered interval for the input, first derivative, and second derivative streams. Keeping it here avoids duplicating the same endpoint arithmetic in IBP and derivative propagation.
Instances For
Ideal-arithmetic reciprocal-denominator bounds from an upper variance bound.
Instances For
Analytic real-arithmetic LayerNorm bounds on the last axis, lifted over leading dimensions.
Instances For
Uniform finite enclosure for last-axis layer normalization.
For a row of length n, exact LayerNorm without affine scale or bias satisfies
|yᵢ| ≤ sqrt n; a singleton row is identically zero. Backends provide the outward-rounded bound
through NonlinearBoundOps.layerNormAbsBound. Returning none is preferable to evaluating the
normalization with unqualified host division and square root.
Instances For
Directed bounds for one LayerNorm row with explicit affine parameters.
The order follows Spec.layerNorm: center the input, center again inside reduceVar, square,
average, clamp variance to zero, add epsilon, take a square root, and divide the first centered
values directly. Each coordinate is then multiplied by its gamma and shifted by its beta;
negative gamma reverses the interval endpoints.
Epsilon must be finite and positive, and the row and affine parameters must be finite. Division
and square root use the selected nonlinear backend. Invalid or non-finite intermediate intervals
return none. An end-to-end soundness theorem for this sequence and a proof of agreement with
native arithmetic remain open obligations.
Instances For
Input-dependent last-axis LayerNorm enclosure with unit scale, zero bias, and the default epsilon.
Each row uses directedLayerNormRow?, including the second centering and direct division in the
specification. Leading dimensions select independent rows.
Instances For
Use the backend's uniform LayerNorm range when available; otherwise propagate the supplied input box through the directed normalization sequence.
Instances For
Enclose a last-axis LayerNorm payload using its full epsilon, gamma, and beta.
The matrix view and affine suffix are checked by the same helpers used in IR evaluation. Bounds are propagated separately for each row, then flattened back into the graph's storage order. Every payload, including one with default parameter values, uses the directed normalization sequence with its stored affine parameters and epsilon.
Instances For
For tensors known to have shape .dim n .scalar, extract the underlying function.
Instances For
Cast a 1D Box along an equality of dimensions.
Instances For
Cast a ReLU relaxation vector across a proven-equal hidden dimension.
Instances For
Cast the input dimension of an affine map across a proven equality.
Instances For
Cast the output dimension of an affine map across a proven equality.
Instances For
Cast a dim-scalar tensor across an equality of dimensions.
We keep this as an abbrev so it unfolds aggressively in simp-based soundness proofs.
Instances For
Casting a flat vector tensor along an equality from a dimension to itself changes no data.
IBP propagation through explicit linear parameters.
Instances For
IBP propagation for a .linear node using ParamStore.linearWB.
Instances For
IBP propagation for a .matmul node (bias-free) using ParamStore.matmulW.
Instances For
IBP transfer for a supported convolution node whose parameters are stored in ParamStore.convCfg.
Instances For
Apply a monotone SomeTensor operation independently to both interval endpoints.