TorchLean API

NN.Proofs.Autograd.Dual

Higher-order semantics of runtime dual numbers #

A nested dual value stores mixed derivatives in independent infinitesimal directions. jet gives those coefficients their mathematical meaning using mathlib's Fréchet derivative. The operation theorems connect that meaning to the actual runtime arithmetic at every finite order.

The last direction occupies the outermost dual layer. Extracting every tangent gives iteratedFDeriv applied to the complete direction tuple, with no factorial scaling. The input space is any real normed space; directions need not be distinct or coordinate vectors.

These are exact-real scalar rules. Model lowering, effectful evaluation, and floating-point rounding require their own links to these semantics.

noncomputable def Runtime.Autograd.Model.Dual.jet {E : Type u_1} [NormedAddCommGroup E] [NormedSpace E] {n : } :
(Fin nE)(E)ENested n

All mixed derivative coefficients of a function along the supplied directions.

Instances For
    theorem Runtime.Autograd.Model.Dual.jet_zero {E : Type u_1} [NormedAddCommGroup E] [NormedSpace E] {n : } (directions : Fin nE) (x : E) :
    jet directions (fun (x : E) => 0) x = 0

    All coefficients of the zero function vanish.

    theorem Runtime.Autograd.Model.Dual.jet_const {E : Type u_1} [NormedAddCommGroup E] [NormedSpace E] {n : } (directions : Fin nE) (value : ) (x : E) :
    jet directions (fun (x : E) => value) x = Nested.ofPrimal n value

    Embedding a constant is valid at every order, not just for one forward pass.

    theorem Runtime.Autograd.Model.Dual.jet_linear {E : Type u_1} [NormedAddCommGroup E] [NormedSpace E] {n : } (directions : Fin nE) (f : E →L[] ) (x : E) :
    jet directions (⇑f) x = Nested.seed (fun (i : Fin n) => f (directions i)) (f x)

    A linear input coordinate has its value and supplied directions, and no higher coefficients.

    theorem Runtime.Autograd.Model.Dual.jet_id {n : } (directions : Fin n) (x : ) :
    jet directions (fun (y : ) => y) x = Nested.seed directions x

    Scalar input seeding agrees with the identity function's full derivative data.

    theorem Runtime.Autograd.Model.Dual.jet_neg {E : Type u_1} [NormedAddCommGroup E] [NormedSpace E] {n : } (directions : Fin nE) (f : E) (x : E) :
    jet directions (fun (y : E) => -f y) x = -jet directions f x

    The runtime negation rule preserves all derivative coefficients.

    theorem Runtime.Autograd.Model.Dual.jet_congr {E : Type u_1} [NormedAddCommGroup E] [NormedSpace E] {n : } (directions : Fin nE) {f g : E} {x : E} (h : f =ᶠ[nhds x] g) :
    jet directions f x = jet directions g x

    The derivative coefficients depend only on the function near the evaluation point.

    theorem Runtime.Autograd.Model.Dual.jet_add_at {E : Type u_1} [NormedAddCommGroup E] [NormedSpace E] {n : } (directions : Fin nE) {f g : E} {x : E} (hf : ContDiffAt (↑n) f x) (hg : ContDiffAt (↑n) g x) :
    jet directions (fun (y : E) => f y + g y) x = jet directions f x + jet directions g x

    Addition preserves mixed derivatives under pointwise smoothness assumptions.

    theorem Runtime.Autograd.Model.Dual.jet_sub_at {E : Type u_1} [NormedAddCommGroup E] [NormedSpace E] {n : } (directions : Fin nE) {f g : E} {x : E} (hf : ContDiffAt (↑n) f x) (hg : ContDiffAt (↑n) g x) :
    jet directions (fun (y : E) => f y - g y) x = jet directions f x - jet directions g x

    Subtraction preserves mixed derivatives under pointwise smoothness assumptions.

    theorem Runtime.Autograd.Model.Dual.jet_mul_at {E : Type u_1} [NormedAddCommGroup E] [NormedSpace E] {n : } (directions : Fin nE) {f g : E} {x : E} (hf : ContDiffAt (↑n) f x) (hg : ContDiffAt (↑n) g x) :
    jet directions (fun (y : E) => f y * g y) x = jet directions f x * jet directions g x

    The runtime product rule needs smoothness only near the evaluation point.

    theorem Runtime.Autograd.Model.Dual.jet_add {E : Type u_1} [NormedAddCommGroup E] [NormedSpace E] {n : } (directions : Fin nE) {f g : E} (hf : ContDiff (↑n) f) (hg : ContDiff (↑n) g) (x : E) :
    jet directions (fun (y : E) => f y + g y) x = jet directions f x + jet directions g x

    Runtime addition preserves every derivative coefficient through order n.

    theorem Runtime.Autograd.Model.Dual.jet_sub {E : Type u_1} [NormedAddCommGroup E] [NormedSpace E] {n : } (directions : Fin nE) {f g : E} (hf : ContDiff (↑n) f) (hg : ContDiff (↑n) g) (x : E) :
    jet directions (fun (y : E) => f y - g y) x = jet directions f x - jet directions g x

    Runtime subtraction preserves mixed derivatives through the requested finite order.

    theorem Runtime.Autograd.Model.Dual.jet_one {E : Type u_1} [NormedAddCommGroup E] [NormedSpace E] {n : } (directions : Fin nE) (x : E) :
    jet directions (fun (x : E) => 1) x = 1

    The constant one has primal one and no derivative coefficients at any depth.

    theorem Runtime.Autograd.Model.Dual.jet_mul {E : Type u_1} [NormedAddCommGroup E] [NormedSpace E] {n : } (directions : Fin nE) {f g : E} (hf : ContDiff (↑n) f) (hg : ContDiff (↑n) g) (x : E) :
    jet directions (fun (y : E) => f y * g y) x = jet directions f x * jet directions g x

    The runtime product rule remains correct under arbitrary nesting.

    theorem Runtime.Autograd.Model.Dual.jet_exp_comp_at {E : Type u_1} [NormedAddCommGroup E] [NormedSpace E] {n : } (directions : Fin nE) {f : E} {x : E} (hf : ContDiffAt (↑n) f x) :
    jet directions (fun (y : E) => Real.exp (f y)) x = MathFunctions.exp (jet directions f x)

    The runtime exponential rule preserves mixed derivatives through order n.

    theorem Runtime.Autograd.Model.Dual.jet_sin_comp_at {E : Type u_1} [NormedAddCommGroup E] [NormedSpace E] {n : } (directions : Fin nE) {f : E} {x : E} (hf : ContDiffAt (↑n) f x) :
    jet directions (fun (y : E) => Real.sin (f y)) x = MathFunctions.sin (jet directions f x)

    The sine and cosine runtime rules preserve one another's higher derivatives.

    theorem Runtime.Autograd.Model.Dual.jet_cos_comp_at {E : Type u_1} [NormedAddCommGroup E] [NormedSpace E] {n : } (directions : Fin nE) {f : E} {x : E} (hf : ContDiffAt (↑n) f x) :
    jet directions (fun (y : E) => Real.cos (f y)) x = MathFunctions.cos (jet directions f x)

    The cosine rule, including its alternating derivative signs, is valid at every order.

    theorem Runtime.Autograd.Model.Dual.jet_tanh_comp_at {E : Type u_1} [NormedAddCommGroup E] [NormedSpace E] {n : } (directions : Fin nE) {f : E} {x : E} (hf : ContDiffAt (↑n) f x) :
    jet directions (fun (y : E) => Real.tanh (f y)) x = MathFunctions.tanh (jet directions f x)

    The runtime tanh rule computes every mixed derivative of a smooth real input function.

    theorem Runtime.Autograd.Model.Dual.jet_sinh_comp_at {E : Type u_1} [NormedAddCommGroup E] [NormedSpace E] {n : } (directions : Fin nE) {f : E} {x : E} (hf : ContDiffAt (↑n) f x) :
    jet directions (fun (y : E) => Real.sinh (f y)) x = MathFunctions.sinh (jet directions f x)

    Hyperbolic sine preserves higher derivatives through the runtime's mutual sinh/cosh rules.

    theorem Runtime.Autograd.Model.Dual.jet_cosh_comp_at {E : Type u_1} [NormedAddCommGroup E] [NormedSpace E] {n : } (directions : Fin nE) {f : E} {x : E} (hf : ContDiffAt (↑n) f x) :
    jet directions (fun (y : E) => Real.cosh (f y)) x = MathFunctions.cosh (jet directions f x)

    Hyperbolic cosine preserves higher derivatives through the runtime's mutual sinh/cosh rules.

    theorem Runtime.Autograd.Model.Dual.jet_exp_comp {E : Type u_1} [NormedAddCommGroup E] [NormedSpace E] {n : } (directions : Fin nE) {f : E} (hf : ContDiff (↑n) f) (x : E) :
    jet directions (fun (y : E) => Real.exp (f y)) x = MathFunctions.exp (jet directions f x)

    The runtime exponential rule preserves mixed derivatives through order n.

    theorem Runtime.Autograd.Model.Dual.jet_sin_comp {E : Type u_1} [NormedAddCommGroup E] [NormedSpace E] {n : } (directions : Fin nE) {f : E} (hf : ContDiff (↑n) f) (x : E) :
    jet directions (fun (y : E) => Real.sin (f y)) x = MathFunctions.sin (jet directions f x)

    The sine and cosine runtime rules preserve one another's higher derivatives.

    theorem Runtime.Autograd.Model.Dual.jet_cos_comp {E : Type u_1} [NormedAddCommGroup E] [NormedSpace E] {n : } (directions : Fin nE) {f : E} (hf : ContDiff (↑n) f) (x : E) :
    jet directions (fun (y : E) => Real.cos (f y)) x = MathFunctions.cos (jet directions f x)

    The cosine rule, including its alternating derivative signs, is valid at every order.

    theorem Runtime.Autograd.Model.Dual.jet_tanh_comp {E : Type u_1} [NormedAddCommGroup E] [NormedSpace E] {n : } (directions : Fin nE) {f : E} (hf : ContDiff (↑n) f) (x : E) :
    jet directions (fun (y : E) => Real.tanh (f y)) x = MathFunctions.tanh (jet directions f x)

    The runtime tanh rule computes every mixed derivative of a smooth real input function.

    theorem Runtime.Autograd.Model.Dual.jet_sinh_comp {E : Type u_1} [NormedAddCommGroup E] [NormedSpace E] {n : } (directions : Fin nE) {f : E} (hf : ContDiff (↑n) f) (x : E) :
    jet directions (fun (y : E) => Real.sinh (f y)) x = MathFunctions.sinh (jet directions f x)

    Hyperbolic sine preserves higher derivatives through the runtime's mutual sinh/cosh rules.

    theorem Runtime.Autograd.Model.Dual.jet_cosh_comp {E : Type u_1} [NormedAddCommGroup E] [NormedSpace E] {n : } (directions : Fin nE) {f : E} (hf : ContDiff (↑n) f) (x : E) :
    jet directions (fun (y : E) => Real.cosh (f y)) x = MathFunctions.cosh (jet directions f x)

    Hyperbolic cosine preserves higher derivatives through the runtime's mutual sinh/cosh rules.

    theorem Runtime.Autograd.Model.Dual.jet_exp {n : } (directions : Fin n) (x : ) :
    jet directions Real.exp x = MathFunctions.exp (Nested.seed directions x)

    Direct scalar exponential evaluation on seeded runtime inputs.

    theorem Runtime.Autograd.Model.Dual.jet_sin {n : } (directions : Fin n) (x : ) :
    jet directions Real.sin x = MathFunctions.sin (Nested.seed directions x)

    Direct scalar sine evaluation on seeded runtime inputs.

    theorem Runtime.Autograd.Model.Dual.jet_cos {n : } (directions : Fin n) (x : ) :
    jet directions Real.cos x = MathFunctions.cos (Nested.seed directions x)

    Direct scalar cosine evaluation on seeded runtime inputs.

    theorem Runtime.Autograd.Model.Dual.jet_tanh {n : } (directions : Fin n) (x : ) :
    jet directions Real.tanh x = MathFunctions.tanh (Nested.seed directions x)

    Direct scalar tanh evaluation on seeded runtime inputs.

    theorem Runtime.Autograd.Model.Dual.jet_sinh {n : } (directions : Fin n) (x : ) :
    jet directions Real.sinh x = MathFunctions.sinh (Nested.seed directions x)

    Direct scalar hyperbolic sine evaluation on seeded runtime inputs.

    theorem Runtime.Autograd.Model.Dual.jet_cosh {n : } (directions : Fin n) (x : ) :
    jet directions Real.cosh x = MathFunctions.cosh (Nested.seed directions x)

    Direct scalar hyperbolic cosine evaluation on seeded runtime inputs.

    theorem Runtime.Autograd.Model.Dual.tangent_jet_at {E : Type u_1} [NormedAddCommGroup E] [NormedSpace E] {n : } (directions : Fin nE) {f : E} {x : E} (hf : ContDiffAt (↑n) f x) :
    (jet directions f x).tangent = (iteratedFDeriv n f x) directions

    Extracting every tangent gives the iterated derivative under local smoothness.

    theorem Runtime.Autograd.Model.Dual.tangent_jet {E : Type u_1} [NormedAddCommGroup E] [NormedSpace E] {n : } (directions : Fin nE) {f : E} (hf : ContDiff (↑n) f) (x : E) :
    (jet directions f x).tangent = (iteratedFDeriv n f x) directions

    The extracted runtime coefficient is mathlib's iterated Fréchet derivative.

    theorem Runtime.Autograd.Model.Dual.jet_foldl_at {E : Type u_1} [NormedAddCommGroup E] [NormedSpace E] {n : } {ι : Type u_2} (indices : List ι) (directions : Fin nE) (step : ) (lifted : Nested nNested nNested n) (hstep : ContDiff n fun (p : × ) => step p.1 p.2) {x : E} (hjet : ∀ (f g : E), ContDiffAt (↑n) f xContDiffAt (↑n) g xjet directions (fun (y : E) => step (f y) (g y)) x = lifted (jet directions f x) (jet directions g x)) {initial : E} {term : ιE} (hinit : ContDiffAt (↑n) initial x) (hterm : iindices, ContDiffAt (↑n) (term i) x) :
    jet directions (fun (y : E) => List.foldl (fun (acc : ) (i : ι) => step acc (term i y)) (initial y) indices) x = List.foldl (fun (acc : Nested n) (i : ι) => lifted acc (jet directions (term i) x)) (jet directions initial x) indices

    A smooth update with a local jet law preserves jets in the supplied traversal order.

    theorem Runtime.Autograd.Model.Dual.jet_foldl_add_at {E : Type u_1} [NormedAddCommGroup E] [NormedSpace E] {n : } {ι : Type u_2} (indices : List ι) (directions : Fin nE) {initial : E} {term : ιE} {x : E} (hinit : ContDiffAt (↑n) initial x) (hterm : iindices, ContDiffAt (↑n) (term i) x) :
    jet directions (fun (y : E) => List.foldl (fun (acc : ) (i : ι) => acc + term i y) (initial y) indices) x = List.foldl (fun (acc : Nested n) (i : ι) => acc + jet directions (term i) x) (jet directions initial x) indices

    Ordered sums preserve locally smooth derivative coefficients of entries and accumulator.

    theorem Runtime.Autograd.Model.Dual.jet_foldl_mul_at {E : Type u_1} [NormedAddCommGroup E] [NormedSpace E] {n : } {ι : Type u_2} (indices : List ι) (directions : Fin nE) {initial : E} {term : ιE} {x : E} (hinit : ContDiffAt (↑n) initial x) (hterm : iindices, ContDiffAt (↑n) (term i) x) :
    jet directions (fun (y : E) => List.foldl (fun (acc : ) (i : ι) => acc * term i y) (initial y) indices) x = List.foldl (fun (acc : Nested n) (i : ι) => acc * jet directions (term i) x) (jet directions initial x) indices

    Ordered products preserve local jets without requiring nonzero factors.

    theorem Runtime.Autograd.Model.Dual.jet_foldl {E : Type u_1} [NormedAddCommGroup E] [NormedSpace E] {n : } {ι : Type u_2} (indices : List ι) (directions : Fin nE) (step : ) (lifted : Nested nNested nNested n) (hstep : ContDiff n fun (p : × ) => step p.1 p.2) (hjet : ∀ (f g : E), ContDiff (↑n) fContDiff (↑n) g∀ (x : E), jet directions (fun (y : E) => step (f y) (g y)) x = lifted (jet directions f x) (jet directions g x)) {initial : E} {term : ιE} (hinit : ContDiff (↑n) initial) (hterm : iindices, ContDiff (↑n) (term i)) (x : E) :
    jet directions (fun (y : E) => List.foldl (fun (acc : ) (i : ι) => step acc (term i y)) (initial y) indices) x = List.foldl (fun (acc : Nested n) (i : ι) => lifted acc (jet directions (term i) x)) (jet directions initial x) indices

    A finite fold preserves jets when its update rule does, in the caller's traversal order.

    theorem Runtime.Autograd.Model.Dual.jet_foldl_add {E : Type u_1} [NormedAddCommGroup E] [NormedSpace E] {n : } {ι : Type u_2} (indices : List ι) (directions : Fin nE) {initial : E} {term : ιE} (hinit : ContDiff (↑n) initial) (hterm : iindices, ContDiff (↑n) (term i)) (x : E) :
    jet directions (fun (y : E) => List.foldl (fun (acc : ) (i : ι) => acc + term i y) (initial y) indices) x = List.foldl (fun (acc : Nested n) (i : ι) => acc + jet directions (term i) x) (jet directions initial x) indices

    Ordered summation propagates all derivatives of its entries and initial accumulator.

    theorem Runtime.Autograd.Model.Dual.jet_foldl_mul {E : Type u_1} [NormedAddCommGroup E] [NormedSpace E] {n : } {ι : Type u_2} (indices : List ι) (directions : Fin nE) {initial : E} {term : ιE} (hinit : ContDiff (↑n) initial) (hterm : iindices, ContDiff (↑n) (term i)) (x : E) :
    jet directions (fun (y : E) => List.foldl (fun (acc : ) (i : ι) => acc * term i y) (initial y) indices) x = List.foldl (fun (acc : Nested n) (i : ι) => acc * jet directions (term i) x) (jet directions initial x) indices

    Ordered products propagate all derivatives without requiring nonzero factors.