TorchLean API

NN.Proofs.Autograd.Dual

Higher-order semantics of runtime dual numbers #

A nested dual value stores mixed derivatives in independent infinitesimal directions. jet gives those coefficients their mathematical meaning using mathlib's Fréchet derivative. The operation theorems connect that meaning to the actual runtime arithmetic at every finite order.

The last direction occupies the outermost dual layer. Extracting every tangent gives iteratedFDeriv applied to the complete direction tuple, with no factorial scaling. The input space is any real normed space; directions need not be distinct or coordinate vectors.

These are exact-real scalar rules. Model lowering, effectful evaluation, and floating-point rounding require their own links to these semantics.

noncomputable def Runtime.Autograd.Model.Dual.jet {E : Type u_1} [NormedAddCommGroup E] [NormedSpace ℝ E] {n : ℕ} :
(Fin n → E) → (E → ℝ) → E → Nested ℝ n

All mixed derivative coefficients of a function along the supplied directions.

Instances For
    theorem Runtime.Autograd.Model.Dual.jet_const {E : Type u_1} [NormedAddCommGroup E] [NormedSpace ℝ E] {n : ℕ} (directions : Fin n → E) (value : ℝ) (x : E) :
    jet directions (fun (x : E) => value) x = Nested.ofPrimal n value

    Embedding a constant is valid at every order, not just for one forward pass.

    theorem Runtime.Autograd.Model.Dual.jet_zero {E : Type u_1} [NormedAddCommGroup E] [NormedSpace ℝ E] {n : ℕ} (directions : Fin n → E) (x : E) :
    jet directions (fun (x : E) => 0) x = 0

    All coefficients of the zero function vanish.

    theorem Runtime.Autograd.Model.Dual.jet_linear {E : Type u_1} [NormedAddCommGroup E] [NormedSpace ℝ E] {n : ℕ} (directions : Fin n → E) (f : E →L[ℝ] ℝ) (x : E) :
    jet directions (⇑f) x = Nested.seed (fun (i : Fin n) => f (directions i)) (f x)

    A linear input coordinate has its value and supplied directions, and no higher coefficients.

    theorem Runtime.Autograd.Model.Dual.jet_id {n : ℕ} (directions : Fin n → ℝ) (x : ℝ) :
    jet directions (fun (y : ℝ) => y) x = Nested.seed directions x

    Scalar input seeding agrees with the identity function's full derivative data.

    theorem Runtime.Autograd.Model.Dual.jet_neg {E : Type u_1} [NormedAddCommGroup E] [NormedSpace ℝ E] {n : ℕ} (directions : Fin n → E) (f : E → ℝ) (x : E) :
    jet directions (fun (y : E) => -f y) x = -jet directions f x

    The runtime negation rule preserves all derivative coefficients.

    theorem Runtime.Autograd.Model.Dual.jet_congr {E : Type u_1} [NormedAddCommGroup E] [NormedSpace ℝ E] {n : ℕ} (directions : Fin n → E) {f g : E → ℝ} {x : E} (h : f =ᶠ[nhds x] g) :
    jet directions f x = jet directions g x

    The derivative coefficients depend only on the function near the evaluation point.

    theorem Runtime.Autograd.Model.Dual.jet_add_at {E : Type u_1} [NormedAddCommGroup E] [NormedSpace ℝ E] {n : ℕ} (directions : Fin n → E) {f g : E → ℝ} {x : E} (hf : ContDiffAt ℝ (↑n) f x) (hg : ContDiffAt ℝ (↑n) g x) :
    jet directions (fun (y : E) => f y + g y) x = jet directions f x + jet directions g x

    Addition preserves mixed derivatives under pointwise smoothness assumptions.

    theorem Runtime.Autograd.Model.Dual.jet_sub_at {E : Type u_1} [NormedAddCommGroup E] [NormedSpace ℝ E] {n : ℕ} (directions : Fin n → E) {f g : E → ℝ} {x : E} (hf : ContDiffAt ℝ (↑n) f x) (hg : ContDiffAt ℝ (↑n) g x) :
    jet directions (fun (y : E) => f y - g y) x = jet directions f x - jet directions g x

    Subtraction preserves mixed derivatives under pointwise smoothness assumptions.

    theorem Runtime.Autograd.Model.Dual.jet_mul_at {E : Type u_1} [NormedAddCommGroup E] [NormedSpace ℝ E] {n : ℕ} (directions : Fin n → E) {f g : E → ℝ} {x : E} (hf : ContDiffAt ℝ (↑n) f x) (hg : ContDiffAt ℝ (↑n) g x) :
    jet directions (fun (y : E) => f y * g y) x = jet directions f x * jet directions g x

    The runtime product rule needs smoothness only near the evaluation point.

    theorem Runtime.Autograd.Model.Dual.jet_add {E : Type u_1} [NormedAddCommGroup E] [NormedSpace ℝ E] {n : ℕ} (directions : Fin n → E) {f g : E → ℝ} (hf : ContDiff ℝ (↑n) f) (hg : ContDiff ℝ (↑n) g) (x : E) :
    jet directions (fun (y : E) => f y + g y) x = jet directions f x + jet directions g x

    Runtime addition preserves every derivative coefficient through order n.

    theorem Runtime.Autograd.Model.Dual.jet_sub {E : Type u_1} [NormedAddCommGroup E] [NormedSpace ℝ E] {n : ℕ} (directions : Fin n → E) {f g : E → ℝ} (hf : ContDiff ℝ (↑n) f) (hg : ContDiff ℝ (↑n) g) (x : E) :
    jet directions (fun (y : E) => f y - g y) x = jet directions f x - jet directions g x

    Runtime subtraction preserves mixed derivatives through the requested finite order.

    theorem Runtime.Autograd.Model.Dual.jet_one {E : Type u_1} [NormedAddCommGroup E] [NormedSpace ℝ E] {n : ℕ} (directions : Fin n → E) (x : E) :
    jet directions (fun (x : E) => 1) x = 1

    The constant one has primal one and no derivative coefficients at any depth.

    theorem Runtime.Autograd.Model.Dual.jet_mul {E : Type u_1} [NormedAddCommGroup E] [NormedSpace ℝ E] {n : ℕ} (directions : Fin n → E) {f g : E → ℝ} (hf : ContDiff ℝ (↑n) f) (hg : ContDiff ℝ (↑n) g) (x : E) :
    jet directions (fun (y : E) => f y * g y) x = jet directions f x * jet directions g x

    The runtime product rule remains correct under arbitrary nesting.

    theorem Runtime.Autograd.Model.Dual.jet_exp_comp_at {E : Type u_1} [NormedAddCommGroup E] [NormedSpace ℝ E] {n : ℕ} (directions : Fin n → E) {f : E → ℝ} {x : E} (hf : ContDiffAt ℝ (↑n) f x) :
    jet directions (fun (y : E) => Real.exp (f y)) x = MathFunctions.exp (jet directions f x)

    The runtime exponential rule preserves mixed derivatives through order n.

    theorem Runtime.Autograd.Model.Dual.jet_sin_comp_at {E : Type u_1} [NormedAddCommGroup E] [NormedSpace ℝ E] {n : ℕ} (directions : Fin n → E) {f : E → ℝ} {x : E} (hf : ContDiffAt ℝ (↑n) f x) :
    jet directions (fun (y : E) => Real.sin (f y)) x = MathFunctions.sin (jet directions f x)

    The sine and cosine runtime rules preserve one another's higher derivatives.

    theorem Runtime.Autograd.Model.Dual.jet_cos_comp_at {E : Type u_1} [NormedAddCommGroup E] [NormedSpace ℝ E] {n : ℕ} (directions : Fin n → E) {f : E → ℝ} {x : E} (hf : ContDiffAt ℝ (↑n) f x) :
    jet directions (fun (y : E) => Real.cos (f y)) x = MathFunctions.cos (jet directions f x)

    The cosine rule, including its alternating derivative signs, is valid at every order.

    theorem Runtime.Autograd.Model.Dual.jet_beq_zero {E : Type u_1} [NormedAddCommGroup E] [NormedSpace ℝ E] {n : ℕ} (directions : Fin n → E) (f : E → ℝ) (x : E) :
    (jet directions f x == 0) = (f x == 0)

    Runtime comparisons inspect the primal value, not its derivative coefficients.

    theorem Runtime.Autograd.Model.Dual.jet_div_at {E : Type u_1} [NormedAddCommGroup E] [NormedSpace ℝ E] {n : ℕ} (directions : Fin n → E) {f g : E → ℝ} {x : E} (hf : ContDiffAt ℝ (↑n) f x) (hg : ContDiffAt ℝ (↑n) g x) (hzero : g x ≠ 0) :
    jet directions (fun (y : E) => f y / g y) x = jet directions f x / jet directions g x

    The runtime quotient rule preserves every derivative coefficient away from a zero denominator. Both functions need be smooth only near the evaluation point.

    theorem Runtime.Autograd.Model.Dual.jet_pos {E : Type u_1} [NormedAddCommGroup E] [NormedSpace ℝ E] {n : ℕ} (directions : Fin n → E) (f : E → ℝ) (x : E) :
    0 < jet directions f x ↔ 0 < f x

    Strict positivity of a jet depends only on its primal value.

    theorem Runtime.Autograd.Model.Dual.jet_sigmoid_comp_at {E : Type u_1} [NormedAddCommGroup E] [NormedSpace ℝ E] {n : ℕ} (directions : Fin n → E) {f : E → ℝ} {x : E} (hf : ContDiffAt ℝ (↑n) f x) :
    jet directions (fun (y : E) => (f y).sigmoid) x = Context.sigmoid (jet directions f x)

    The analytic sigmoid rule preserves every mixed derivative of a smooth real input function.

    The scalar range repair is inactive over exact reals; the runtime's weighted pullback reduces to the derivative used by mathlib. No floating-point accuracy claim is made here.

    theorem Runtime.Autograd.Model.Dual.jet_tanh_comp_at {E : Type u_1} [NormedAddCommGroup E] [NormedSpace ℝ E] {n : ℕ} (directions : Fin n → E) {f : E → ℝ} {x : E} (hf : ContDiffAt ℝ (↑n) f x) :
    jet directions (fun (y : E) => Real.tanh (f y)) x = MathFunctions.tanh (jet directions f x)

    The runtime tanh rule computes every mixed derivative of a smooth real input function.

    theorem Runtime.Autograd.Model.Dual.jet_sinh_comp_at {E : Type u_1} [NormedAddCommGroup E] [NormedSpace ℝ E] {n : ℕ} (directions : Fin n → E) {f : E → ℝ} {x : E} (hf : ContDiffAt ℝ (↑n) f x) :
    jet directions (fun (y : E) => Real.sinh (f y)) x = MathFunctions.sinh (jet directions f x)

    Hyperbolic sine preserves higher derivatives through the runtime's mutual sinh/cosh rules.

    theorem Runtime.Autograd.Model.Dual.jet_cosh_comp_at {E : Type u_1} [NormedAddCommGroup E] [NormedSpace ℝ E] {n : ℕ} (directions : Fin n → E) {f : E → ℝ} {x : E} (hf : ContDiffAt ℝ (↑n) f x) :
    jet directions (fun (y : E) => Real.cosh (f y)) x = MathFunctions.cosh (jet directions f x)

    Hyperbolic cosine preserves higher derivatives through the runtime's mutual sinh/cosh rules.

    theorem Runtime.Autograd.Model.Dual.jet_exp_comp {E : Type u_1} [NormedAddCommGroup E] [NormedSpace ℝ E] {n : ℕ} (directions : Fin n → E) {f : E → ℝ} (hf : ContDiff ℝ (↑n) f) (x : E) :
    jet directions (fun (y : E) => Real.exp (f y)) x = MathFunctions.exp (jet directions f x)

    The runtime exponential rule preserves mixed derivatives through order n.

    theorem Runtime.Autograd.Model.Dual.jet_sin_comp {E : Type u_1} [NormedAddCommGroup E] [NormedSpace ℝ E] {n : ℕ} (directions : Fin n → E) {f : E → ℝ} (hf : ContDiff ℝ (↑n) f) (x : E) :
    jet directions (fun (y : E) => Real.sin (f y)) x = MathFunctions.sin (jet directions f x)

    The sine and cosine runtime rules preserve one another's higher derivatives.

    theorem Runtime.Autograd.Model.Dual.jet_cos_comp {E : Type u_1} [NormedAddCommGroup E] [NormedSpace ℝ E] {n : ℕ} (directions : Fin n → E) {f : E → ℝ} (hf : ContDiff ℝ (↑n) f) (x : E) :
    jet directions (fun (y : E) => Real.cos (f y)) x = MathFunctions.cos (jet directions f x)

    The cosine rule, including its alternating derivative signs, is valid at every order.

    theorem Runtime.Autograd.Model.Dual.jet_tanh_comp {E : Type u_1} [NormedAddCommGroup E] [NormedSpace ℝ E] {n : ℕ} (directions : Fin n → E) {f : E → ℝ} (hf : ContDiff ℝ (↑n) f) (x : E) :
    jet directions (fun (y : E) => Real.tanh (f y)) x = MathFunctions.tanh (jet directions f x)

    The runtime tanh rule computes every mixed derivative of a smooth real input function.

    theorem Runtime.Autograd.Model.Dual.jet_sinh_comp {E : Type u_1} [NormedAddCommGroup E] [NormedSpace ℝ E] {n : ℕ} (directions : Fin n → E) {f : E → ℝ} (hf : ContDiff ℝ (↑n) f) (x : E) :
    jet directions (fun (y : E) => Real.sinh (f y)) x = MathFunctions.sinh (jet directions f x)

    Hyperbolic sine preserves higher derivatives through the runtime's mutual sinh/cosh rules.

    theorem Runtime.Autograd.Model.Dual.jet_cosh_comp {E : Type u_1} [NormedAddCommGroup E] [NormedSpace ℝ E] {n : ℕ} (directions : Fin n → E) {f : E → ℝ} (hf : ContDiff ℝ (↑n) f) (x : E) :
    jet directions (fun (y : E) => Real.cosh (f y)) x = MathFunctions.cosh (jet directions f x)

    Hyperbolic cosine preserves higher derivatives through the runtime's mutual sinh/cosh rules.

    theorem Runtime.Autograd.Model.Dual.jet_exp {n : ℕ} (directions : Fin n → ℝ) (x : ℝ) :
    jet directions Real.exp x = MathFunctions.exp (Nested.seed directions x)

    Direct scalar exponential evaluation on seeded runtime inputs.

    theorem Runtime.Autograd.Model.Dual.jet_sin {n : ℕ} (directions : Fin n → ℝ) (x : ℝ) :
    jet directions Real.sin x = MathFunctions.sin (Nested.seed directions x)

    Direct scalar sine evaluation on seeded runtime inputs.

    theorem Runtime.Autograd.Model.Dual.jet_cos {n : ℕ} (directions : Fin n → ℝ) (x : ℝ) :
    jet directions Real.cos x = MathFunctions.cos (Nested.seed directions x)

    Direct scalar cosine evaluation on seeded runtime inputs.

    theorem Runtime.Autograd.Model.Dual.jet_tanh {n : ℕ} (directions : Fin n → ℝ) (x : ℝ) :
    jet directions Real.tanh x = MathFunctions.tanh (Nested.seed directions x)

    Direct scalar tanh evaluation on seeded runtime inputs.

    theorem Runtime.Autograd.Model.Dual.jet_sinh {n : ℕ} (directions : Fin n → ℝ) (x : ℝ) :
    jet directions Real.sinh x = MathFunctions.sinh (Nested.seed directions x)

    Direct scalar hyperbolic sine evaluation on seeded runtime inputs.

    theorem Runtime.Autograd.Model.Dual.jet_cosh {n : ℕ} (directions : Fin n → ℝ) (x : ℝ) :
    jet directions Real.cosh x = MathFunctions.cosh (Nested.seed directions x)

    Direct scalar hyperbolic cosine evaluation on seeded runtime inputs.

    theorem Runtime.Autograd.Model.Dual.tangent_jet_at {E : Type u_1} [NormedAddCommGroup E] [NormedSpace ℝ E] {n : ℕ} (directions : Fin n → E) {f : E → ℝ} {x : E} (hf : ContDiffAt ℝ (↑n) f x) :
    (jet directions f x).tangent = (iteratedFDeriv ℝ n f x) directions

    Extracting every tangent gives the iterated derivative under local smoothness.

    theorem Runtime.Autograd.Model.Dual.tangent_jet {E : Type u_1} [NormedAddCommGroup E] [NormedSpace ℝ E] {n : ℕ} (directions : Fin n → E) {f : E → ℝ} (hf : ContDiff ℝ (↑n) f) (x : E) :
    (jet directions f x).tangent = (iteratedFDeriv ℝ n f x) directions

    The extracted runtime coefficient is mathlib's iterated Fréchet derivative.

    theorem Runtime.Autograd.Model.Dual.jet_foldl_at {E : Type u_1} [NormedAddCommGroup E] [NormedSpace ℝ E] {n : ℕ} {ι : Type u_2} (indices : List ι) (directions : Fin n → E) (step : ℝ → ℝ → ℝ) (lifted : Nested ℝ n → Nested ℝ n → Nested ℝ n) (hstep : ContDiff ℝ ↑n fun (p : ℝ × ℝ) => step p.1 p.2) {x : E} (hjet : ∀ (f g : E → ℝ), ContDiffAt ℝ (↑n) f x → ContDiffAt ℝ (↑n) g x → jet directions (fun (y : E) => step (f y) (g y)) x = lifted (jet directions f x) (jet directions g x)) {initial : E → ℝ} {term : ι → E → ℝ} (hinit : ContDiffAt ℝ (↑n) initial x) (hterm : ∀ i ∈ indices, ContDiffAt ℝ (↑n) (term i) x) :
    jet directions (fun (y : E) => List.foldl (fun (acc : ℝ) (i : ι) => step acc (term i y)) (initial y) indices) x = List.foldl (fun (acc : Nested ℝ n) (i : ι) => lifted acc (jet directions (term i) x)) (jet directions initial x) indices

    A smooth update with a local jet law preserves jets in the supplied traversal order.

    theorem Runtime.Autograd.Model.Dual.jet_foldl_add_at {E : Type u_1} [NormedAddCommGroup E] [NormedSpace ℝ E] {n : ℕ} {ι : Type u_2} (indices : List ι) (directions : Fin n → E) {initial : E → ℝ} {term : ι → E → ℝ} {x : E} (hinit : ContDiffAt ℝ (↑n) initial x) (hterm : ∀ i ∈ indices, ContDiffAt ℝ (↑n) (term i) x) :
    jet directions (fun (y : E) => List.foldl (fun (acc : ℝ) (i : ι) => acc + term i y) (initial y) indices) x = List.foldl (fun (acc : Nested ℝ n) (i : ι) => acc + jet directions (term i) x) (jet directions initial x) indices

    Ordered sums preserve locally smooth derivative coefficients of entries and accumulator.

    theorem Runtime.Autograd.Model.Dual.jet_foldl_mul_at {E : Type u_1} [NormedAddCommGroup E] [NormedSpace ℝ E] {n : ℕ} {ι : Type u_2} (indices : List ι) (directions : Fin n → E) {initial : E → ℝ} {term : ι → E → ℝ} {x : E} (hinit : ContDiffAt ℝ (↑n) initial x) (hterm : ∀ i ∈ indices, ContDiffAt ℝ (↑n) (term i) x) :
    jet directions (fun (y : E) => List.foldl (fun (acc : ℝ) (i : ι) => acc * term i y) (initial y) indices) x = List.foldl (fun (acc : Nested ℝ n) (i : ι) => acc * jet directions (term i) x) (jet directions initial x) indices

    Ordered products preserve local jets without requiring nonzero factors.

    theorem Runtime.Autograd.Model.Dual.jet_foldl_add {E : Type u_1} [NormedAddCommGroup E] [NormedSpace ℝ E] {n : ℕ} {ι : Type u_2} (indices : List ι) (directions : Fin n → E) {initial : E → ℝ} {term : ι → E → ℝ} (hinit : ContDiff ℝ (↑n) initial) (hterm : ∀ i ∈ indices, ContDiff ℝ (↑n) (term i)) (x : E) :
    jet directions (fun (y : E) => List.foldl (fun (acc : ℝ) (i : ι) => acc + term i y) (initial y) indices) x = List.foldl (fun (acc : Nested ℝ n) (i : ι) => acc + jet directions (term i) x) (jet directions initial x) indices

    Ordered summation propagates all derivatives of its entries and initial accumulator.

    theorem Runtime.Autograd.Model.Dual.jet_foldl_mul {E : Type u_1} [NormedAddCommGroup E] [NormedSpace ℝ E] {n : ℕ} {ι : Type u_2} (indices : List ι) (directions : Fin n → E) {initial : E → ℝ} {term : ι → E → ℝ} (hinit : ContDiff ℝ (↑n) initial) (hterm : ∀ i ∈ indices, ContDiff ℝ (↑n) (term i)) (x : E) :
    jet directions (fun (y : E) => List.foldl (fun (acc : ℝ) (i : ι) => acc * term i y) (initial y) indices) x = List.foldl (fun (acc : Nested ℝ n) (i : ι) => acc * jet directions (term i) x) (jet directions initial x) indices

    Ordered products propagate all derivatives without requiring nonzero factors.