5. Floating Point and Native Boundaries🔗

A dot product looks like a sum of products on paper. To run it, we also choose a number format and an order of evaluation. Products and partial sums may round, so two implementations of the same real formula can return different values.

In TorchLean, the scalar type selects a format from FloatLib. We will use binary32 for a first example, then change the precision through the same interface. Custom exponent and fraction widths are available on the typed CPU path; CUDA providers support native binary32 and binary64. The useful question is what a precision change preserves in the actual calculation.

An error bound adds another step. We need to relate the rounded operations to their real-valued specification, including what happens near zero, at overflow, and on exceptional inputs. FloatLib supplies the scalar arithmetic and rounding theorems. TorchLean carries the relevant bounds through tensor operations and records the numerical choices of native providers. A bound for one accumulation order applies to a kernel only once that kernel's order is accounted for.