PCA (spec model)

Principal Component Analysis is represented as a linear projection onto learned components, plus an explicit mean for centering.

The exact model operations are the transform and inverse transform. A separate reference helper below constructs a one-component approximation with power iteration; its name records that numerical limitation explicitly.

The model follows the usual centered linear projection used by PCA. The fitting helper is deliberately narrower: it approximates one leading component by power iteration.

References:

• Pearson (1901), "On Lines and Planes of Closest Fit to Systems of Points in Space". https://doi.org/10.1080/14786440109462720
• Hotelling (1933), "Analysis of a complex of statistical variables into principal components". https://doi.org/10.2307/2333955

Implementation status

No API builder implements this model, and no theorem is proved about it. It is a reference definition only.

structure Spec.PCASpec (α : Type) [TorchLean.Storage α] (inDim outDim : ℕ) : Type

Parameters for PCA as a linear map plus centering.

We store:

• components : outDim × inDim (rows are principal directions),
• mean : inDim (for centering),
• explainedVariance : outDim (eigenvalues for the selected components).

This matches the typical PCA API: you can transform to outDim coordinates and inverse back to inDim.

• components : TorchLean.Tensor α [outDim, inDim]

  Principal directions, one row for each output coordinate.

• mean : TorchLean.Tensor α [inDim]

  Coordinate-wise sample mean subtracted before projection.

• explainedVariance : TorchLean.Tensor α [outDim]

  Covariance eigenvalue associated with each selected component.

def Spec.pcaForwardSpec {α : Type} [TorchLean.Storage α] [Context α] {inDim outDim : ℕ} (m : PCASpec α inDim outDim) (input : TorchLean.Tensor α [inDim]) : TorchLean.Tensor α [outDim]

Forward pass: center and project: y = components · (x - mean).

def Spec.pcaInverseSpec {α : Type} [TorchLean.Storage α] [Context α] {inDim outDim : ℕ} (m : PCASpec α inDim outDim) (reduced : TorchLean.Tensor α [outDim]) : TorchLean.Tensor α [inDim]

Inverse transform: reconstruct x ≈ componentsᵀ · y + mean.

def Spec.pcaComponentsDerivSpec {α : Type} [TorchLean.Storage α] [Context α] {inDim outDim : ℕ} (m : PCASpec α inDim outDim) (input : TorchLean.Tensor α [inDim]) (gradOutput : TorchLean.Tensor α [outDim]) : TorchLean.Tensor α [outDim, inDim]

VJP contribution for components: outer product dL/dy ⊗ (x - mean).

def Spec.pcaMeanDerivSpec {α : Type} [TorchLean.Storage α] [Context α] {inDim outDim : ℕ} (m : PCASpec α inDim outDim) (gradOutput : TorchLean.Tensor α [outDim]) : TorchLean.Tensor α [inDim]

VJP contribution for mean: dL/dmean = -componentsᵀ · dL/dy.

def Spec.pcaInputDerivSpec {α : Type} [TorchLean.Storage α] [Context α] {inDim outDim : ℕ} (m : PCASpec α inDim outDim) (gradOutput : TorchLean.Tensor α [outDim]) : TorchLean.Tensor α [inDim]

VJP contribution for input: dL/dx = componentsᵀ · dL/dy.

structure Spec.PCAGradients (α : Type) [TorchLean.Storage α] (inDim outDim : ℕ) : Type

Gradients for a PCASpec projection.

• componentsGradient : TorchLean.Tensor α [outDim, inDim]

  Gradient with respect to the component matrix.

• meanGradient : TorchLean.Tensor α [inDim]

  Gradient with respect to the stored mean.

• inputGradient : TorchLean.Tensor α [inDim]

  Gradient with respect to the projected input.

def Spec.pcaBackwardSpec {α : Type} [TorchLean.Storage α] [Context α] {inDim outDim : ℕ} (m : PCASpec α inDim outDim) (input : TorchLean.Tensor α [inDim]) (gradOutput : TorchLean.Tensor α [outDim]) : PCAGradients α inDim outDim

Full backward pass for a PCA projection.

def Spec.pcaFitLeadingComponentApproxSpec {α : Type} [TorchLean.Storage α] [Context α] {nSamples inDim : ℕ} (data : TorchLean.Tensor α [nSamples, inDim]) (iterations : ℕ) (hSamples : 1 < nSamples) (hDim : 0 < inDim) : PCASpec α inDim 1

Approximate one leading PCA component with deterministic multistart power iteration.

The fit centers the samples and uses covariance Xᵀ X / (n - 1). It runs iterations steps from the normalized all-ones vector and every coordinate vector, then keeps the largest Rayleigh quotient. The coordinate starts span the input space, so a dominant eigenspace cannot be orthogonal to every start. Convergence still depends on the spectral gap and iteration count.

Scaling the covariance before iteration avoids squaring its original magnitude when normalizing iterates. This does not prevent overflow while forming the covariance itself. Zero covariance retains a unit direction and reports zero variance. The output has exactly one component; the iteration cost is cubic in inDim, with inDim + 1 starts.

def Spec.pcaFitLeadingComponentApproxSpec.iterate {α : Type} [TorchLean.Storage α] [Context α] {inDim : ℕ} (iterationMatrix : TorchLean.Tensor α [inDim, inDim]) (direction : TorchLean.Tensor α [inDim]) (remaining : ℕ) : α × TorchLean.Tensor α [inDim]

def Spec.pcaTransformSpec {α : Type} [TorchLean.Storage α] [Context α] {nSamples inDim outDim : ℕ} (m : PCASpec α inDim outDim) (data : TorchLean.Tensor α [nSamples, inDim]) : TorchLean.Tensor α [nSamples, outDim]

Apply a fitted PCA transform to a batch of samples.

def Spec.pcaReconstructionErrorSpec {α : Type} [TorchLean.Storage α] [Context α] {inDim outDim : ℕ} (m : PCASpec α inDim outDim) (input : TorchLean.Tensor α [inDim]) (h : inDim ≠ 0) : α

Reconstruction error: ||x - inverse(transform(x))||_2^2 (sum of squared coordinates).

PyTorch analogy: torch.sum((x - x_hat) ** 2).

def Spec.pcaCumulativeExplainedVarianceSpec {α : Type} [Add α] [Zero α] [TorchLean.Storage α] {inDim outDim : ℕ} (m : PCASpec α inDim outDim) : TorchLean.Tensor α [outDim]

Cumulative explained variance, obtained by prefix-summing explainedVariance.