TorchLean Optimizers

Public optimizer configuration records and runtime optimizer constructors.

@[reducible, inline]

abbrev TorchLean.optim.Optimizer : Type

Optimizer hyperparameter configuration for the supervised training helpers.

This configuration covers the optimizer choices exposed by the public training helpers. It mirrors a few common PyTorch optimizers by name/defaults, but it does not try to cover the full option surface of torch.optim.*.

structure TorchLean.optim.SgdConfig : Type

Public SGD optimizer configuration.

• lr : Float

  Learning rate.

• momentum : Float

  Momentum coefficient.

def TorchLean.optim.instReprSgdConfig.repr : SgdConfig → ℕ → Std.Format

@[implicit_reducible]

instance TorchLean.optim.instReprSgdConfig : Repr SgdConfig

structure TorchLean.optim.AdamConfig : Type

Public Adam optimizer configuration.

• lr : Float

  Learning rate.

• beta1 : Float

  First moment coefficient.

• beta2 : Float

  Second moment coefficient.

• epsilon : Float

  Numerical stabilizer.

@[implicit_reducible]

instance TorchLean.optim.instReprAdamConfig : Repr AdamConfig

def TorchLean.optim.instReprAdamConfig.repr : AdamConfig → ℕ → Std.Format

structure TorchLean.optim.AdamWConfigextends TorchLean.optim.AdamConfig : Type

Public AdamW optimizer configuration.

• lr : Float
• beta1 : Float
• beta2 : Float
• epsilon : Float
• weightDecay : Float

  Decoupled weight decay.

@[implicit_reducible]

instance TorchLean.optim.instReprAdamWConfig : Repr AdamWConfig

def TorchLean.optim.instReprAdamWConfig.repr : AdamWConfig → ℕ → Std.Format

def TorchLean.optim.sgd (cfg : SgdConfig) : Optimizer

SGD optimizer config, written optim.sgd { lr := 0.05 }.

def TorchLean.optim.momentumSGD (cfg : SgdConfig) : Optimizer

Momentum SGD optimizer config. Pass lr and optionally override momentum.

def TorchLean.optim.adam (cfg : AdamConfig) : Optimizer

Adam optimizer config, written optim.adam { lr := 1e-3 }.

def TorchLean.optim.adamw (cfg : AdamWConfig) : Optimizer

AdamW optimizer config, written optim.adamw { lr := 1e-3, weightDecay := 0.01 }.

@[reducible, inline]

abbrev TorchLean.optim.Kind : Type

Optimizer algorithm accepted by simple CLI commands that expose a --optim flag.

@[reducible, inline]

abbrev TorchLean.optim.Kind.parse (s : String) : Except String NN.API.TorchLean.Trainer.OptimizerKind

Parse an optimizer name accepted by a command-line --optim flag.

def TorchLean.optim.Kind.name (kind : Kind) : String

Human-readable optimizer name used in logs.

def TorchLean.optim.Kind.toOptimizer (kind : Kind) (lr : Float) : Optimizer

Build a public optimizer config for this optimizer kind and learning rate.

def TorchLean.optim.runtimeAdam {α : Type} [Runtime.TensorScalar α] (lr beta1 beta2 epsilon : α) {paramShapes : List Shape} : Runtime.Autograd.TorchLean.Optimizer α paramShapes

Runtime Adam optimizer for module-level training.

def TorchLean.optim.runtimeAdamW {α : Type} [Runtime.TensorScalar α] (lr weightDecay beta1 beta2 epsilon : α) {paramShapes : List Shape} : Runtime.Autograd.TorchLean.Optimizer α paramShapes

Runtime AdamW optimizer for module-level training.

def TorchLean.optim.runtimeSGD {α : Type} [Runtime.TensorScalar α] (lr : α) {paramShapes : List Shape} : Runtime.Autograd.TorchLean.Optimizer α paramShapes

Runtime SGD optimizer for module-level training.

def TorchLean.optim.runtimeMomentumSGD {α : Type} [Runtime.TensorScalar α] (lr momentum : α) {paramShapes : List Shape} : Runtime.Autograd.TorchLean.Optimizer α paramShapes

Runtime momentum-SGD optimizer for module-level training.