TorchLean API

NN.Spec.RL

Spec reinforcement learning #

Umbrella import for RL environments and MDP specifications.

The files here keep transition systems, rewards, and policy-facing environment contracts separate from runtime training code, so RL examples can be checked against a small pure semantics.

Folder map:

The names MDP, ValueFunction, and bellmanPolicy appear in multiple namespaces on purpose: Spec.RL for deterministic finite MDPs, Spec.RL.FiniteStochastic for finite stochastic MDPs, and Spec.RL.Markov for measurable-space MDPs. Keeping these layers separate avoids forcing every RL development into the heaviest probability-theory abstraction.