Engine

The engine is the subsystem that executes a run. The two currently implemented concrete engines are the trainer, which fits a model, and the batch inference engine, which applies a trained model to new data. Both share the structure described here.

An engine is constructed from a configuration file (YAML, JSON, or TOML). From its configuration it lazily assembles four subsystems:

• Services supply shared, run-scoped capabilities and data.

• Policy defines the task and the contracts each component must satisfy.

• Components are the configurable building blocks of the model.

• Callbacks observe and extend the run through lifecycle hooks.

A run is started by calling execute. Engines can be wrapped for distributed, multi-rank execution without changing their configuration.

Engine Subsystems

• Services
  • How it Works
  • Registering a new service
• Policy
  • Usage
  • How it works
  • Variants
  • Registering a new policy
• Components
  • Model
  • Transformer
  • Normalizer
  • Optimizer
  • Loss
• Callbacks