Optimizers
This module provides a framework for integrating PyTorch optimizers with a configurable schema, allowing dynamic configuration and extension of optimizers.
Adding a Custom Optimizer
To add a custom optimizer, you need to define a new class that extends the BaseOptimizer class. You can configure the optimizer using a schema and implement the optimization logic in the step method.
Registering Optimizers
The module automatically registers all valid optimizer classes from torch.optim as subclasses of BaseOptimizer. This allows for seamless integration and configuration of PyTorch’s built-in optimizers.
For more information on the available optimizers and their parameters, refer to the official PyTorch documentation: torch.optim.
BaseOptimizer
- class lis2.optimizer.BaseOptimizer(*args, **kwargs)[source]
Bases:
TypedConfigurable,OptimizerBase class for PyTorch optimizers integrated with TypedConfigurable.
Enables dynamic subclass generation for each optimizer. Check torch.optim documentation for more information on how to implement custom optimizers.
- Configuration:
lr (float): The learning rate for the optimizer.
Example
Here’s how you can create and use a custom optimizer by inheriting from BaseOptimizer:
import torch from torch.optim.optimizer import Optimizer class MyCustomOptimizer(BaseOptimizer): schema = { "lr": Schema(float, optional=True, default=0.01), } def __init__(self, params): defaults = {"lr": self.lr} super().__init__(params, defaults) def step(self, closure=None): # Implementation of the optimization step for group in self.param_groups: for p in group['params']: if p.grad is None: continue d_p = p.grad.data p.data.add_(-group['lr'], d_p) # Usage example model = MyModel() # Your PyTorch model optimizer = MyCustomOptimizer(model.parameters(), lr=0.01)
- config_schema: Dict = {'lr': Schema(type=<class 'float'>, aliases=[], optional=True, default=0.01)}
The BaseOptimizer class serves as the base class for integrating PyTorch optimizers with the TypedConfigurable framework. It enables dynamic subclass generation for each optimizer, allowing for easy configuration and extension.