Trackers
This module provides a framework for tracking and logging metrics during model training and evaluation using various tracking systems, including Weights and Biases (Wandb), MLflow, and CSV logging.
BaseTracker
- class lis2.trackers.BaseTracker(*args, **kwargs)[source]
Bases:
ABC,TypedConfigurableAbstract base class for all trackers.
This class provides a common interface for logging metrics during model training and evaluation. To create a new tracker, subclass
BaseTrackerand implement theinit,log, and optionallyclosemethods.- Configuration:
output_run (str): Path to the directory where logs are stored.
- Example:
class MyCustomTracker(BaseTracker): def init(self): # Initialize custom tracker pass def log(self, epoch, log_dict): # Log metrics using custom logic pass def close(self): # Clean up resources if necessary pass tracker = MyCustomTracker(output_run='./logs') tracker.init() tracker.log(epoch=1, log_dict={'loss': 0.1, 'accuracy': 0.9}) tracker.close()
The BaseTracker class is an abstract base class that provides a common interface for logging metrics. To create a new tracker, subclass BaseTracker and implement the init, log, and optionally close methods.
Trackers Zoo
Wandb Tracker
- class lis2.trackers.Wandb(*args, **kwargs)[source]
Bases:
BaseTrackerTracker using Weights and Biases (Wandb) for logging.
- Configuration:
entity (str): Wandb entity.
- config_schema: Dict = {'entity': Schema(type=<class 'str'>, aliases=[], optional=False, default=None)}
The Wandb class is a tracker that uses Weights and Biases (Wandb) for logging metrics. It initializes a Wandb run and logs metrics using the Wandb API.
- Attributes:
config_schema (dict): Configuration schema for validating Wandb parameters.
MLflow Tracker
- class lis2.trackers.Mlflow(*args, **kwargs)[source]
Bases:
BaseTrackerTracker using MLflow for logging.
- Configuration:
tracking_uri (str): URI for the MLflow server.
experiment_name (str): Name of the MLflow experiment.
- config_schema: Dict = {'experiment_name': Schema(type=<class 'str'>, aliases=[], optional=False, default=None), 'tracking_uri': Schema(type=<class 'str'>, aliases=[], optional=False, default=None)}
The Mlflow class is a tracker that uses MLflow for logging metrics. It initializes an MLflow run and logs metrics using the MLflow API.
- Attributes:
tracking_uri (str): URI for the MLflow server.
experiment_name (str): Name of the MLflow experiment.
CSV Logger
- class lis2.trackers.CsvLogger(*args, **kwargs)[source]
Bases:
BaseTrackerTracker for logging metrics to a CSV file.
- - **csv_filename**
Path to the CSV file where logs are saved.
- Type:
str
- - **file_initialized**
Indicates whether the CSV file is initialized.
- Type:
bool
- - **fieldnames**
List of field names for the CSV file.
- Type:
list[str]
- fieldnames: list[str] | None
The CsvLogger class is a tracker that logs metrics to a CSV file. It initializes the CSV file with headers and logs metrics as rows in the file.
- Attributes:
csv_filename (str): Path to the CSV file where logs are saved.
file_initialized (bool): Indicates whether the CSV file is initialized.
fieldnames (list[str]): List of field names for the CSV file.
Trackers Manager
The Trackers class manages multiple trackers and coordinates logging. It initializes all trackers based on the provided configurations and logs metrics to all initialized trackers.
- class lis2.trackers.Trackers(loggers_configs: list[dict[str, Any]], output_run: str)[source]
Bases:
objectManages multiple trackers and coordinates logging.
- \*\*loggers**
List of initialized tracker instances.
- Type:
list[BaseTracker]
- \*\*loggers_configs**
Configuration for the trackers.
- Type:
list[dict[str, any]]
- \*\*output_run**
Path to the directory where logs are stored.
- Type:
str
- \*\*is_init**
Indicates whether trackers are initialized.
- Type:
bool
- log(epoch: int, log_dict: dict[str, float]) None[source]
Logs metrics to all initialized trackers.
- Parameters:
epoch (int) – The current epoch number.
log_dict (dict[str, float]) – A dictionary of metrics to log.
- loggers: list[BaseTracker]