Trackers

This module provides a framework for tracking and logging metrics during model training and evaluation using various tracking systems, including Weights and Biases (Wandb), MLflow, and CSV logging.

BaseTracker

class lis2.trackers.BaseTracker(*args, **kwargs)[source]

Bases: ABC, TypedConfigurable

Abstract base class for all trackers.

This class provides a common interface for logging metrics during model training and evaluation. To create a new tracker, subclass BaseTracker and implement the init, log, and optionally close methods.

Configuration:
  • output_run (str): Path to the directory where logs are stored.

Example:
class MyCustomTracker(BaseTracker):
    def init(self):
        # Initialize custom tracker
        pass

    def log(self, epoch, log_dict):
        # Log metrics using custom logic
        pass

    def close(self):
        # Clean up resources if necessary
        pass

tracker = MyCustomTracker(output_run='./logs')
tracker.init()
tracker.log(epoch=1, log_dict={'loss': 0.1, 'accuracy': 0.9})
tracker.close()
close() → None[source]

Closes the tracker and performs cleanup if needed. Can be optionally overridden by subclasses.

abstractmethod init() → None[source]

Initializes the tracker. Should be implemented by subclasses.

abstractmethod log(epoch: int, log_dict: dict[str, float]) → None[source]

Logs metrics or parameters for the given epoch.

Parameters:
  • epoch (int) – The current epoch number.

  • log_dict (dict[str, float]) – A dictionary of metrics to log.

The BaseTracker class is an abstract base class that provides a common interface for logging metrics. To create a new tracker, subclass BaseTracker and implement the init, log, and optionally close methods.

Trackers Zoo

Wandb Tracker

class lis2.trackers.Wandb(*args, **kwargs)[source]

Bases: BaseTracker

Tracker using Weights and Biases (Wandb) for logging.

Configuration:
  • entity (str): Wandb entity.

close() → None[source]

Closes the Wandb tracker.

config_schema: Dict = {'entity': Schema(type=<class 'str'>, aliases=[], optional=False, default=None)}
init(config: dict[str, Any] | None = None) → None[source]

Initializes the Wandb tracker with the provided configuration.

log(epoch: int, log_dict: dict[str, float]) → None[source]

Logs metrics to Wandb.

Parameters:
  • epoch (int) – The current epoch number.

  • log_dict (dict[str, float]) – A dictionary of metrics to log.

The Wandb class is a tracker that uses Weights and Biases (Wandb) for logging metrics. It initializes a Wandb run and logs metrics using the Wandb API.

Attributes:
  • config_schema (dict): Configuration schema for validating Wandb parameters.

MLflow Tracker

class lis2.trackers.Mlflow(*args, **kwargs)[source]

Bases: BaseTracker

Tracker using MLflow for logging.

Configuration:
  • tracking_uri (str): URI for the MLflow server.

  • experiment_name (str): Name of the MLflow experiment.

close() → None[source]

Ends the current MLflow run.

config_schema: Dict = {'experiment_name': Schema(type=<class 'str'>, aliases=[], optional=False, default=None), 'tracking_uri': Schema(type=<class 'str'>, aliases=[], optional=False, default=None)}
init() → None[source]

Initializes the MLflow tracker and starts a new run.

log(epoch: int, log_dict: dict[str, float]) → None[source]

Logs metrics to MLflow.

Parameters:
  • **epoch** (int) – The current epoch number.

  • **log_dict** (dict[str, float]) – A dictionary of metrics to log.

The Mlflow class is a tracker that uses MLflow for logging metrics. It initializes an MLflow run and logs metrics using the MLflow API.

Attributes:
  • tracking_uri (str): URI for the MLflow server.

  • experiment_name (str): Name of the MLflow experiment.

CSV Logger

class lis2.trackers.CsvLogger(*args, **kwargs)[source]

Bases: BaseTracker

Tracker for logging metrics to a CSV file.

- **csv_filename**

Path to the CSV file where logs are saved.

Type:

str

- **file_initialized**

Indicates whether the CSV file is initialized.

Type:

bool

- **fieldnames**

List of field names for the CSV file.

Type:

list[str]

close() → None[source]

Finalizes the CSV logger by resetting initialization flags.

fieldnames: list[str] | None
init() → None[source]

Initializes the CSV logger. This method does nothing for CSV logging.

log(epoch: int, log_dict: dict[str, float]) → None[source]

Logs metrics to the CSV file.

Parameters:
  • epoch (int) – The current epoch number.

  • log_dict (dict[str, float]) – A dictionary of metrics to log.

The CsvLogger class is a tracker that logs metrics to a CSV file. It initializes the CSV file with headers and logs metrics as rows in the file.

Attributes:
  • csv_filename (str): Path to the CSV file where logs are saved.

  • file_initialized (bool): Indicates whether the CSV file is initialized.

  • fieldnames (list[str]): List of field names for the CSV file.

Trackers Manager

The Trackers class manages multiple trackers and coordinates logging. It initializes all trackers based on the provided configurations and logs metrics to all initialized trackers.

class lis2.trackers.Trackers(loggers_configs: list[dict[str, Any]], output_run: str)[source]

Bases: object

Manages multiple trackers and coordinates logging.

\*\*loggers**

List of initialized tracker instances.

Type:

list[BaseTracker]

\*\*loggers_configs**

Configuration for the trackers.

Type:

list[dict[str, any]]

\*\*output_run**

Path to the directory where logs are stored.

Type:

str

\*\*is_init**

Indicates whether trackers are initialized.

Type:

bool

finish() → None[source]

Finalizes all trackers and performs cleanup.

init() → None[source]

Initializes all trackers based on the provided configurations.

log(epoch: int, log_dict: dict[str, float]) → None[source]

Logs metrics to all initialized trackers.

Parameters:
  • epoch (int) – The current epoch number.

  • log_dict (dict[str, float]) – A dictionary of metrics to log.

loggers: list[BaseTracker]