Controller

This module provides the Controller class, which is responsible for assigning the data to the right set (i.e. train, valid, test). If the configuration is a k-fold, then the controller role is to assign each patch to an area, and then each area to a fold.

Controller Class

The Controller classes are responsible for managing the complete lifecycle of fold creation, assignation of the data to these folds and split these folds into train-valid-test.

class lis2.controller.FoldsController(*args, **kwargs)[source]

Bases: TypedConfigurable

Abstract base class for fold control.

Defines the interface for classes that perform fold assignments using different strategies.

close()[source]

Controller Process

The Controller orchestrates the process of generating k-fold splits for dataset training, validation, and testing, with an emphasis on patch-level organization and area-based grouping. Below is a detailed breakdown of its internal workings. Currently, two strategies (i.e. random or naive) are implemented, and they both follow this framework:

  1. Initialization

    The controller initializes by loading the dataset from the specified dataset_path. It sets up the fold indices path (indices_path) and determines whether to load precomputed indices or generate new ones. The k-fold splits are generated using generate_kfold_splits, considering the training ratio (k_train). The _create_folds method is invoked to distribute patches into folds based on their spatial organization.

  2. Preconditions

    The preconditions method ensures the validity of k-fold parameters. It checks that (k * k_train) % 2 == 0, ensuring an even distribution of training data.

  3. Fold Creation Process (_create_folds)

    This method is responsible for distributing dataset patches into k-folds. The patches are grouped into areas based on their coordinates within the dataset. If an indices file exists, the method loads precomputed fold assignments. If no precomputed indices exist, it dynamically computes area groups and saves them if save_indices is enabled.

  4. Area Grouping Strategy

    Each patch is assigned to a specific area based on its top-left corner coordinates. The area coordinates are determined by the area_size parameter, while overlap is accounted for to avoid boundary issues. Patches that fall outside their assigned area due to overlap constraints are excluded.

  5. Assigning Areas to Folds

    The controller distributes areas to folds using the defined strategy (see subtleties)

  6. Loading and Saving Fold Assignments

    If save_indices is True, the computed fold assignments are saved to indices_path for future use. If an existing indices file is found, it is used instead of recomputing fold assignments.

  7. Logging and Debugging

    The controller logs key steps, such as: - Loading precomputed indices (if available). - Assigning patches to areas and folds. - Saving computed fold indices. This helps with debugging and reproducibility.

Subtleties:

  • Area-based Patch Assignment: Ensures that patches close together remain in the same fold, preventing data leakage.

  • Implemented strategies:
    • RandomController (round-robin manner) iterates through all computed area groups. Each area is assigned sequentially to one of the k folds. This ensures an even and systematic distribution of patches across training, validation, and testing sets.

    • NaiveController divides the images into k equal parts and assign each part to a fold.

  • Reproducibility: Precomputed indices allow for consistent fold assignments across multiple runs.

  • Scalability: Works efficiently with large datasets by structuring patches into manageable area groups.

RandomController

class lis2.controller.RandomController(*args, **kwargs)[source]

Bases: FoldsController

RandomController for managing and generating k-fold splits with patch-level data organization and area-based grouping in a round-robin manner.

This controller organizes dataset patches into areas and assigns these areas to different folds in a round-robin manner. It supports saving and loading precomputed fold indices for reproducibility and efficiency.

Configuration:
  • dataset_path (Union[Path, str]): Path to the dataset file.

  • k (int): Total number of folds for k-fold cross-validation (default: 1).

  • k_train (float): Ratio of folds to be used for training (default: 0.80).

  • indices_path (Union[Path, str]): Path to store or load precomputed fold indices.

  • save_indices (bool): Whether to save computed indices to a file.

  • area_size (int): Size of areas to group patches for fold assignment (default: 64).

  • patch_size (int): Size of each patch in the dataset (default: 32).

  • overlap (int): Number of pixels overlapping between adjacent areas (default: 0).

Example Configuration (YAML):
dataset_path: "/path/to/dataset.h5"
k: 5
k_train: 0.80
indices_path: "/path/to/indices"
save_indices: True
area_size: 64
patch_size: 32
overlap: 0
config_schema: Dict = {'area_size': Schema(type=<class 'int'>, aliases=[], optional=True, default=64), 'dataset_path': Schema(type=<class 'pathlib.Path'>, aliases=[], optional=False, default=None), 'indices_path': Schema(type=<class 'pathlib.Path'>, aliases=[], optional=False, default=None), 'k': Schema(type=<class 'int'>, aliases=['nb_folds'], optional=True, default=1), 'k_train': Schema(type=<class 'float'>, aliases=['train_ratio'], optional=True, default=0.8), 'overlap': Schema(type=<class 'int'>, aliases=[], optional=True, default=0), 'patch_size': Schema(type=<class 'int'>, aliases=[], optional=True, default=32), 'save_indices': Schema(type=<class 'bool'>, aliases=[], optional=False, default=None)}
preconditions()[source]

Check if all preconditions are met before running the algorithm.

NaiveController

class lis2.controller.NaiveController(*args, **kwargs)[source]

Bases: FoldsController

Manage and generate k-fold splits. It creates the folds by grouping patches into areas and assigning areas to folds in a naive manner (i.e. divides the image into k equal-sized areas).

This controller divides the dataset into k equal-sized areas and assigns them to different folds in a naive manner. It supports saving and loading precomputed fold indices for reproducibility.

Configuration:
  • dataset_path (Union[Path, str]): Path to the dataset file.

  • k (int): Total number of folds for k-fold cross-validation (default: 1).

  • k_train (float): Ratio of folds to be used for training (default: 0.80).

  • indices_path (Union[Path, str]): Path to store or load precomputed fold indices.

  • save_indices (bool): Whether to save computed indices to a file.

  • area_size (int): Size of areas to group patches for fold assignment (default: 64).

  • patch_size (int): Size of each patch in the dataset (default: 32).

  • overlap (int): Number of pixels overlapping between adjacent areas (default: 0).

Example Configuration (YAML):
dataset_path: "/path/to/dataset.h5"
k: 5
k_train: 0.80
indices_path: "/path/to/indices"
save_indices: True
area_size: 64
patch_size: 32
overlap: 0
config_schema: Dict = {'area_size': Schema(type=<class 'int'>, aliases=[], optional=True, default=64), 'dataset_path': Schema(type=<class 'pathlib.Path'>, aliases=[], optional=False, default=None), 'indices_path': Schema(type=<class 'pathlib.Path'>, aliases=[], optional=False, default=None), 'k': Schema(type=<class 'int'>, aliases=['nb_folds'], optional=True, default=1), 'k_train': Schema(type=<class 'float'>, aliases=['train_ratio'], optional=True, default=0.8), 'overlap': Schema(type=<class 'int'>, aliases=[], optional=True, default=0), 'patch_size': Schema(type=<class 'int'>, aliases=[], optional=True, default=32), 'save_indices': Schema(type=<class 'bool'>, aliases=[], optional=False, default=None)}
preconditions()[source]

Check if all preconditions are met before running the algorithm.

Utils : Split generation

lis2.controller.generate_kfold_splits(k: int, k_train: float) → list[tuple[list[int], list[int], list[int]]][source]

Generate exactly k splits where each fold takes turns being the validation and test set.

This method generates k-fold splits for cross-validation, ensuring that each fold is used once as the validation set and once as the test set. It handles the special case where k = 1 by returning a single split with all data in the training set and empty validation and test sets.

Parameters:
  • k (int) – Total number of folds.

  • k_train (int) – Number of folds in the training set. For k > 1, k_train is typically k - 2.

Returns:

A list of tuples, where each tuple contains:
  • i_train (list[int]): Indices of the training set.

  • i_valid (list[int]): Indices of the validation set.

  • i_test (list[int]): Indices of the test set.

Return type:

list[tuple[list[int], list[int], list[int]]]

Raises:

ValueError – If invalid parameters are provided, such as k <= 0 or k_train >= k.

A method called generate_kfold_splits is provided. It takes as input k, the number of folds, and k_train, which is the percentage of folds to put in the training set. Some edges cases are not handled :

  • k must be pair or equal to one

  • k - (k * k_train) must be equal to 2 or 4.

  • k_train must be even

Example of usage and output for k=10 and k_train = 0.60:

splits = generate_kfold_splits(10, 0.60)
splits
[
    ([4, 5, 6, 7, 8, 9], [0, 1], [2, 3]),
    ([0, 5, 6, 7, 8, 9], [1, 2], [3, 4]),
    ([0, 1, 6, 7, 8, 9], [2, 3], [4, 5]),
    ([0, 1, 2, 7, 8, 9], [3, 4], [5, 6]),
    ([0, 1, 2, 3, 8, 9], [4, 5], [6, 7]),
    ([0, 1, 2, 3, 4, 9], [5, 6], [7, 8]),
    ([0, 1, 2, 3, 4, 5], [6, 7], [8, 9]),
    ([1, 2, 3, 4, 5, 6], [7, 8], [9, 0]),
    ([2, 3, 4, 5, 6, 7], [8, 9], [0, 1]),
    ([3, 4, 5, 6, 7, 8], [9, 0], [1, 2]),
]

Adding Custom Controller

If you need a new controller (i.e. if you need to assign differently patches to folds), follow this recipe:

First, make a new class that inherits from FoldsController, and defines it config_schema:

class NewController(FoldsController):
    """
    Some documentation about the controller
    """

    config_schema = {
        "dataset_path": Schema(Union[Path, str]),
        "k": Schema(int, aliases=["nb_folds"], default=1),
        "k_train": Schema(float, aliases=["train_ratio"], default=0.80),
        "indices_path": Schema(Union[Path, str]),
        "save_indices": Schema(bool),
        "area_size": Schema(int, default=64),
        "patch_size": Schema(int, default=32),
        "overlap": Schema(int, default=0),
    }

In the rest of the controller, the only mandatory method is _create_folds. We suggest to use the __init__ function to load the dataset, create the splits and get the fold assignments, like this:

def __init__(self):
    self.dataset = h5py.File(self.dataset_path, "r")
    self.splits = generate_kfold_splits(self.k, self.k_train)
    self.fold_assignments = self._create_folds()

Then, you need to implement your own _create_folds methods, where the intelligence of the controller lies. Please refer to these functions in RandomController and NaiveController for some inspiration. We strongly suggest to store the computed indices on disk to reduce the computing time and reproducibility for future runs

def _create_folds(self):
    # Your code here
    return fold_assignements

Finally, you can rewrite a generate_kfold_splits if the function does not cover your use case.