Controller
This module provides the Controller class, which is responsible for assigning the data to the right set (i.e. train, valid, test). If the configuration is a k-fold, then the controller role is to assign each patch to an area, and then each area to a fold.
Controller Class
The Controller classes are responsible for managing the complete lifecycle of fold creation, assignation of the data to these folds and split these folds into train-valid-test.
Controller Process
The Controller orchestrates the process of generating k-fold splits for dataset training, validation, and testing, with an emphasis on patch-level organization and area-based grouping. Below is a detailed breakdown of its internal workings. Currently, two strategies (i.e. random or naive) are implemented, and they both follow this framework:
Initialization
The controller initializes by loading the
datasetfrom the specifieddataset_path. It sets up the fold indices path (indices_path) and determines whether to loadprecomputed indicesorgeneratenew ones. Thek-fold splitsare generated usinggenerate_kfold_splits, considering the training ratio (k_train). The_create_foldsmethod is invoked to distributepatchesintofoldsbased on theirspatial organization.Preconditions
The preconditions method ensures the validity of k-fold parameters. It checks that
(k * k_train) % 2 == 0, ensuring an even distribution of training data.Fold Creation Process (_create_folds)
This method is responsible for
distributingdatasetpatchesintok-folds. The patches are grouped intoareasbased on theircoordinateswithin the dataset. If anindicesfile exists, the method loadsprecomputedfold assignments. If no precomputed indices exist, it dynamically computesarea groupsand saves them if save_indices is enabled.Area Grouping Strategy
Each patch is assigned to a specific
areabased on itstop-left corner coordinates. Theareacoordinates are determined by thearea_sizeparameter, whileoverlapis accounted for to avoid boundary issues.Patchesthat fall outside their assigned area due to overlap constraints areexcluded.Assigning Areas to Folds
The controller distributes
areasto folds using thedefined strategy(see subtleties)Loading and Saving Fold Assignments
If
save_indicesis True, the computedfold assignmentsare saved toindices_pathfor future use. If an existing indices file is found, it is used instead of recomputing fold assignments.Logging and Debugging
The controller logs key steps, such as: - Loading precomputed indices (if available). - Assigning patches to areas and folds. - Saving computed fold indices. This helps with debugging and reproducibility.
Subtleties:
Area-based Patch Assignment: Ensures that patches close together remain in the same fold, preventing data leakage.
- Implemented strategies:
RandomController(round-robin manner) iterates through all computed area groups. Each area is assigned sequentially to one of the k folds. This ensures an even and systematic distribution of patches across training, validation, and testing sets.NaiveControllerdivides the images into k equal parts and assign each part to a fold.
Reproducibility: Precomputed indices allow for consistent fold assignments across multiple runs.
Scalability: Works efficiently with large datasets by structuring patches into manageable area groups.
RandomController
- class lis2.controller.RandomController(*args, **kwargs)[source]
Bases:
FoldsControllerRandomController for managing and generating k-fold splits with patch-level data organization and area-based grouping in a round-robin manner.
This controller organizes dataset patches into areas and assigns these areas to different folds in a round-robin manner. It supports saving and loading precomputed fold indices for reproducibility and efficiency.
- Configuration:
dataset_path (Union[Path, str]): Path to the dataset file.
k (int): Total number of folds for k-fold cross-validation (default: 1).
k_train (float): Ratio of folds to be used for training (default: 0.80).
indices_path (Union[Path, str]): Path to store or load precomputed fold indices.
save_indices (bool): Whether to save computed indices to a file.
area_size (int): Size of areas to group patches for fold assignment (default: 64).
patch_size (int): Size of each patch in the dataset (default: 32).
overlap (int): Number of pixels overlapping between adjacent areas (default: 0).
- Example Configuration (YAML):
dataset_path: "/path/to/dataset.h5" k: 5 k_train: 0.80 indices_path: "/path/to/indices" save_indices: True area_size: 64 patch_size: 32 overlap: 0
- config_schema: Dict = {'area_size': Schema(type=<class 'int'>, aliases=[], optional=True, default=64), 'dataset_path': Schema(type=<class 'pathlib.Path'>, aliases=[], optional=False, default=None), 'indices_path': Schema(type=<class 'pathlib.Path'>, aliases=[], optional=False, default=None), 'k': Schema(type=<class 'int'>, aliases=['nb_folds'], optional=True, default=1), 'k_train': Schema(type=<class 'float'>, aliases=['train_ratio'], optional=True, default=0.8), 'overlap': Schema(type=<class 'int'>, aliases=[], optional=True, default=0), 'patch_size': Schema(type=<class 'int'>, aliases=[], optional=True, default=32), 'save_indices': Schema(type=<class 'bool'>, aliases=[], optional=False, default=None)}
NaiveController
- class lis2.controller.NaiveController(*args, **kwargs)[source]
Bases:
FoldsControllerManage and generate k-fold splits. It creates the folds by grouping patches into areas and assigning areas to folds in a naive manner (i.e. divides the image into k equal-sized areas).
This controller divides the dataset into k equal-sized areas and assigns them to different folds in a naive manner. It supports saving and loading precomputed fold indices for reproducibility.
- Configuration:
dataset_path (Union[Path, str]): Path to the dataset file.
k (int): Total number of folds for k-fold cross-validation (default: 1).
k_train (float): Ratio of folds to be used for training (default: 0.80).
indices_path (Union[Path, str]): Path to store or load precomputed fold indices.
save_indices (bool): Whether to save computed indices to a file.
area_size (int): Size of areas to group patches for fold assignment (default: 64).
patch_size (int): Size of each patch in the dataset (default: 32).
overlap (int): Number of pixels overlapping between adjacent areas (default: 0).
- Example Configuration (YAML):
dataset_path: "/path/to/dataset.h5" k: 5 k_train: 0.80 indices_path: "/path/to/indices" save_indices: True area_size: 64 patch_size: 32 overlap: 0
- config_schema: Dict = {'area_size': Schema(type=<class 'int'>, aliases=[], optional=True, default=64), 'dataset_path': Schema(type=<class 'pathlib.Path'>, aliases=[], optional=False, default=None), 'indices_path': Schema(type=<class 'pathlib.Path'>, aliases=[], optional=False, default=None), 'k': Schema(type=<class 'int'>, aliases=['nb_folds'], optional=True, default=1), 'k_train': Schema(type=<class 'float'>, aliases=['train_ratio'], optional=True, default=0.8), 'overlap': Schema(type=<class 'int'>, aliases=[], optional=True, default=0), 'patch_size': Schema(type=<class 'int'>, aliases=[], optional=True, default=32), 'save_indices': Schema(type=<class 'bool'>, aliases=[], optional=False, default=None)}
Utils : Split generation
- lis2.controller.generate_kfold_splits(k: int, k_train: float) list[tuple[list[int], list[int], list[int]]][source]
Generate exactly k splits where each fold takes turns being the validation and test set.
This method generates k-fold splits for cross-validation, ensuring that each fold is used once as the validation set and once as the test set. It handles the special case where k = 1 by returning a single split with all data in the training set and empty validation and test sets.
- Parameters:
k (int) – Total number of folds.
k_train (int) – Number of folds in the training set. For k > 1, k_train is typically k - 2.
- Returns:
- A list of tuples, where each tuple contains:
i_train (list[int]): Indices of the training set.
i_valid (list[int]): Indices of the validation set.
i_test (list[int]): Indices of the test set.
- Return type:
list[tuple[list[int], list[int], list[int]]]
- Raises:
ValueError – If invalid parameters are provided, such as k <= 0 or k_train >= k.
A method called generate_kfold_splits is provided. It takes as input k, the number of folds, and k_train, which is the percentage of folds to put in the training set. Some edges cases are not handled :
k must be pair or equal to one
k - (k * k_train) must be equal to 2 or 4.
k_train must be even
Example of usage and output for k=10 and k_train = 0.60:
splits = generate_kfold_splits(10, 0.60)
splits
[
([4, 5, 6, 7, 8, 9], [0, 1], [2, 3]),
([0, 5, 6, 7, 8, 9], [1, 2], [3, 4]),
([0, 1, 6, 7, 8, 9], [2, 3], [4, 5]),
([0, 1, 2, 7, 8, 9], [3, 4], [5, 6]),
([0, 1, 2, 3, 8, 9], [4, 5], [6, 7]),
([0, 1, 2, 3, 4, 9], [5, 6], [7, 8]),
([0, 1, 2, 3, 4, 5], [6, 7], [8, 9]),
([1, 2, 3, 4, 5, 6], [7, 8], [9, 0]),
([2, 3, 4, 5, 6, 7], [8, 9], [0, 1]),
([3, 4, 5, 6, 7, 8], [9, 0], [1, 2]),
]
Adding Custom Controller
If you need a new controller (i.e. if you need to assign differently patches to folds), follow this recipe:
First, make a new class that inherits from FoldsController, and defines it config_schema:
class NewController(FoldsController):
"""
Some documentation about the controller
"""
config_schema = {
"dataset_path": Schema(Union[Path, str]),
"k": Schema(int, aliases=["nb_folds"], default=1),
"k_train": Schema(float, aliases=["train_ratio"], default=0.80),
"indices_path": Schema(Union[Path, str]),
"save_indices": Schema(bool),
"area_size": Schema(int, default=64),
"patch_size": Schema(int, default=32),
"overlap": Schema(int, default=0),
}
In the rest of the controller, the only mandatory method is _create_folds. We suggest to use the __init__ function
to load the dataset, create the splits and get the fold assignments, like this:
def __init__(self):
self.dataset = h5py.File(self.dataset_path, "r")
self.splits = generate_kfold_splits(self.k, self.k_train)
self.fold_assignments = self._create_folds()
Then, you need to implement your own _create_folds methods, where the intelligence of the controller lies. Please refer to
these functions in RandomController and NaiveController for some inspiration. We strongly suggest to store the computed indices
on disk to reduce the computing time and reproducibility for future runs
def _create_folds(self):
# Your code here
return fold_assignements
Finally, you can rewrite a generate_kfold_splits if the function does not cover your use case.