Models

This module provides a collection of deep learning models implemented in PyTorch. Each model is designed for different tasks, including segmentation, classification, and denoising.

Adding a Custom Model

This tutorial walks you through the process of adding a custom model to the existing codebase. We will define a new model by extending the BaseModel class, configure it, and update the __init__.py file to ensure the model is accessible.

Prerequisites

Before you start, ensure you have the following:

  • Basic understanding of Python and PyTorch.

  • Familiarity with the existing codebase and its structure.

  • Access to the necessary datasets and configuration files.

Step 1: Define the Custom Model Class

To create a custom model, you need to define a new class that extends the BaseModel class. This class should implement the abstract methods: core_forward, preprocess_forward, and postprocess_forward.

Here is an example of how to define a custom model class:

from models import BaseModel, nn
from torch import nn
from configurable import TypedConfigurable, Schema

class CustomModel(BaseModel):
    """
    CustomModel is an example model extending BaseModel.

    This model implements specific logic for preprocessing, core processing,
    and postprocessing steps. It uses a configurable number of layers.

    Configuration:
        - name (str): The name of the model.
        - type (str): The type of the model.
        - n_layer (int): The number of layers in the model. Default is 5.
    """

    config_schema = {
        "n_layer": Schema(int, default=5)
    }

    def __init__(self, *args, **kwargs):
        super().__init__(*args, **kwargs)
        # Initialize layers based on the configuration
        self.layers = nn.ModuleList([nn.Linear(10, 10) for _ in range(self.n_layer)])

    def core_forward(self, x: torch.Tensor) -> torch.Tensor:
        # Example core logic using multiple layers
        for layer in self.layers:
            x = layer(x)
        return x

    def preprocess_forward(self, *args, **kwargs) -> torch.Tensor:
        # Example preprocessing logic
        data, _ = args[0]  # Assuming args[0] is a tuple from the dataset
        return data.float() / 255.0  # Normalize data

    def postprocess_forward(self, x: torch.Tensor) -> torch.Tensor:
        # Example postprocessing logic
        loss = torch.mean(x)  # Example loss computation
        return loss

Step 2: Configure the Custom Model

After defining the custom model class, you need to configure it using a YAML configuration file. Here is an example configuration tuto_config.yml:

name: "custom_model"
type: "CustomModel"
n_layer: 10

Make sure to place this configuration file in the appropriate directory and reference it when initializing the model.

Step 3: Update the __init__.py File

To ensure that your custom model is accessible, you need to update the __init__.py file in the appropriate module directory. This file should import your custom model class so that it can be easily accessed by other parts of the codebase.

Here is an example of how to update the __init__.py file:

from .custom_model import CustomModel

__all__ = [
    'BaseModel',
    'CustomModel',  # Add your custom model here
    # Other models and classes
]

This ensures that CustomModel is imported when the module is imported, making it accessible for use in other parts of the application.

Step 4: Initialize and Use the Custom Model

To initialize and use the custom model, you can use the following code:

from models import CustomModel

model = CustomModel.from_config("/tuto_config.yml")
print(model)

Conclusion

You have now successfully added a custom model to the codebase by extending the BaseModel class, configuring it properly, and updating the __init__.py file. You can further customize the model by modifying the logic in the core_forward, preprocess_forward, and postprocess_forward methods according to your needs.

For more detailed information on each component, refer to the respective sections in the documentation:

BaseModel Class

class lis2.models.BaseModel(*args, **kwargs)[source]

Bases: ABC, TypedConfigurable, Module

BaseModel for creating neural network models with preprocessing, core processing, and postprocessing steps.

This class serves as a template for creating neural network models following the Template Method design pattern. It defines a structure that includes preprocessing of input data, core processing where the main model logic resides, and postprocessing to compute the final output or loss. Subclasses must implement the abstract methods to define specific behavior for these steps.

Important note:

  • The forward method orchestrates the sequence of operations and should not be overridden. Instead, subclasses should implement the core_forward, preprocess_forward, and postprocess_forward methods to define their specific processing logic.

  • The arguments *args: Any, **kwargs: Any in the forward method are dependent on the dataset used, as defined in BaseDataset.

Configuration:
  • name (str): The name of the model.

Example Configuration for from_snapshot (YAML):
name: "example_model"
abstractmethod core_forward(*args: Any, **kwargs: Any) → Any[source]

Abstract method for the core forward pass logic. Must be implemented by subclasses.

This method contains the main logic of the model, such as the neural network layers and their interactions. It processes the preprocessed input data and produces an output that will be further processed to compute the loss. The input to this method is the output from the preprocess_forward method.

Parameters:
  • *args – Variable length arguments for the core forward pass. These arguments are dependent on the output of the preprocess_forward method.

  • **kwargs – Arbitrary keyword arguments for the core forward pass.

Returns:

The processed output from the core forward pass, which will be used in the

postprocess_forward method to compute the loss.

Return type:

Any

forward(*args: Any, **kwargs: Any) → Tensor[source]

Forward pass of the model, which includes preprocessing, core processing, and postprocessing.

This method orchestrates the sequence of operations for the forward pass:

  1. Preprocess the input data using preprocess_forward().

  2. Apply the core model logic using core_forward().

  3. Postprocess the output to compute the loss using postprocess_forward().

Note: This method should not be overridden in subclasses. Instead, implement the abstract methods core_forward(), preprocess_forward(), and postprocess_forward().

classmethod from_snapshot(snapshot: dict[str, Any]) → BaseModel[source]

Load the model from a snapshot.

This method loads a model from a snapshot, which can be a dictionary containing the model’s configuration and state, or a path to a file containing this information. It recursively searches for required keys in the snapshot dictionary to extract the model configuration and state, then initializes the model and loads its state.

Parameters:

snapshot (dict[str, Any]) – A dictionary containing the snapshot or a path to the snapshot file.

Returns:

An instance of the model loaded from the snapshot.

Return type:

BaseModel

Raises:

ValueError – If the snapshot is missing required keys or cannot be loaded.

abstractmethod postprocess_forward(x: Any) → Tensor[source]

Postprocess the output of the core forward pass to compute the loss.

This method takes the output from the core forward pass and computes the final loss value. It can include operations such as applying a loss function or any other final transformations. The input to this method is the output from the core_forward method.

Parameters:

x (Any) – The output from the core forward pass.

Returns:

The computed loss value.

Return type:

torch.Tensor

abstractmethod preprocess_forward(*args: Any, **kwargs: Any) → Any[source]

Abstract method for preprocessing input data before the core forward pass. Must be implemented by subclasses.

This method handles any necessary preprocessing of the input data, such as normalization, reshaping, or any other transformations required before feeding the data into the core model logic. The input to this method is dependent on the output of the __getitem__ method of the dataset.

Parameters:
  • *args – Variable length arguments for preprocessing. These arguments are dependent on the dataset’s __getitem__ method.

  • **kwargs – Arbitrary keyword arguments for preprocessing.

Returns:

The preprocessed input data, which will be passed to the core_forward method.

Return type:

Any

The BaseModel class serves as the foundation for all models, providing common functionalities such as initialization, weight loading, and saving.

Models Zoo

UNet Model

class lis2.models.BaseUNet(*args, **kwargs)[source]

Bases: BaseModel

BaseUNet for configurable UNet implementations with customizable blocks and position encoder.

A base class for UNet implementations that allows configuration of various parameters such as input/output channels, features, number of blocks, dimensionality, and encoder settings. This class provides a flexible structure for building UNet models with different configurations.

Configuration:
  • name (str): The name of the UNet model.

  • in_channels (int): Number of input channels.

  • out_channels (int): Number of output channels.

  • features (int): Number of features in the UNet.

  • num_blocks (int): Number of blocks in the UNet.

  • dim (int): Dimensionality of the UNet (e.g., 2D, 3D).

  • encoder (Config, optional): Configuration for the encoder (BaseEncoder). Default is None.

  • encoder_cat_position (Literal[“before”, “middle”, “after”], optional): Position to concatenate the encoder output. Default is “before”.

Example Configuration:
config = {
    "name": "example_unet",
    "in_channels": 3,
    "out_channels": 1,
    "features": 64,
    "num_blocks": 4,
    "dim": 2,
    "encoder": "configs/encoder/encoderLin.yml",
    "encoder_cat_position": "before"
}
Aliases:
  • unet

  • base_unet

aliases = ['unet', 'base_unet']
config_schema: Dict = {'dim': Schema(type=<class 'int'>, aliases=['dimension'], optional=False, default=None), 'encoder': Schema(type=typing.Union[dict, str], aliases=[], optional=True, default=None), 'encoder_cat_position': Schema(type=typing.Literal['before', 'middle', 'after'], aliases=['encoder_pos'], optional=True, default=None), 'features': Schema(type=<class 'int'>, aliases=[], optional=False, default=None), 'in_channels': Schema(type=<class 'int'>, aliases=[], optional=False, default=None), 'num_blocks': Schema(type=<class 'int'>, aliases=[], optional=False, default=None), 'out_channels': Schema(type=<class 'int'>, aliases=[], optional=False, default=None)}
core_forward(batch: Tensor | tuple[Tensor, Tensor | None]) → Tensor[source]

Core forward pass of the UNet model.

Parameters:

batch (Union[torch.Tensor, tuple[torch.Tensor, Optional[torch.Tensor]]]) – Input batch, which can be a tensor or a tuple containing the tensor and position encoding.

Returns:

The output tensor after the forward pass.

Return type:

torch.Tensor

init_layers()[source]

Initialize the layers of the UNet model.

postprocess_forward(output: Tensor, **kwargs) → Tensor[source]

Postprocess the output data after the core forward pass.

Parameters:
  • output (torch.Tensor) – Output tensor.

  • **kwargs – Additional keyword arguments.

Returns:

The postprocessed output tensor.

Return type:

torch.Tensor

preprocess_forward(patch: Tensor, positions: Tensor | None = None, **kwargs) → tuple[Tensor, Tensor | None][source]

Preprocess the input data before the core forward pass.

Parameters:
  • patch (torch.Tensor) – Input tensor.

  • positions (Optional[torch.Tensor]) – Position encoding tensor.

  • **kwargs – Additional keyword arguments.

Returns:

The preprocessed input tensor and position encoding.

Return type:

tuple[torch.Tensor, Optional[torch.Tensor]]

The BaseUNet class implements a generic U-Net architecture, which is widely used for image segmentation tasks. It supports customizable depth and feature sizes.

Example usage:

from models.base_unet import BaseUNet

config = {
    "name": "example_unet",
    "in_channels": 3,
    "out_channels": 1,
    "features": 64,
    "num_blocks": 4,
    "dim": 2
}

model = BaseUNet.from_config(config)
print(model)

Encoder Position Enum

The encoder_pos enumeration specifies the position of the encoder in the U-Net architecture. The possible values are:

  • BEFORE: The encoder is placed before the encoding process begins. This configuration concatenates the position encoding with the input tensor at the start of the forward pass.

  • MIDDLE: The encoder is placed after the bottleneck layer. This configuration concatenates the position encoding with the tensor immediately after the bottleneck, influencing the upsampling path of the U-Net.

  • AFTER: The encoder is placed after the decoding process is complete. This configuration concatenates the position encoding with the output tensor at the end of the forward pass.

These positions determine when the position encoding is integrated into the U-Net model’s forward pass, affecting how spatial information is utilized throughout the network.

CNN1D Model

class lis2.models.CNN1D(*args, **kwargs)[source]

Bases: BaseModel

1D CNN model for processing one-dimensional data.

This class implements a 1D Convolutional Neural Network (CNN) model designed to process one-dimensional data. It includes a sequence of convolutional layers, ReLU activations, pooling, dropout, and fully connected layers.

Configuration: name (str): The name of the 1D CNN model. in_channels (int): Number of input channels. Default is 1. out_channels (int): Number of output channels. Default is 1. kernel_size (int): Size of the convolutional kernel. Default is 16. stride (int): Stride for the convolution. Default is 2. padding (int): Padding for the convolution. Default is 0. dropout_rate (float): Dropout rate for the dropout layers. Default is 0.5. linear_features (int): Number of features in the linear layers. Default is 64.

Example Configuration (Python):
config = {
    "name": "example_cnn1d",
    "in_channels": 1,
    "out_channels": 1,
    "kernel_size": 16,
    "stride": 2,
    "padding": 0,
    "dropout_rate": 0.5,
    "linear_features": 64
}
config_schema: Dict = {'dropout_rate': Schema(type=<class 'float'>, aliases=[], optional=True, default=0.5), 'in_channels': Schema(type=<class 'int'>, aliases=[], optional=True, default=1), 'kernel_size': Schema(type=<class 'int'>, aliases=[], optional=True, default=16), 'linear_features': Schema(type=<class 'int'>, aliases=[], optional=True, default=64), 'out_channels': Schema(type=<class 'int'>, aliases=[], optional=True, default=1), 'padding': Schema(type=<class 'int'>, aliases=[], optional=True, default=0), 'stride': Schema(type=<class 'int'>, aliases=[], optional=True, default=2)}
core_forward(x)[source]

Core forward pass of the 1D CNN model.

Parameters:

x (torch.Tensor) – Input tensor.

Returns:

The output tensor after the forward pass.

Return type:

torch.Tensor

postprocess_forward(x: Any) → Tensor[source]

Postprocess the output data after the core forward pass.

Parameters:

x (Any) – Output tensor.

Returns:

The postprocessed output tensor.

Return type:

torch.Tensor

preprocess_forward(patch, *args, **kwargs)[source]

Preprocess the input data before the core forward pass.

Parameters:
  • patch (torch.Tensor) – Input tensor.

  • *args – Additional positional arguments.

  • **kwargs – Additional keyword arguments.

Returns:

The preprocessed input tensor.

Return type:

torch.Tensor

The CNN1D model implements a 1D convolutional neural network architecture, suitable for tasks involving sequential or time-series data.

DNCNN Model

class lis2.models.DnCNN(*args, **kwargs)[source]

Bases: BaseModel

DnCNN model for image denoising.

This class implements the DnCNN (Denoising Convolutional Neural Network) model, which is designed for image denoising tasks. The model consists of multiple convolutional layers with ReLU activations and batch normalization.

Configuration: name (str): The name of the DnCNN model. num_layers (int): Number of layers in the model. Default is 10. num_features (int): Number of features in the convolutional layers. Default is 64. kernel_size (int): Size of the convolutional kernel. Default is 3. stride (int): Stride for the convolution. Default is 1. padding (int): Padding for the convolution. Default is 1. use_batch_norm (bool): Whether to use batch normalization. Default is True. activation (str): Activation function to use. Default is ‘relu’.

Example Configuration (Python):
config = {
    "name": "example_dncnn",
    "num_layers": 10,
    "num_features": 64,
    "kernel_size": 3,
    "stride": 1,
    "padding": 1,
    "use_batch_norm": True,
    "activation": "relu"
}

Aliases:

dncnn denoising_cnn

config_schema: Dict = {'activation': Schema(type=<class 'str'>, aliases=[], optional=True, default=relu), 'kernel_size': Schema(type=<class 'int'>, aliases=[], optional=True, default=3), 'num_features': Schema(type=<class 'int'>, aliases=[], optional=True, default=64), 'num_layers': Schema(type=<class 'int'>, aliases=[], optional=True, default=10), 'padding': Schema(type=<class 'int'>, aliases=[], optional=True, default=1), 'stride': Schema(type=<class 'int'>, aliases=[], optional=True, default=1), 'use_batch_norm': Schema(type=<class 'bool'>, aliases=[], optional=True, default=True)}
core_forward(batch)[source]

Core forward pass of the DnCNN model.

Parameters:

batch (torch.Tensor) – Input batch tensor.

Returns:

The denoised output tensor.

Return type:

torch.Tensor

postprocess_forward(patch, *args, **kwargs)[source]

Postprocess the input data after the core forward pass.

Parameters:
  • patch (torch.Tensor) – Input tensor.

  • *args – Additional positional arguments.

  • **kwargs – Additional keyword arguments.

Returns:

The postprocessed output tensor.

Return type:

torch.Tensor

preprocess_forward(patch, *args, **kwargs)[source]

Preprocess the input data before the core forward pass.

Parameters:
  • patch (torch.Tensor) – Input tensor.

  • *args – Additional positional arguments.

  • **kwargs – Additional keyword arguments.

Returns:

The preprocessed input tensor.

Return type:

torch.Tensor

The DnCNN model is a denoising convolutional neural network, often used for image denoising applications.

Unet++ Model

class lis2.models.UNetPP(*args, **kwargs)[source]

Bases: BaseModel

UNetPlusPlus model for image segmentation.

This class implements the UNet++ architecture, which is an enhanced version of the UNet model designed for image segmentation tasks. It includes additional skip connections and deep supervision to improve performance.

Configuration: name (str): The name of the UNet++ model. in_channels (int): Number of input channels. Default is 1. n_classes (int): Number of output channels. Default is 1. feature_scale (int): Scale factor for the number of features in each layer. Default is 4. is_deconv (bool): Whether to use deconvolution layers for upsampling. Default is True. is_batchnorm (bool): Whether to use batch normalization layers. Default is True. is_ds (bool): Whether to use deep supervision. Default is True. filters (list): List of filter sizes for each layer. Default is [64, 128, 256, 512, 1024]. kernel_size (int): Size of the convolutional kernel. Default is 3. padding (int): Padding for the convolution. Default is 1. activation (str): Activation function to use. Default is ‘relu’.

Example Configuration (Python):
config = {
    "name": "example_unetpp",
    "in_channels": 1,
    "n_classes": 1,
    "feature_scale": 4,
    "is_deconv": True,
    "is_batchnorm": True,
    "is_ds": True,
    "filters": [64, 128, 256, 512, 1024],
    "kernel_size": 3,
    "padding": 1,
    "activation": "relu"
}

Aliases:

unetpp

aliases = ['unetpp']
config_schema: Dict = {'activation': Schema(type=<class 'str'>, aliases=[], optional=True, default=relu), 'feature_scale': Schema(type=<class 'int'>, aliases=[], optional=True, default=4), 'filters': Schema(type=<class 'list'>, aliases=[], optional=True, default=[64, 128, 256, 512, 1024]), 'in_channels': Schema(type=<class 'int'>, aliases=[], optional=True, default=1), 'is_batchnorm': Schema(type=<class 'bool'>, aliases=[], optional=True, default=True), 'is_deconv': Schema(type=<class 'bool'>, aliases=[], optional=True, default=True), 'is_ds': Schema(type=<class 'bool'>, aliases=[], optional=True, default=True), 'kernel_size': Schema(type=<class 'int'>, aliases=[], optional=True, default=3), 'n_classes': Schema(type=<class 'int'>, aliases=[], optional=True, default=1), 'padding': Schema(type=<class 'int'>, aliases=[], optional=True, default=1)}
core_forward(x)[source]

Abstract method for the core forward pass logic. Must be implemented by subclasses.

This method contains the main logic of the model, such as the neural network layers and their interactions. It processes the preprocessed input data and produces an output that will be further processed to compute the loss. The input to this method is the output from the preprocess_forward method.

Parameters:
  • *args – Variable length arguments for the core forward pass. These arguments are dependent on the output of the preprocess_forward method.

  • **kwargs – Arbitrary keyword arguments for the core forward pass.

Returns:

The processed output from the core forward pass, which will be used in the

postprocess_forward method to compute the loss.

Return type:

Any

postprocess_forward(x: Any) → Tensor[source]

Postprocess the output of the core forward pass to compute the loss.

This method takes the output from the core forward pass and computes the final loss value. It can include operations such as applying a loss function or any other final transformations. The input to this method is the output from the core_forward method.

Parameters:

x (Any) – The output from the core forward pass.

Returns:

The computed loss value.

Return type:

torch.Tensor

preprocess_forward(patch, *args, **kwargs)[source]

Abstract method for preprocessing input data before the core forward pass. Must be implemented by subclasses.

This method handles any necessary preprocessing of the input data, such as normalization, reshaping, or any other transformations required before feeding the data into the core model logic. The input to this method is dependent on the output of the __getitem__ method of the dataset.

Parameters:
  • *args – Variable length arguments for preprocessing. These arguments are dependent on the dataset’s __getitem__ method.

  • **kwargs – Arbitrary keyword arguments for preprocessing.

Returns:

The preprocessed input data, which will be passed to the core_forward method.

Return type:

Any

The UNetPP model extends the U-Net architecture with dense connections, improving performance for medical imaging and segmentation tasks.

VGG1D Model

class lis2.models.VGG1D(*args, **kwargs)[source]

Bases: BaseModel

VGG1D model for processing one-dimensional data.

This class implements a 1D version of the VGG architecture, which is designed for processing one-dimensional data. It includes multiple convolutional layers with batch normalization and ReLU activations, followed by fully connected layers.

Configuration: name (str): The name of the VGG1D model. in_channels (int): Number of input channels. Default is 1. out_channels (int): Number of output channels. Default is 1. n_convs (list of int): Number of convolutions in each layer. Default is [2, 2, 3, 3, 3]. fc_units (list of int): Number of units in the fully connected layers. Default is [4096, 4096]. kernel_size (int): Size of the convolutional kernel. Default is 3. stride (int): Stride for the convolution. Default is 1. padding (int): Padding for the convolution. Default is 1. dropout_rate (float): Dropout rate for the dropout layers. Default is 0.5. activation (str): Activation function to use. Default is ‘relu’.

Example Configuration (Python):
config = {
    "name": "example_vgg1d",
    "in_channels": 1,
    "out_channels": 1,
    "n_convs": [2, 2, 3, 3, 3],
    "fc_units": [4096, 4096],
    "kernel_size": 3,
    "stride": 1,
    "padding": 1,
    "dropout_rate": 0.5,
    "activation": "relu"
}

Aliases:

vgg1d

config_schema: Dict = {'activation': Schema(type=<class 'str'>, aliases=[], optional=True, default=relu), 'dropout_rate': Schema(type=<class 'float'>, aliases=[], optional=True, default=0.5), 'fc_units': Schema(type=<class 'list'>, aliases=[], optional=True, default=[4096, 4096]), 'in_channels': Schema(type=<class 'int'>, aliases=[], optional=True, default=1), 'kernel_size': Schema(type=<class 'int'>, aliases=[], optional=True, default=3), 'n_convs': Schema(type=<class 'list'>, aliases=[], optional=True, default=[2, 2, 3, 3, 3]), 'out_channels': Schema(type=<class 'int'>, aliases=[], optional=True, default=1), 'padding': Schema(type=<class 'int'>, aliases=[], optional=True, default=1), 'stride': Schema(type=<class 'int'>, aliases=[], optional=True, default=1)}
core_forward(x)[source]

Core forward pass of the VGG1D model.

Parameters:

x (torch.Tensor) – Input tensor.

Returns:

The output tensor after the forward pass.

Return type:

torch.Tensor

preprocess_forward(patch, *args, **kwargs)[source]

Preprocess the input data before the core forward pass.

Parameters:
  • patch (torch.Tensor) – Input tensor.

  • *args – Additional positional arguments.

  • **kwargs – Additional keyword arguments.

Returns:

The preprocessed input tensor.

Return type:

torch.Tensor

The VGG1D model is a 1D adaptation of the VGG architecture, commonly used for time-series classification.

VNet Model

class lis2.models.VNet(*args, **kwargs)[source]

Bases: BaseModel

VNet model for 2d image segmentation.

This class implements the VNet architecture, which is designed for 2d image segmentation tasks. It includes convolutional layers, batch normalization, and residual connections to improve performance.

Configuration: name (str): The name of the VNet model. elu (bool): Whether to use ELU activation function. Default is True. nll (bool): Whether to use negative log-likelihood loss. Default is False. dropout_rate (float): Dropout rate for the dropout layers. Default is 0.5. activation (str): Activation function to use. Default is ‘elu’. n_convs (list of int): Number of convolutions in each layer. Default is [1, 2, 3, 2, 2].

Example Configuration (Python):
config = {
    "name": "example_vnet",
    "elu": True,
    "nll": False,
    "dropout_rate": 0.5,
    "activation": "elu",
    "n_convs": [1, 2, 3, 2, 2]
}

Aliases:

vnet

config_schema: Dict = {'activation': Schema(type=<class 'str'>, aliases=[], optional=True, default=elu), 'dropout_rate': Schema(type=<class 'float'>, aliases=[], optional=True, default=0.5), 'elu': Schema(type=<class 'bool'>, aliases=[], optional=True, default=True), 'n_convs': Schema(type=<class 'list'>, aliases=[], optional=True, default=[1, 2, 3, 2, 2]), 'nll': Schema(type=<class 'bool'>, aliases=[], optional=True, default=False)}
core_forward(x)[source]

Core forward pass of the VNet model.

Parameters:

x (torch.Tensor) – Input tensor.

Returns:

The output tensor after the forward pass.

Return type:

torch.Tensor

postprocess_forward(x: Any) → Tensor[source]

Postprocess the output data after the core forward pass.

Parameters:

x (Any) – The output tensor after the forward pass.

Returns:

The postprocessed output tensor.

Return type:

torch.Tensor

preprocess_forward(patch, *args, **kwargs)[source]

Preprocess the input data before the core forward pass.

Parameters:
  • patch (torch.Tensor) – Input tensor.

  • *args – Additional positional arguments.

  • **kwargs – Additional keyword arguments.

Returns:

The preprocessed input tensor.

Return type:

torch.Tensor

The VNet model is a 3D segmentation network designed for medical imaging tasks, particularly volumetric segmentation.

Notes

  • All models inherit from BaseModel and follow a modular design for easy extensibility.

Position Encoders

Encoders are responsible for encoding positional information before passing it to a model. They transform raw position data into meaningful representations that can be used in deep learning architectures.

Available Encoders for position

Several built-in encoders are provided:

  • encoder.BaseEncoder - Base class for all encoders.

  • encoder.PositionEncoding - Handles multiple variables for position encoding.

  • encoder.SinPositionEncoding - Uses sine-based encoding.

  • encoder.LinPositionEncoding - Uses linear scaling.

  • encoder.IdentityPositionEncoding - Directly passes input positions without transformation.

Configuration

Each encoder requires a configuration dictionary that defines how the positional encoding should behave.

Example Configuration

name: "sin_encoder"
type: "SinPositionEncoding"
vars_config:
  - index: 0
    expand_dims: 2
    scale: 1.0
    offset: 0.0
    unsqueeze: True
    angle: 30.0
  • index: Specifies which dimension of the input data should be encoded.

  • expand_dims: Defines how the encoding should be expanded spatially.

  • scale: Scaling factor applied before encoding.

  • offset: Offset added to the input before applying transformations.

  • unsqueeze: If True, expands dimensions to match the model’s expected input.

  • angle (optional): Used in encoder.SinPositionEncoding to adjust frequency.

Using an Encoder

Encoders can be instantiated and used as follows:

import torch
from encoder import SinPositionEncoding

positions = torch.randn(10, 5, 1)  # Batch of 10, 5 positional indices
encoder_config = {
    "type": "SinPositionEncoding",
    "vars_config": [
        {"index": 0, "expand_dims": 2, "scale": 1.0, "offset": 0.0, "unsqueeze": True, "angle": 30.0}
    ]
}

encoder = BaseEncoder.from_config(encoder_config)
encoded_positions = encoder(positions)
print(encoded_positions.shape)  # Expected shape: (10, X, X), where X depends on `expand_dims`

Creating a Custom Encoder

You can create a custom encoder by subclassing encoder.PositionEncoding and implementing forward:

from encoder import PositionEncoding
import torch

class CustomPositionEncoding(PositionEncoding):
    def forward(self, positions):
        encoded = []
        for v in self.vars:
            pe = torch.log1p(positions[:, v.index, 0] * v.scale + v.offset)
            if v.unsqueeze:
                pe = torch.unsqueeze(
                    torch.unsqueeze(pe, dim=2).expand(pe.shape[0], v.expand_dims, v.expand_dims), dim=1
                )
            encoded.append(pe)
        return torch.cat(encoded, dim=1)

# Usage example
custom_encoder_config = {
    "type": "CustomPositionEncoding",
    "vars_config": [
        {"index": 0, "expand_dims": 2, "scale": 2.0, "offset": 1.0, "unsqueeze": True}
    ]
}

custom_encoder = BaseEncoder.from_config(custom_encoder_config)
test_positions = torch.randn(10, 5, 1)
output = custom_encoder(test_positions)
print(output.shape)

Encoder Position Zoo

BaseEncoder

class lis2.models.BaseEncoder(*args, **kwargs)[source]

Bases: TypedConfigurable, ABC

Encoder base class for encoding mechanisms.

This abstract class serves as the foundation for different position encoding implementations.

abstractmethod forward(positions)[source]

Encodes the given positions.

VariableEncoding

class lis2.models.VariableEncoding(*args, **kwargs)[source]

Bases: Configurable

VariableEncoding for defining position encoding parameters.

Configuration:
  • index (int): Index of the variable.

  • expand_dims (int): Number of dimensions to expand.

  • scale (float): Scaling factor for the variable.

  • offset (float): Offset value. Default is 0.0.

  • unsqueeze (bool): Whether to unsqueeze the dimensions.

  • angle (float, optional): Angle parameter.

Example Configuration (YAML):
index: 0
expand_dims: 2
scale: 1.0
offset: 0.0
unsqueeze: True
angle: 30.0
config_schema: Dict = {'angle': Schema(type=<class 'float'>, aliases=[], optional=True, default=None), 'expand_dims': Schema(type=<class 'int'>, aliases=[], optional=False, default=None), 'index': Schema(type=<class 'int'>, aliases=[], optional=False, default=None), 'offset': Schema(type=<class 'float'>, aliases=[], optional=True, default=0.0), 'scale': Schema(type=<class 'float'>, aliases=[], optional=False, default=None), 'unsqueeze': Schema(type=<class 'bool'>, aliases=[], optional=False, default=None)}

PositionEncoding

class lis2.models.PositionEncoding(*args, **kwargs)[source]

Bases: BaseEncoder

PositionEncoding for encoding positional information.

This is a base class for encoding positions using different transformation functions. It processes input positions using a set of VariableEncoding configurations, which define how each variable contributes to the encoding.

Configuration:
  • name (str): The name of the encoding instance.

  • vars_config (list): A list of VariableEncoding configurations defining the encoding parameters.

Example Configuration (YAML):
name: "position_encoding"
vars_config:
  - index: 0
    expand_dims: 2
    scale: 1.0
    offset: 0.0
    unsqueeze: True
    angle: 30.0
Aliases:

Encoding

aliases = ['Encoding']
config_schema: Dict = {'vars_config': Schema(type=<class 'list'>, aliases=[], optional=False, default=None)}

SinPositionEncoding

class lis2.models.SinPositionEncoding(*args, **kwargs)[source]

Bases: PositionEncoding

SinPositionEncoding for sine-based position encoding.

Configuration:

Inherits from PositionEncoding.

Aliases:

SinEncoding, Sin

aliases = ['SinEncoding', 'Sin']
forward(positions)[source]

Encodes the given positions.

LinPositionEncoding

class lis2.models.LinPositionEncoding(*args, **kwargs)[source]

Bases: PositionEncoding

LinPositionEncoding for linear position encoding.

Configuration:

Inherits from PositionEncoding.

Aliases:

LinEncoding, Lin

aliases = ['LinEncoding', 'Lin']
forward(positions)[source]

Encodes the given positions.

IdentityPositionEncoding

class lis2.models.IdentityPositionEncoding(*args, **kwargs)[source]

Bases: BaseEncoder

IdentityPositionEncoding for identity transformation of positional information.

Configuration:

Inherits from Encoder.

Aliases:

IdentityEncoding, Identity

aliases = ['IdentityEncoding', 'Identity']
forward(positions)[source]

Encodes the given positions.