NN Layers

Residual Blocks

class dpipe.layers.resblock.ResBlock(*args: Any, **kwargs: Any)[source]

Bases: Module

Performs a sequence of two convolutions with residual connection (Residual Block).

Parameters

  • in_channels (int) – the number of incoming channels.

  • out_channels (int) – the number of the ResBlock output channels. Note, if in_channels != out_channels, then linear transform will be applied to the shortcut.

  • kernel_size (int, tuple) – size of the convolving kernel.

  • stride (int, tuple, optional) – stride of the convolution. Default is 1. Note, if stride is greater than 1, then linear transform will be applied to the shortcut.

  • padding (int, tuple, optional) – zero-padding added to all spatial sides of the input. Default is 0.

  • dilation (int, tuple, optional) – spacing between kernel elements. Default is 1.

  • bias (bool) – if True, adds a learnable bias to the output. Default is False.

  • activation_module (None, nn.Module, optional) – module to build up activation layer. Default is torch.nn.ReLU.

  • conv_module (nn.Module) – module to build up convolution layer with given parameters, e.g. torch.nn.Conv3d.

  • batch_norm_module (nn.Module) – module to build up batch normalization layer, e.g. torch.nn.BatchNorm3d.

  • kwargs – additional arguments passed to conv_module.

FPN

class dpipe.layers.fpn.FPN(*args: Any, **kwargs: Any)[source]

Bases: Module

Feature Pyramid Network - a generalization of UNet.

Parameters

  • layer (Callable) – the structural block of each level, e.g. torch.nn.Conv2d.

  • downsample (nn.Module) – the downsampling layer, e.g. torch.nn.MaxPool2d.

  • upsample (nn.Module) – the upsampling layer, e.g. torch.nn.Upsample.

  • merge (Callable(left, down)) – a function that merges the upsampled features map with the one coming from the left branch, e.g. torch.add.

  • structure (Sequence[Union[Sequence[int], nn.Module]]) – a collection of channels sequences, see Examples section for details.

  • last_level (bool) – If True only the result of the last level is returned (as in UNet), otherwise the results from all levels are returned (as in FPN).

  • kwargs – additional arguments passed to layer.

Examples

>>> from dpipe.layers import ResBlock2d
>>>
>>> structure = [
>>>     [[16, 16, 16],       [16, 16, 16]],  # level 1, left and right
>>>     [[16, 32, 32],       [32, 32, 16]],  # level 2, left and right
>>>                [32, 64, 32]              # final level
>>> ]
>>>
>>> upsample = nn.Upsample(scale_factor=2, mode='bilinear')
>>> downsample = nn.MaxPool2d(kernel_size=2)
>>>
>>> ResUNet = FPN(
>>>     ResBlock2d, downsample, upsample, torch.add,
>>>     structure, kernel_size=3, dilation=1, padding=1, last_level=True
>>> )

References

make_consistent_seq FPN UNet

Structure

dpipe.layers.structure.make_consistent_seq(layer: Callable, channels: Sequence[int], *args, **kwargs)[source]

Builds a sequence of layers that have consistent input and output channels/features.

args and kwargs are passed as additional parameters.

Examples

>>> make_consistent_seq(nn.Conv2d, [16, 32, 64, 128], kernel_size=3, padding=1)
>>> # same as
>>> nn.Sequential(
>>>     nn.Conv2d(16, 32, kernel_size=3, padding=1),
>>>     nn.Conv2d(32, 64, kernel_size=3, padding=1),
>>>     nn.Conv2d(64, 128, kernel_size=3, padding=1),
>>> )
class dpipe.layers.structure.ConsistentSequential(*args: Any, **kwargs: Any)[source]

Bases: Sequential

A sequence of layers that have consistent input and output channels/features.

args and kwargs are passed as additional parameters.

Examples

>>> ConsistentSequential(nn.Conv2d, [16, 32, 64, 128], kernel_size=3, padding=1)
>>> # same as
>>> nn.Sequential(
>>>     nn.Conv2d(16, 32, kernel_size=3, padding=1),
>>>     nn.Conv2d(32, 64, kernel_size=3, padding=1),
>>>     nn.Conv2d(64, 128, kernel_size=3, padding=1),
>>> )
class dpipe.layers.structure.PreActivation(*args: Any, **kwargs: Any)[source]

Bases: Module

Runs a sequence of batch_norm, activation, and layer.

in -> (BN -> activation -> layer) -> out

Parameters

  • in_features (int) – the number of incoming features/channels.

  • out_features (int) – the number of the output features/channels.

  • batch_norm_module – module to build up batch normalization layer, e.g. torch.nn.BatchNorm3d.

  • activation_module – module to build up activation layer. Default is torch.nn.ReLU.

  • layer_module (Callable(in_features, out_features, **kwargs)) – module to build up the main layer, e.g. torch.nn.Conv3d or torch.nn.Linear.

  • kwargs – additional arguments passed to layer_module.

class dpipe.layers.structure.PostActivation(*args: Any, **kwargs: Any)[source]

Bases: Module

Performs a sequence of layer, batch_norm and activation:

in -> (layer -> BN -> activation) -> out

Parameters

  • in_features (int) – the number of incoming features/channels.

  • out_features (int) – the number of the output features/channels.

  • batch_norm_module – module to build up batch normalization layer, e.g. torch.nn.BatchNorm3d.

  • activation_module – module to build up activation layer. Default is torch.nn.ReLU.

  • layer_module (Callable(in_features, out_features, **kwargs)) – module to build up the main layer, e.g. torch.nn.Conv3d or torch.nn.Linear.

  • kwargs – additional arguments passed to layer_module.

Notes

If layer supports a bias term, make sure to pass bias=False.

class dpipe.layers.structure.Lambda(*args: Any, **kwargs: Any)[source]

Bases: Module

Applies func to the incoming tensor.

kwargs are passed as additional arguments.

class dpipe.layers.conv.PreActivationND(*args: Any, **kwargs: Any)[source]

Bases: PreActivation

Performs a sequence of batch_norm, activation, and convolution

in -> (BN -> activation -> Conv) -> out

Parameters

  • in_channels (int) – the number of incoming channels.

  • out_channels (int) – the number of the PreActivation output channels.

  • kernel_size (int, tuple) – size of the convolving kernel.

  • stride (int, tuple, optional) – stride of the convolution. Default is 1.

  • padding (int, tuple, optional) – zero-padding added to all spatial sides of the input. Default is 0.

  • dilation (int, tuple, optional) – spacing between kernel elements. Default is 1.

  • groups (int, optional) – number of blocked connections from input channels to output channels. Default is 1.

  • bias (bool) – if True, adds a learnable bias to the output. Default is False

  • batch_norm_module (nn.Module) – module to build up batch normalization layer, e.g. torch.nn.BatchNorm3d.

  • activation_module (nn.Module) – module to build up activation layer. Default is torch.nn.ReLU.

  • conv_module (nn.Module) – module to build up convolution layer with given parameters, e.g. torch.nn.Conv3d.

  • kwargs – additional arguments passed to layer_module

class dpipe.layers.conv.PostActivationND(*args: Any, **kwargs: Any)[source]

Bases: PostActivation

Performs a sequence of convolution, batch_norm and activation:

in -> (Conv -> BN -> activation) -> out

Parameters

  • in_channels (int) – the number of incoming channels.

  • out_channels (int) – the number of the PostActivation output channels.

  • kernel_size (int, tuple) – size of the convolving kernel.

  • stride (int, tuple, optional) – stride of the convolution. Default is 1.

  • padding (int, tuple, optional) – zero-padding added to all spatial sides of the input. Default is 0.

  • dilation (int, tuple, optional) – spacing between kernel elements. Default is 1.

  • groups (int, optional) – number of blocked connections from input channels to output channels. Default is 1.

  • batch_norm_module (nn.Module) – module to build up batch normalization layer, e.g. torch.nn.BatchNorm3d.

  • activation_module (nn.Module) – module to build up activation layer. Default is torch.nn.ReLU.

  • conv_module (nn.Module) – module to build up convolution layer with given parameters, e.g. torch.nn.Conv3d.

  • kwargs – additional arguments passed to layer_module

Shape Operations

class dpipe.layers.shape.InterpolateToInput(*args: Any, **kwargs: Any)[source]

Bases: Module

Interpolates the result of path to the original shape along the spatial axis.

Parameters

  • path (nn.Module) – arbitrary neural network module to calculate the result.

  • mode (str) – algorithm used for upsampling. Should be one of ‘nearest’ | ‘linear’ | ‘bilinear’ | ‘trilinear’ | ‘area’. Default is ‘nearest’.

  • axis (AxesLike, None, optional) – spatial axes to interpolate result along. If axes is None, the result is interpolated along all the spatial axes.

class dpipe.layers.shape.Reshape(*args: Any, **kwargs: Any)[source]

Bases: Module

Reshape the incoming tensor to the given shape.

Parameters

shape (Union[int, str]) – the resulting shape. String values denote indices in the input tensor’s shape.

Examples

>>> layer = Reshape('0', '1', 500, 500)
>>> layer(x)
>>> # same as
>>> x.reshape(x.shape[0], x.shape[1], 500, 500)
class dpipe.layers.shape.Softmax(*args: Any, **kwargs: Any)[source]

Bases: Module

A multidimensional version of softmax.

class dpipe.layers.shape.PyramidPooling(*args: Any, **kwargs: Any)[source]

Bases: Module

Implements the pyramid pooling operation.

Parameters

  • pooling (Callable) – the pooling to be applied, e.g. torch.nn.functional.max_pool2d.

  • levels (int) – the number of pyramid levels, default is 1 which is the global pooling operation.