NN Layers¶
Residual Blocks¶
- class dpipe.layers.resblock.ResBlock(*args: Any, **kwargs: Any)[source]¶
Bases:
Module
Performs a sequence of two convolutions with residual connection (Residual Block).
- Parameters
in_channels (int) – the number of incoming channels.
out_channels (int) – the number of the
ResBlock
output channels. Note, ifin_channels
!=out_channels
, then linear transform will be applied to the shortcut.kernel_size (int, tuple) – size of the convolving kernel.
stride (int, tuple, optional) – stride of the convolution. Default is 1. Note, if stride is greater than 1, then linear transform will be applied to the shortcut.
padding (int, tuple, optional) – zero-padding added to all spatial sides of the input. Default is 0.
dilation (int, tuple, optional) – spacing between kernel elements. Default is 1.
bias (bool) – if
True
, adds a learnable bias to the output. Default isFalse
.activation_module (None, nn.Module, optional) – module to build up activation layer. Default is
torch.nn.ReLU
.conv_module (nn.Module) – module to build up convolution layer with given parameters, e.g.
torch.nn.Conv3d
.batch_norm_module (nn.Module) – module to build up batch normalization layer, e.g.
torch.nn.BatchNorm3d
.kwargs – additional arguments passed to
conv_module
.
FPN¶
- class dpipe.layers.fpn.FPN(*args: Any, **kwargs: Any)[source]¶
Bases:
Module
Feature Pyramid Network - a generalization of UNet.
- Parameters
layer (Callable) – the structural block of each level, e.g.
torch.nn.Conv2d
.downsample (nn.Module) – the downsampling layer, e.g.
torch.nn.MaxPool2d
.upsample (nn.Module) – the upsampling layer, e.g.
torch.nn.Upsample
.merge (Callable(left, down)) – a function that merges the upsampled features map with the one coming from the left branch, e.g.
torch.add
.structure (Sequence[Union[Sequence[int], nn.Module]]) – a collection of channels sequences, see Examples section for details.
last_level (bool) – If True only the result of the last level is returned (as in UNet), otherwise the results from all levels are returned (as in FPN).
kwargs – additional arguments passed to
layer
.
Examples
>>> from dpipe.layers import ResBlock2d >>> >>> structure = [ >>> [[16, 16, 16], [16, 16, 16]], # level 1, left and right >>> [[16, 32, 32], [32, 32, 16]], # level 2, left and right >>> [32, 64, 32] # final level >>> ] >>> >>> upsample = nn.Upsample(scale_factor=2, mode='bilinear') >>> downsample = nn.MaxPool2d(kernel_size=2) >>> >>> ResUNet = FPN( >>> ResBlock2d, downsample, upsample, torch.add, >>> structure, kernel_size=3, dilation=1, padding=1, last_level=True >>> )
References
Structure¶
- dpipe.layers.structure.make_consistent_seq(layer: Callable, channels: Sequence[int], *args, **kwargs)[source]¶
Builds a sequence of layers that have consistent input and output channels/features.
args
andkwargs
are passed as additional parameters.Examples
>>> make_consistent_seq(nn.Conv2d, [16, 32, 64, 128], kernel_size=3, padding=1) >>> # same as >>> nn.Sequential( >>> nn.Conv2d(16, 32, kernel_size=3, padding=1), >>> nn.Conv2d(32, 64, kernel_size=3, padding=1), >>> nn.Conv2d(64, 128, kernel_size=3, padding=1), >>> )
- class dpipe.layers.structure.ConsistentSequential(*args: Any, **kwargs: Any)[source]¶
Bases:
Sequential
A sequence of layers that have consistent input and output channels/features.
args
andkwargs
are passed as additional parameters.Examples
>>> ConsistentSequential(nn.Conv2d, [16, 32, 64, 128], kernel_size=3, padding=1) >>> # same as >>> nn.Sequential( >>> nn.Conv2d(16, 32, kernel_size=3, padding=1), >>> nn.Conv2d(32, 64, kernel_size=3, padding=1), >>> nn.Conv2d(64, 128, kernel_size=3, padding=1), >>> )
- class dpipe.layers.structure.PreActivation(*args: Any, **kwargs: Any)[source]¶
Bases:
Module
Runs a sequence of batch_norm, activation, and
layer
.in -> (BN -> activation -> layer) -> out
- Parameters
in_features (int) – the number of incoming features/channels.
out_features (int) – the number of the output features/channels.
batch_norm_module – module to build up batch normalization layer, e.g.
torch.nn.BatchNorm3d
.activation_module – module to build up activation layer. Default is
torch.nn.ReLU
.layer_module (Callable(in_features, out_features, **kwargs)) – module to build up the main layer, e.g.
torch.nn.Conv3d
ortorch.nn.Linear
.kwargs – additional arguments passed to
layer_module
.
- class dpipe.layers.structure.PostActivation(*args: Any, **kwargs: Any)[source]¶
Bases:
Module
Performs a sequence of layer, batch_norm and activation:
in -> (layer -> BN -> activation) -> out
- Parameters
in_features (int) – the number of incoming features/channels.
out_features (int) – the number of the output features/channels.
batch_norm_module – module to build up batch normalization layer, e.g.
torch.nn.BatchNorm3d
.activation_module – module to build up activation layer. Default is
torch.nn.ReLU
.layer_module (Callable(in_features, out_features, **kwargs)) – module to build up the main layer, e.g.
torch.nn.Conv3d
ortorch.nn.Linear
.kwargs – additional arguments passed to
layer_module
.
Notes
If
layer
supports a bias term, make sure to passbias=False
.
- class dpipe.layers.structure.Lambda(*args: Any, **kwargs: Any)[source]¶
Bases:
Module
Applies
func
to the incoming tensor.kwargs
are passed as additional arguments.
- class dpipe.layers.conv.PreActivationND(*args: Any, **kwargs: Any)[source]¶
Bases:
PreActivation
Performs a sequence of batch_norm, activation, and convolution
in -> (BN -> activation -> Conv) -> out
- Parameters
in_channels (int) – the number of incoming channels.
out_channels (int) – the number of the
PreActivation
output channels.kernel_size (int, tuple) – size of the convolving kernel.
stride (int, tuple, optional) – stride of the convolution. Default is 1.
padding (int, tuple, optional) – zero-padding added to all spatial sides of the input. Default is 0.
dilation (int, tuple, optional) – spacing between kernel elements. Default is 1.
groups (int, optional) – number of blocked connections from input channels to output channels. Default is 1.
bias (bool) – if
True
, adds a learnable bias to the output. Default isFalse
batch_norm_module (nn.Module) – module to build up batch normalization layer, e.g.
torch.nn.BatchNorm3d
.activation_module (nn.Module) – module to build up activation layer. Default is
torch.nn.ReLU
.conv_module (nn.Module) – module to build up convolution layer with given parameters, e.g.
torch.nn.Conv3d
.kwargs – additional arguments passed to
layer_module
- class dpipe.layers.conv.PostActivationND(*args: Any, **kwargs: Any)[source]¶
Bases:
PostActivation
Performs a sequence of convolution, batch_norm and activation:
in -> (Conv -> BN -> activation) -> out
- Parameters
in_channels (int) – the number of incoming channels.
out_channels (int) – the number of the
PostActivation
output channels.kernel_size (int, tuple) – size of the convolving kernel.
stride (int, tuple, optional) – stride of the convolution. Default is 1.
padding (int, tuple, optional) – zero-padding added to all spatial sides of the input. Default is 0.
dilation (int, tuple, optional) – spacing between kernel elements. Default is 1.
groups (int, optional) – number of blocked connections from input channels to output channels. Default is 1.
batch_norm_module (nn.Module) – module to build up batch normalization layer, e.g.
torch.nn.BatchNorm3d
.activation_module (nn.Module) – module to build up activation layer. Default is
torch.nn.ReLU
.conv_module (nn.Module) – module to build up convolution layer with given parameters, e.g.
torch.nn.Conv3d
.kwargs – additional arguments passed to
layer_module
Shape Operations¶
- class dpipe.layers.shape.InterpolateToInput(*args: Any, **kwargs: Any)[source]¶
Bases:
Module
Interpolates the result of
path
to the original shape along the spatialaxis
.- Parameters
path (nn.Module) – arbitrary neural network module to calculate the result.
mode (str) – algorithm used for upsampling. Should be one of ‘nearest’ | ‘linear’ | ‘bilinear’ | ‘trilinear’ | ‘area’. Default is ‘nearest’.
axis (AxesLike, None, optional) – spatial axes to interpolate result along. If
axes
isNone
, the result is interpolated along all the spatial axes.
- class dpipe.layers.shape.Reshape(*args: Any, **kwargs: Any)[source]¶
Bases:
Module
Reshape the incoming tensor to the given
shape
.- Parameters
shape (Union[int, str]) – the resulting shape. String values denote indices in the input tensor’s shape.
Examples
>>> layer = Reshape('0', '1', 500, 500) >>> layer(x) >>> # same as >>> x.reshape(x.shape[0], x.shape[1], 500, 500)
- class dpipe.layers.shape.Softmax(*args: Any, **kwargs: Any)[source]¶
Bases:
Module
A multidimensional version of softmax.
- class dpipe.layers.shape.PyramidPooling(*args: Any, **kwargs: Any)[source]¶
Bases:
Module
Implements the pyramid pooling operation.
- Parameters
pooling (Callable) – the pooling to be applied, e.g.
torch.nn.functional.max_pool2d
.levels (int) – the number of pyramid levels, default is 1 which is the global pooling operation.