Miscellaneous Layers#

Classes#

class BatchNorm(shapes, **kwargs)[source]#

Bases: BaseSingleLayer

Batch Normalisation layer

Attributes:

groupint: Layer group, if 0 it will always be used, else it will only be used if its group matches the Networks
descriptionstr: Description of the layer
layersSequential: Layers to loop through in the forward pass

Parameters:

shapesShapes: Shape of the outputs from each layer
**kwargs: Leftover parameters to pass to base layer for checking

class Checkpoint(shapes, check_shapes, **kwargs)[source]#

Bases: BaseLayer

Constructs a layer to create a checkpoint for using the output from the previous layer in future layers.

Attributes:

groupint: Layer group, if 0 it will always be used, else it will only be used if its group matches the Networks
descriptionstr: Description of the layer

Parameters:

shapesShapes: Shape of the outputs from each layer
check_shapesShapes: Shape of the outputs from each checkpoint
**kwargs: Leftover parameters to pass to base layer for checking

extra_repr()[source]#

Displays layer parameters when printing the network

Returns:

str: Layer parameters

forward(x, checkpoints, *_, **__)[source]#

Forward pass of the checkpoint layer.

Parameters:

xTensorListLike: Input with tensors of shape (N,…) and type float, where N is the batch size
checkpointslist[TensorListLike]: List of checkpoint values with tensors of shape (N,…) and type float

Returns:

TensorListLike: Output with tensors of shape (N,…) and type float

class Dropout(prob, *, shapes=None, **kwargs)[source]#

Bases: BaseSingleLayer

Constructs a dropout layer to randomly zero out elements of the input tensor.

Attributes:

groupint: Layer group, if 0 it will always be used, else it will only be used if its group matches the Networks
descriptionstr: Description of the layer
layersSequential: Layers to loop through in the forward pass

Parameters:

probfloat: Probability of dropout
shapesShapes | NoneType, Optional: Shape of the outputs from each layer, only required if tracking layer outputs is necessary
**kwargs: Leftover parameters to pass to base layer for checking

class DropPath(prob, *, shapes=None, **kwargs)[source]#

Bases: BaseLayer

Constructs a drop path layer to drop samples in a batch.

See FractalNet: Ultra-Deep Neural Networks without Residuals by Larsson et al. (2016) for details.

Custom version of implementation by timm.

Attributes:

groupint: Layer group, if 0 it will always be used, else it will only be used if its group matches the Networks
descriptionstr: Description of the layer

Parameters:

probfloat: Probability of dropout
shapesShapes | NoneType, Optional: Shape of the outputs from each layer, only required if tracking layer outputs is necessary
**kwargs: Leftover parameters to pass to base layer for checking

extra_repr()[source]#

Displays layer parameters when printing the network

Returns:

str: Layer parameters

forward(x, *_, **__)[source]#

Forward pass of the drop path layer.

Parameters:

xTensor: Input tensor of shape (N,…) and type float, where N is the batch size

Returns:

Tensor: Output tensor of shape (N,…) and type float

class Index(*args, map_=True, index=None, shapes=None, **kwargs)[source]#

Bases: BaseLayer

Constructs a layer to slice the last dimension from the output from the previous layer.

Attributes:

groupint: Layer group, if 0 it will always be used, else it will only be used if its group matches the Networks
descriptionstr: Description of the layer

Overloads:

class Index(self, *, index: int | list[int | None] | tuple[int | None, ...] | slice, map_: bool = ..., shapes: Shapes | None = ..., **kwargs: Any) → None

class Index(self, number: int, *, map_: bool = ..., greater: bool = ..., shapes: Shapes | None = ..., **kwargs: Any) → None

Parameters:

indexint | list[int | NoneType] | tuple[int | NoneType, …] | slice: Index or slice to apply to the last dimension of the input tensor(s) or if map_ is False, the DataList itself
map_bool, Optional: If slicing should be applied to each tensor in a DataList, or if the DataList should be sliced itself, only used if input is a DataList, default = True
shapesShapes | NoneType, Optional: Shape of the outputs from each layer, only required if tracking layer outputs is necessary
**kwargs: Leftover parameters to pass to base layer for checking

extra_repr()[source]#

Displays layer parameters when printing the network.

Returns:

str: Layer parameters

forward(x, *_, **__)[source]#

Forward pass of the indexing layer.

Parameters:

xTensor: Input tensor of shape (N,…) and type float, where N is the batch size

Returns:

Tensor: Output tensor of shape (N,…) and type float

class LayerNorm(*, dims=None, shape=None, shapes=None, **kwargs)[source]#

Bases: BaseLayer

Constructs a layer normalisation layer that is the same as PyTorch’s LayerNorm, except normalisation priority is the first dimension after batch dimension to follow ConvNeXt implementation.

Attributes:

groupint: Layer group, if 0 it will always be used, else it will only be used if its group matches the Networks
descriptionstr: Description of the layer

Parameters:

dimsint | NoneType, Optional: Number of dimensions to normalise starting with the first dimension, ignoring batch dimension, requires shapes argument, won’t be used if shape is provided
shapelist[int] | NoneType, Optional: Input shape or shape of the first dimension to normalise, will be used if provided, else dims will be used
shapesShapes | NoneType, Optional: Shape of the outputs from each layer, only required if tracking layer outputs is necessary
**kwargs: Leftover parameters to pass to base layer for checking

forward(x, *_, **__)[source]#

Forward pass of the layer normalisation layer.

Parameters:

xTensor: Input tensor of shape (N,…) and type float, where N is the batch size

Returns:

Tensor: Output Tensor of shape (N,…) and type float

class Pad(pad, *, dims=-1, shapes=None, pad_kwargs=None, **kwargs)[source]#

Bases: BaseLayer

Constructs a padding layer to pad the output from the previous layer.

Attributes:

groupint: Layer group, if 0 it will always be used, else it will only be used if its group matches the Networks
descriptionstr: Description of the layer

Overloads:

class Pad(self, pad: list[int] | list[tuple[int, int]] | tuple[int, ...], *, shapes: Shapes | None = ..., pad_kwargs: dict[str, Any] | None = ..., **kwargs: Any) → None

class Pad(self, pad: int, *, dims: int = ..., shapes: Shapes = ..., pad_kwargs: dict[str, Any] | None = ..., **kwargs: Any) → None

Parameters:

padint | list[int] | list[tuple[int, int]] | tuple[int, …]: Amount of padding to apply to each dimension of the input tensor, ignoring batch dimension and from right to left, if int is provided, same padding will be applied to left and right of all dimensions according to dims, if list of integers is provided, each integer will be applied to left and right of the corresponding dimension, if list of tuple of integers is provided, then asymmetric (left, right) padding will be applied, if tuple of integers is provided, each pair of integers will be applied to left and right of the corresponding dimension
dimsint, Optional: Number of dimensions to pad starting with the last dimension, ignoring batch dimension, if shapes is not provided, dims must be greater than 0, if negative, excludes the first abs(dims) dimensions, if 0 pads all dimensions, won’t be used if padding is a list of integers or list of tuple of integers, default = -1
shapesShapes | NoneType, Optional: Shape of the outputs from each layer, only required if tracking layer outputs is necessary
pad_kwargsdict[str, Any] | NoneType, Optional: Additional keyword arguments to pass to the padding function
**kwargs: Leftover parameters to pass to base layer for checking

extra_repr()[source]#

Displays layer parameters when printing the network.

Returns:

str: Layer parameters

forward(x, *_, **__)[source]#

Forward pass of the padding layer.

Parameters:

xTensorListLike: Input with tensor(s) of shape (N,…) and type float, where N is the batch size

Returns:

TensorListLike: Output with tensor(s) of shape (N,…) and type float

class Reshape(shape, *, factor=False, layer=None, net_out=None, shapes=None, **kwargs)[source]#

Bases: BaseLayer

Constructs a reshaping layer to change the data dimensions.

Attributes:

groupint: Layer group, if 0 it will always be used, else it will only be used if its group matches the Networks
descriptionstr: Description of the layer

Parameters:

shapelist[int]: Desired shape of the output tensor, ignoring first dimension
factorbool, Optional: If reshape should be relative to the network output shape, or if layer is provided, which layer to be relative to, requires tracking layer outputs, default = False
layerint | NoneType, Optional: If factor is True, which layer for factor to be relative to, if None, network output will be used
net_outlist[int] | NoneType, Optional: Shape of the network’s output, required only if factor is True
shapesShapes | NoneType, Optional: Shape of the outputs from each layer, only required if tracking layer outputs is necessary
**kwargs: Leftover parameters to pass to base layer for checking

extra_repr()[source]#

Displays layer parameters when printing the network

Returns:

str: Layer parameters

forward(x, *_, **__)[source]#

Forward pass of reshaping tensors.

Parameters:

xTensor: Input tensor of shape (N,…) and type float, where N is the batch size

Returns:

Tensor: Output tensor of shape (N,…) and type float

class Scale(dims, scale, shapes, *, first=True, **kwargs)[source]#

Bases: BaseLayer

Constructs a scaling layer that scales the output by a learnable tensor.

Attributes:

groupint: Layer group, if 0 it will always be used, else it will only be used if its group matches the Networks
descriptionstr: Description of the layer

Parameters:

dimsint: Number of dimensions to have individual scales for
scalefloat: Initial scale factor
shapesShapes: Shape of the outputs from each layer
firstbool, Optional: If dims should count from the first dimension after the batch dimension, or from the final dimension backwards, default = True
**kwargs: Leftover parameters to pass to base layer for checking

extra_repr()[source]#

Return the extra representation of the module.

To print customized extra information, you should re-implement this method in your own modules. Both single-line and multi-line strings are acceptable.

forward(x, *_, **__)[source]#

Forward pass of the scale layer.

Parameters:

xTensor: Input tensor of shape (N,…) and type float, where N is the batch size

Returns:

Tensor: Output tensor of shape (N,…) and type float

to(*args, **kwargs)[source]#: See torch.nn.Module.to().