Normalising Flow Networks

Classes

class NormFlow(save_num, states_dir, net, *, overwrite=False, mix_precision=False, save_freq=1, learning_rate=0.001, description='', verbose='epoch', transform=None, in_transform=None, optimiser_kwargs=None, scheduler_kwargs=None)

Bases: BaseArchitecture

Transforms a simple distribution into a distribution that reflects the input data

Requires last layer to be a normalising flow and will not pass input data through the network.

Attributes:

netBaseNetwork

Neural spline flow

descriptionstr

Description of the architecture

lossestuple[list[LossCT], list[LossCT]]

Architecture training and validation losses as a float or dictionary of losses for each loss function

transformsdict[str, list[BaseTransform] | BaseTransform | NoneType]

Keys for the output data from predict and corresponding transforms

idxs: ndarray | None

Training data indices with shape (N) and type int, where N is the number of elements in the training dataset

optimiserOptimizer

Architecture optimiser

schedulerLRScheduler

Optimiser scheduler

Parameters:

save_numint | str

File number or name to save the architecture

states_dirstr

Directory to save the architecture

netModule | BaseNetwork

Network to predict low-dimensional data

overwritebool, Optional

If saving can overwrite an existing save file, if False and file with the same name exists, an error will be raised, default = False

mix_precisionbool, Optional

If mixed precision should be used, default = False

save_freqint, Optional

Frequency of epochs to save the architecture, default = 1

learning_ratefloat | tuple[float, …], Optional

Optimiser initial learning rate, default = 1e-3

descriptionstr, Optional

Description of the architecture

verbose{‘epoch’, ‘full’, ‘plot’, ‘progress’, NoneType}

If details about each epoch should be printed (‘epoch’), details about epoch and epoch progress (full), details about epoch and an ASCII plot of the loss progress (‘plot’), just total progress (‘progress’), or nothing (None)

transformlist[BaseTransform] | BaseTransform | NoneType, Optional

Transformation(s) of the network’s output(s)

in_transformlist[BaseTransform] | BaseTransform | NoneType, Optional

Transformation(s) for the network’s input(s)

optimiser_kwargsdict[str, Any] | NoneType, Optional

Optional keyword arguments to pass to init_optimiser

scheduler_kwargsdict[str, Any] | NoneType, Optional

Optional keyword arguments to pass to init_scheduler

predict(loader=None, *, path='', num=None, **__)

Generates probability distributions for a dataset and can save to a file.

Parameters:

loaderDataLoader[Any] | NoneType, Optional

Unused, present for compatibility with BaseArchitecture

pathstr, Optional

Path as pkl file to save the predictions if they should be saved

numlist[int] | NoneType, Optional

Number of samples, S, to generate, default = [1e3]

Returns:

dict[str, NDArrayLike]

Predicted distribution, with shape (N,…,S) and type float for dataset of size N

class NormFlowEncoder(save_num, states_dir, net, *, overwrite=False, mix_precision=False, net_checkpoint=None, description='', verbose='epoch', train_epochs=(0, -1), learning_rate=(0.001, 0.001), classes=None, loss_func=None, transform=None, in_transform=None, optimiser_kwargs=None, scheduler_kwargs=None)

Bases: BaseEncoder

Calculates the loss for a network and normalising flow that takes high-dimensional data and predicts a low-dimensional data distribution.

Requires the normalising flow to be the last layer in the network.

Attributes:

descriptionstr

Description of the architecture

lossestuple[list[LossCT], list[LossCT]]

Architecture training and validation losses as a float or dictionary of losses for each loss function

transformsdict[str, list[BaseTransform] | BaseTransform | NoneType]

Keys for the output data from predict and corresponding transforms

idxs: ndarray | None

Training data indices with shape (N) and type int, where N is the number of elements in the training dataset

classesTensor | NoneType

Unique classes of shape (C) and type int/float, where C is the number of classes

optimiserOptimizer

Architecture optimiser

schedulerLRScheduler

Optimiser scheduler

netBaseNetwork

Neural network

Parameters:

save_numint | str

File number or name to save the flow

states_dirstr

Directory to save the architecture and flow

netModule | BaseNetwork[ModuleList]

Normalising flow to predict low-dimensional data distribution

overwritebool, Optional

If saving can overwrite an existing save file, if True and file with the same name exists, an error will be raised, default = False

mix_precision: bool, Optional

If mixed precision should be used, default = False

net_checkpointint, Optional

Network checkpoint for calculating the encoder’s loss, if none, will use output from the network if output is a Tensor, else no encoder loss will be used

descriptionstr, Optional

Description of the architecture training

verbose{‘full’, ‘progress’, NoneType}

If details about epoch should be printed (‘full’), just a progress bar (‘progress’), or nothing (None)

train_epochstuple[int, int], Optional

Epoch when to start training the normalising flow and epoch when to stop training the encoder, if both are zero, then flow will be trained from the beginning and encoder will not be trained, if both are -1, then flow will never be trained and encoder will always be trained, default = (0,-1)

learning_ratetuple[float, float], Optional

Optimiser initial learning rate for encoder and normalising flow, if None, no optimiser or scheduler will be set, default = (1e-3,1e-3)

classesTensor, Optional

Unique classes of shape (C) and type int/float, where C is the number of classes

loss_funcBaseLoss | NoneType, Optional

Loss function for the encoder, if None MSELoss will be used if classes is None, else CrossEntropyLoss will be used

in_transformBaseTransform, Optional

Transformation for the input data

optimiser_kwargsdict[str, Any] | NoneType, Optional

Optional keyword arguments to pass to init_optimiser

scheduler_kwargsdict[str, Any] | NoneType, Optional

Optional keyword arguments to pass to init_scheduler

batch_predict(data, *, num=None, **_)

Generates probability distributions for the data batch

Parameters:

dataTensorListLike

Data of shape (N,…) and type float to generate distributions for, where N is the batch size

numlist[int] | NoneType, Optional

Number of samples, S, to generate, default = [1e3]

Returns:

tuple[NDArrayListLike | NoneType, ndarray | NoneType]

Architecture output with shape (N,…) and type float and samples of shape (N,S) and type float from each probability distribution

extra_repr()

Additional representation of the architecture.

Returns:

str

Architecture specific representation

get_hyperparams()

Get the hyperparameters of the normalising flow encoder.

Returns:

dict[str, Any]

Hyperparameters of the normalising flow encoder

get_param_groups(learning_rate)

Gets the parameter groups for the optimiser.

Parameters:

learning_ratefloat | tuple[float, …] | NoneType

Learning rate or learning rates for the parameter groups

Returns:

ParamsT

Parameter groups for the optimiser

predict(loader, *_, inputs=False, bin_num=100, path='', num=None, **__)

Generates probability distributions for a dataset and can save to a file

Parameters:

loaderDataLoader

Dataset to generate predictions for

inputsbool, Optional

If the input data should be returned and saved, default = False,

bin_numint, Optional

Number of bins for calculating the probability of the target and maximum of the distribution, higher is more precise but requires more samples, default = 100

pathstr, Optional

Path as a pkl file to save the predictions if they should be saved

numlist[int] | NoneType, Optional

Number of samples, S, to generate from the predicted distribution, default = [1e3]

Returns:

dict[str, NDArrayLike]

Prediction IDs of shape (N), Optional inputs, target values, target probability, distribution maximum, distribution median of shape (N,…), and predicted distribution of shape (N,…,S) and type float for dataset of size N