RevelioNN Modules

single_mapping_net

class revelionn.mapping_nets.single_mapping_net.SingleMappingNet(in_features, num_neurons_list)[source]

Single Mapping Network for RevelioNN.

It is a fully connected network that receives as input the layer activations reduced to a single dimension or the concatenation of activations of convolutional network layers. It has a ReLU activation function in its hidden layers and a sigmoid in its output. In connection with this there must be 1 neuron in the output layer.

in_features

Input number of neuron activations.

Type:: str

num_neurons_list

The number of neurons in consecutive fully connected layers.

Type:: list[int]

forward(x)[source]: Determines how the data will pass through the neural network.

get_num_neurons_list()[source]: Returns the number of neurons in consecutive fully connected layers.

forward(x)[source]

Determines how the data will pass through the neural network. Returns the data received after processing by the neural network.

Parameters:: x (torch.tensor) – The input activations tensor reduced to one dimension.
Returns:: Output tensor.
Return type:: x

get_in_features()[source]

Returns the input number of neuron activations.

Returns:: in_features – Input number of neuron activations.
Return type:: int

get_num_neurons_list()[source]

Returns the number of neurons in consecutive fully connected layers.

Returns:: num_neurons_list – The number of neurons in consecutive fully connected layers.
Return type:: list[int]

simultaneous_mapping_net

class revelionn.mapping_nets.simultaneous_mapping_net.LayerDecoder(in_channels, out_channels)[source]

Module consisting of a 1x1 convolution layer, followed by a ReLU activation function, a global average pooling layer, and a flattening layer.

Parameters:

in_channels (int) – The number of input channels to the 1x1 convolution layer.
out_channels (int) – The number of output channels from the 1x1 convolution layer.

layers

A sequential container of the layers that make up this module.

Type:: nn.Sequential

forward(x)[source]: Forward pass through the module.

forward(x)[source]

Forward pass through the module.

Parameters:: x (torch.Tensor) – The input tensor of shape (batch_size, in_channels, height, width).
Returns:: The output tensor of shape (batch_size, out_channels).
Return type:: torch.Tensor

class revelionn.mapping_nets.simultaneous_mapping_net.MappingModule(in_features, num_shared_neurons, num_output_neurons, num_outs=1)[source]

A module representing a common fully connected part of a simultaneous mapping network and blocks of concepts.

common_layers

The shared layers.

Type:: nn.Sequential

output_layers_list

A list of output layers, each of which maps the input tensor to an output tensor.

Type:: nn.ModuleList

sigmoid

The sigmoid function used to transform the output tensor(s).

Type:: nn.Sigmoid

generate_layers(num_neurons)[source]: Generates a list of PyTorch layers based on the number of neurons in each layer.

forward(x)[source]: Forward pass through the module.

forward(x)[source]

Forward pass through the module.

Parameters:: x (torch.Tensor) – The input tensor.
Returns:: The output tensor(s).
Return type:: tuple[torch.Tensor]

static generate_layers(num_neurons)[source]

Generates a list of PyTorch layers based on the number of neurons in each layer.

Parameters:: num_neurons (list[int]) – The number of neurons in consecutive fully connected layers.
Returns:: A list of PyTorch layers.
Return type:: list[nn.Module]

class revelionn.mapping_nets.simultaneous_mapping_net.SimultaneousMappingNet(activation_extractor, decoder_channels, num_shared_neurons, num_output_neurons, num_outs)[source]

Simultaneous Mapping Network for RevelioNN.

Receives an input tuple of activations of the specified convolutional network layers, after which the input tensors are processed by decoder blocks. The output tensors of each of the decoders are concatenated and fed into a common fully connected part of the network. This is followed by blocks of concepts (one for each of the concepts), which are sets of fully connected layers having 1 neuron and a sigmoid at the output.

decoder_channels

The number of decoder channels. The output number of channels of the convolutional layer of the decoder or the output number of neurons of the decoder of the fully connected layer.

Type:: int

num_shared_neurons

The number of neurons in consecutive fully connected layers of the common part of the network (internal representation of the simultaneous extraction network).

Type:: list[int]

num_output_neurons

The number of neurons in consecutive fully connected layers of each of the concept blocks.

Type:: list[int]

num_outs

The number of outputs of the simultaneous extraction network. It is determined by the number of extracted concepts.

Type:: int

decoders

Contains the generated decoder blocks in the list.

Type:: torch.nn.ModuleList

forward(x)[source]: Forward pass through the network.

get_decoder_channels()[source]: Returns the number of decoder channels.

get_num_shared_neurons()[source]: Returns the number of neurons in consecutive fully connected layers of the common part of the network.

get_num_output_neurons()[source]: Returns the number of neurons in consecutive fully connected layers of each of the concept blocks.

get_num_outs()[source]: Returns the number of outputs of the simultaneous extraction network.

forward(activations)[source]

Forward pass through the network.

Parameters:: activations (tuple[torch.Tensor]) – A list of input activations.
Returns:: The output tensor.
Return type:: torch.Tensor

get_decoder_channels()[source]

Return the number of decoder channels.

Returns:: The number of decoder channels. The output number of channels of the convolutional layer of the decoder or the output number of neurons of the decoder of the fully connected layer.
Return type:: int

get_num_output_neurons()[source]

Return the number of output neurons.

Returns:: The number of neurons in consecutive fully connected layers of each of the concept blocks.
Return type:: list[int]

get_num_outs()[source]

Return the number of outputs of the simultaneous extraction network.

Returns:: The number of outputs of the simultaneous extraction network. It is determined by the number of extracted concepts.
Return type:: int

get_num_shared_neurons()[source]

Return the number of shared neurons.

Returns:: The number of neurons in consecutive fully connected layers of the common part of the network (internal representation of the simultaneous extraction network).
Return type:: list[int]

main_module

class revelionn.main_module.MainModelProcessing(main_net, device)[source]

Class for training, evaluation and processing the main network model.

device

Tensor processing device.

Type:: torch.device

main_net

The model of the main neural network.

Type:: torch.nn.Module

classes

Names of neural network output classes.

Type:: dict

load_model(path_to_model_dict)[source]: Loads the weights of the neural network model from a file.

train_model(patience, epochs, file_name, class_label_name, module_name,: main_net_class, transformation_name, img_size, num_channels)

Training and validation of the main neural network.

evaluate_model(test_loader)[source]: Evaluation of the model on the test set.

get_main_net()[source]: Returns the main neural network.

get_class_labels()[source]: Returns names of neural network output classes.

get_device()[source]: Returns the current tensor processing device.

evaluate_model(test_loader)[source]

Evaluation of the model on the test set.

Parameters:

test_loader (torch.utils.data.DataLoader) – Training data loader. Combines a dataset and a sampler, and provides an iterable over the given dataset.

Returns:

test_loss (float) – Test loss.
test_acc (float) – Accuracy on the test set.
test_auc (float) – ROC AUC on the test set.

get_class_labels()[source]

Returns names of neural network output classes.

Returns:: classes – Names of neural network output classes.
Return type:: dict

get_device()[source]

Returns the current tensor processing device.

Returns:: device – Tensor processing device.
Return type:: torch.device

get_main_net()[source]

Returns the main neural network.

Returns:: main_net – The main neural network.
Return type:: MainNet(nn.Module)

load_model(path_to_model)[source]

Loads the weights of the neural network model from a file.

Parameters:: path_to_model (str) – The path to the file containing weights.
Return type:: None

train_model(train_loader, valid_loader, patience, epochs, filename, class_label, module_name, main_net_class, transformation_name, img_size, num_channels)[source]

Training and validation of the main neural network.

Parameters:

patience (int) – How many epochs to wait after last time validation loss improved.
epochs (int) – The number of training epochs of the main neural network.
filename (str) – The name of the file in which the parameters of the trained model will be saved.
class_label (str) – The name of the label of the class used for training.
module_name (str) – The name of the file containing the main network class.
main_net_class (str) – Name of the main network class.
transformation_name (str) – Name of the variable storing transformations.
img_size (int) – The size of the image side.
num_channels (int) – The number of image channels.

Return type:

None

activation_extraction

class revelionn.activation_extraction.ActivationExtractor(main_module, layers_types, is_concatenate)[source]

Class for identifying layers of a convolutional neural network and for extracting activations produced during network inference from a selected set of layers.

It has two modes of operation: 1. Activations of the given layers are concatenated and transformed to a one-dimensional tensor. It is used to train a single mapping network. 2. Activations of the specified layers are returned as a tuple without transformations. It is used for training simultaneous mapping network.

device

Tensor processing device.

Type:: torch.device

main_net

The model of the main neural network.

Type:: torch.nn.Module

layers_types_dict

The dictionary contains the names of layer types as keys, and the corresponding values represent the layer class in PyTorch.

Type:: dict

layers_types

Types of layers to be found in the hierarchy of network layers. This list should contain only the names of the layer types that are in ‘layers_types_dict’ dictionary.

Type:: list

layers_dict

Dictionary of neural network layers. The keys of this dictionary represent the unique names of each of the layers of the convolutional network.

Type:: dict

activation

Activation values of the specified layers.

Type:: dict

layers_for_research

A list of the studied convolutional network layers. Set by the user with keys in the ‘layers_dict’ dictionary.

Type:: list

is_concatenate

Logical parameter that sets the mode of operation of ActivationExtractor. If True, the activations of the given layers are concatenated and transformed to a one-dimensional tensor. If False, the activations of the specified layers are returned as a tuple without transformations.

Type:: bool

get_layers_types()[source]: Returns user-defined types of layers to be found in the hierarchy of network layers.

find_layer_predicate_recursive(model, predicate)[source]: Recursively searches through a PyTorch model and returns a list of all layers that satisfy a given predicate.

find_layers_types_recursive(model, layers_types)[source]: Recursively searches through a PyTorch model and returns a list of all layers that are of a given type or types.

create_layers_dict(model, cur_layers_types)[source]: Creates a dictionary of PyTorch layers of the given types from a PyTorch model.

get_layers_dict()[source]: Returns a dictionary of neural network layers.

get_layer_name_by_number(number)[source]: Returns the layer name in the layers dictionary by its number.

get_activation(name)[source]: Saves the values of layer activations.

register_hooks()[source]: Registers the interception of activations of the studied layers.

count_num_activations(num_channels, width_img, height_img)[source]: Returns the number of activations of neurons of the studied layers.

get_activations(mapping_batch_size)[source]: Returns the activation tensor of the studied layers.

set_layers_for_research(layers)[source]: Sets the list of layers to be examined.

get_layers_for_research()[source]: Returns a list of the layers under study.

get_main_net()[source]: Returns the main neural network.

count_num_activations(num_channels, width_img, height_img)[source]

Returns the number of activations of neurons of the studied layers.

Parameters:

num_channels (int) – Number of channels in the input image.
width_img (int) – Width of the input image.
height_img (int) – Height of the input image.

Returns:

num_activations – Number of neuron activations.

Return type:

int

create_layers_dict(model, cur_layers_types)[source]

Creates a dictionary of PyTorch layers of the given types from a PyTorch model.

Parameters:

model (torch.nn.Module) – The PyTorch model to extract layers from.
cur_layers_types (list) – A list of strings representing the types of layers to extract.

Returns:

A dictionary mapping layer names to PyTorch layers of the corresponding type.

Return type:

dict

find_layer_predicate_recursive(model, predicate)[source]

Recursively searches through a PyTorch model and returns a list of all layers that satisfy a given predicate.

Parameters:

model (torch.nn.Module) – The PyTorch model to search through.
predicate (function) – A function that takes a PyTorch layer as input and returns a boolean indicating whether or not the layer satisfies the predicate.

Returns:

A list of all layers in the model that satisfy the given predicate.

Return type:

list

find_layers_types_recursive(model, layers_types)[source]

Recursively searches through a PyTorch model and returns a list of all layers that are of a given type or types.

Parameters:

model (torch.nn.Module) – The PyTorch model to search through.
layers_types (list) – A list of PyTorch layer types to search for.

Returns:

A list of all layers in the model that are of one of the given layer types.

Return type:

list

get_activation(name)[source]

Saves the values of layer activations.

Parameters:: name (str) – User-defined name of the neural network layer.
Return type:: hook

get_activations(mapping_batch_size)[source]

Returns the activation tensor of the studied layers.

Parameters:: mapping_batch_size (int) – The size of the data batch for training the mapping network. The size of the training set must be a multiple of the size of the batch.
Returns:: cur_acts – Activations of the studied layers.
Return type:: torch.Tensor or tuple[torch.Tensor]

get_layer_name_by_number(number)[source]

Returns the layer name in the layers dictionary by its number.

Parameters:: number (int) – Layer number.
Returns:: Layer name.
Return type:: str

get_layers_dict()[source]

Returns a dictionary of neural network layers.

Returns:: layers_dict – A dictionary of neural network layers.
Return type:: dict

get_layers_for_research()[source]

Returns a list of the layers under study.

Returns:: layers_for_research – The list of layers to be examined.
Return type:: list

get_layers_types()[source]

Returns user-defined types of layers to be found in the hierarchy of network layers.

Returns:: Layers types.
Return type:: list[str]

get_main_net()[source]

Returns the main neural network.

Returns:: main_net – The main neural network.
Return type:: torch.nn.Module

register_hooks()[source]: Registers the interception of activations of the studied layers.

set_layers_for_research(layers)[source]

Sets the list of layers to be examined.

Parameters:: layers (list[str] or list[int]) – Contains layer names or layer numbers (indexing from 0) available in the dictionary of layers.

mapping_module

class revelionn.mapping_module.MappingModelProcessing(activation_extractor, mapping_net, device)[source]

Class for training, evaluation and processing the mapping network model.

device

Tensor processing device.

Type:: torch.device

activation_extractor

Class for identifying layers of a convolutional neural network and for extracting activations produced during network inference from a selected set of layers.

Type:: MainNetExplanation

mapping_net

The model of the mapping neural network.

Type:: MappingNet(nn.Module)

class_labels

Names of mapping network output classes.

Type:: dict

train_model_single(train_loader, valid_loader, optimizer, early_stopping, epochs, filename, class_label,

main_net_module_name, main_net_class, main_model_filename, transformation_name, img_size, num_channels): Trains a single mapping network for a given concept.

train_model_simultaneous(train_loader, valid_loader, optimizer, early_stopping, epochs, filename, class_labels,

main_net_module_name, main_net_class, main_model_filename, transformation_name, img_size, num_channels): Trains a simultaneous mapping network for a given set of concepts.

train_model_semisupervised(train_loader, valid_loader, optimizer, early_stopping, epochs, semantic_loss,

sem_loss_weight, filename, class_labels, main_net_module_name, main_net_class, main_model_filename,

transformation_name, img_size, num_channels): Trains a simultaneous mapping network for a given set of concepts using semi-supervised learning, in which a semantic loss is calculated for unlabeled samples, taking into account the relationships between the concepts.

evaluate_model(self, test_loader)[source]: Evaluates the mapping network model on the test set.

get_mapping_net()[source]: Returns the mapping network.

get_class_labels()[source]: Returns names of mapping network output classes.

get_activation_extractor()[source]: Returns the ActivationExtractor object.

load_model(path_to_model_dict)[source]: Loads weights and class labels of the mapping network model from a file.

evaluate_model(test_loader)[source]: Evaluation of the model on the test set.

evaluate_model(test_loader)[source]

Evaluates the mapping network model on the test set.

Parameters:

test_loader (torch.utils.data.DataLoader) – Training data loader. Combines a dataset and a sampler, and provides an iterable over the given dataset.

Returns:

res_test_concepts_auc (list[float]) – ROC AUC values for each of the concepts.
test_auc (float) – The ROC AUC value of a single mapping network or the ROC AUC value for all labels of a simultaneous mapping network.

get_activation_extractor()[source]

Returns the ActivationExtractor object.

Returns:: Class for identifying layers of a convolutional neural network and for extracting activations produced during network inference.
Return type:: ActivationExtractor

get_class_labels()[source]

Returns names of mapping network output classes.

Returns:: classes – Names of mapping network output classes.
Return type:: dict

get_mapping_net()[source]

Returns the mapping network.

Returns:: mapping_net – The mapping network.
Return type:: torch.nn.Module

load_model(path_to_model)[source]

Loads weights and class labels of the mapping network model from a file.

Parameters:: path_to_model (str) – The path to the file containing weights.

train_model_semisupervised(train_loader, valid_loader, optimizer, early_stopping, epochs, semantic_loss, sem_loss_weight, filename, class_labels, main_net_module_name, main_net_class, main_model_filename, transformation_name, img_size, num_channels)[source]

Trains a simultaneous mapping network for a given set of concepts using semi-supervised learning, in which a semantic loss is calculated for unlabeled samples, taking into account the relationships between the concepts.

Parameters:

train_loader (torch.utils.data.DataLoader) – Training data loader. Combines a dataset and a sampler, and provides an iterable over the given dataset.
valid_loader (torch.utils.data.DataLoader) – Validation data loader. Combines a dataset and a sampler, and provides an iterable over the given dataset.
optimizer (torch.optim.Optimizer) – The used weight optimizer of the mapping network.
early_stopping (EarlyStopping) – Class to stop training when validation loss stops improving.
epochs (int) – The number of training epochs of the mapping neural network.
semantic_loss (semantic_loss_pytorch.SemanticLoss) – An object of the semantic loss class, for initialization of which it is necessary to use the generated .sdd and .vtree.
sem_loss_weight (float) – The contribution of semantic loss to the overall loss function.
filename (str) – The name of the file in which the parameters of the trained model will be saved.
class_labels (list[str]) – Names of class labels used for training.
main_net_module_name (str) – The name of the file containing the main network class.
main_net_class (str) – Name of the main network class.
main_model_filename (str) – The file containing the parameters of the main network model.
transformation_name (str) – Name of the variable storing transformations.
img_size (int) – The size of the image side.
num_channels (int) – The number of image channels.

train_model_simultaneous(train_loader, valid_loader, optimizer, early_stopping, epochs, filename, class_labels, main_net_module_name, main_net_class, main_model_filename, transformation_name, img_size, num_channels)[source]

Trains a simultaneous mapping network for a given set of concepts.

Parameters:

train_loader (torch.utils.data.DataLoader) – Training data loader. Combines a dataset and a sampler, and provides an iterable over the given dataset.
valid_loader (torch.utils.data.DataLoader) – Validation data loader. Combines a dataset and a sampler, and provides an iterable over the given dataset.
optimizer (torch.optim.Optimizer) – The used weight optimizer of the mapping network.
early_stopping (EarlyStopping) – Class to stop training when validation loss stops improving.
epochs (int) – The number of training epochs of the mapping neural network.
filename (str) – The name of the file in which the parameters of the trained model will be saved.
class_labels (list[str]) – Names of class labels used for training.
main_net_module_name (str) – The name of the file containing the main network class.
main_net_class (str) – Name of the main network class.
main_model_filename (str) – The file containing the parameters of the main network model.
transformation_name (str) – Name of the variable storing transformations.
img_size (int) – The size of the image side.
num_channels (int) – The number of image channels.

train_model_single(train_loader, valid_loader, optimizer, early_stopping, epochs, filename, class_label, main_net_module_name, main_net_class, main_model_filename, transformation_name, img_size, num_channels)[source]

Trains a single mapping network for a given concept.

Parameters:

train_loader (torch.utils.data.DataLoader) – Training data loader. Combines a dataset and a sampler, and provides an iterable over the given dataset.
valid_loader (torch.utils.data.DataLoader) – Validation data loader. Combines a dataset and a sampler, and provides an iterable over the given dataset.
optimizer (torch.optim.Optimizer) – The used weight optimizer of the mapping network.
early_stopping (EarlyStopping) – Class to stop training when validation loss stops improving.
epochs (int) – The number of training epochs of the mapping neural network.
filename (str) – The name of the file in which the parameters of the trained model will be saved.
class_label (str) – The name of the label of the class used for training.
main_net_module_name (str) – The name of the file containing the main network class.
main_net_class (str) – Name of the main network class.
main_model_filename (str) – The file containing the parameters of the main network model.
transformation_name (str) – Name of the variable storing transformations.
img_size (int) – The size of the image side.
num_channels (int) – The number of image channels.

mapping_trainer

class revelionn.mapping_trainer.MappingTrainer(main_model_filepath, main_net_modules_directory, layers_types, patience, epochs, path_to_save, device, path_to_images, path_to_train_csv, path_to_valid_csv, image_names_column, batch_size, num_workers, path_to_test_csv)[source]

Mapping Trainer class provides an interface for learning/evaluating mapping networks

train_single_model(mapping_neurons, concept, layer_names)[source]: Trains a single mapping network for a given concept based on the activations of given layers.

train_simultaneous_model(concepts, decoder_channels, num_shared_neurons, num_output_neurons)[source]: Trains a simultaneous mapping network for a given set of concepts based on the activations of layers of previously defined types.

train_simultaneous_model_semisupervised(concepts, decoder_channels, num_shared_neurons, num_output_neurons,

semantic_loss, sem_loss_weight, unlabeled_samples): Trains a simultaneous mapping network for a given set of concepts using semi-supervised learning, in which a semantic loss is calculated for unlabeled samples, taking into account the relationships between the concepts.

evaluate_model()[source]: Evaluates the mapping network model on the test set using the ROC AUC.

evaluate_model()[source]

Evaluates the mapping network model on the test set using the ROC AUC.

Returns:: The ROC AUC value of a single mapping network or the ROC AUC value for all labels of a simultaneous mapping network.
Return type:: float

train_simultaneous_model(concepts, decoder_channels, num_shared_neurons, num_output_neurons)[source]

Trains a simultaneous mapping network for a given set of concepts based on the activations of layers of previously defined types.

Parameters:

concepts (list[str]) – The target concepts for which to train the mapping network.
decoder_channels (int) – The number of decoder channels. The output number of channels of the convolutional layer of the decoder or the output number of neurons of the decoder of the fully connected layer.
num_shared_neurons (list[int]) – The number of neurons in consecutive fully connected layers of the common part of the network (internal representation of the simultaneous extraction network).
num_output_neurons (list[int]) – The number of neurons in consecutive fully connected layers of each of the concept blocks.

train_simultaneous_model_semisupervised(concepts, decoder_channels, num_shared_neurons, num_output_neurons, semantic_loss, sem_loss_weight, unlabeled_samples)[source]

Trains a simultaneous mapping network for a given set of concepts using semi-supervised learning, in which a semantic loss is calculated for unlabeled samples, taking into account the relationships between the concepts.

Parameters:

concepts (list[str]) – The target concepts for which to train the mapping network.
decoder_channels (int) – The number of decoder channels. The output number of channels of the convolutional layer of the decoder or the output number of neurons of the decoder of the fully connected layer.
num_shared_neurons (list[int]) – The number of neurons in consecutive fully connected layers of the common part of the network (internal representation of the simultaneous extraction network).
num_output_neurons (list[int]) – The number of neurons in consecutive fully connected layers of each of the concept blocks.
semantic_loss (semantic_loss_pytorch.SemanticLoss) – An object of the semantic loss class, for initialization of which it is necessary to use the generated .sdd and .vtree.
sem_loss_weight (float) – The contribution of semantic loss to the overall loss function.
unlabeled_samples (int or float) – The number of unlabeled samples to include. If float, it represents the fraction of unlabeled samples.

train_single_model(mapping_neurons, concept, layer_names)[source]

Trains a single mapping network for a given concept based on the activations of given layers.

Parameters:

mapping_neurons (list[int]) – The number of neurons in consecutive fully connected layers. The output layer should always have 1 neuron.
concept (str) – The target concept for which to train the mapping network.
layer_names (list[str]) – A list of layer names to consider for training and evaluation.

concept_extraction

class revelionn.concept_extraction.ConceptExtractor(mapping_trainer, nxonto)[source]

A class that provides concept extraction algorithms.

ontology

Ontology represented as a graph, where edge direction goes from superterm to subterm.

Type:: nxontology.NXOntology

trainer

An instance of the MappingTrainer class that provides an interface for training mapping networks.

Type:: MappingTrainer

create_subgraph(graph, node)[source]: Returns a subgraph containing all child nodes for a given, including this one.

order_concepts(target_concept, ontology)[source]: Performs topological sorting of a subgraph formed by a given parent node (target concept).

exhaustive_search(concept, layer_names, mapping_neurons)[source]: Trains and evaluates mapping networks based on the activations of each of the specified layers of the network.

linear_search(concept, top_layer_num, patience_layers, mapping_neurons)[source]: Trains and evaluates mapping networks based on the activations of each of the layers starting from the specified one, until the value of the quality metric deteriorates over several layers (the value of patience).

heuristic_search(target_concept, top_layer_num, patience_layers, mapping_neurons)[source]: Due to the heuristic reduction of the set of specified layers, mapping networks are not trained for every combination of layer-concept. Uses linear search.

simultaneous_extraction(target_concept, decoder_channels, num_shared_neurons, num_output_neurons,: sdd_path=None, vtree_path=None, sem_loss_weight=None, unlabeled_samples=None)

Trains a mapping network that can simultaneously extract a set of relevant concepts from the entire set of layers of specified types (the types are set when initializing the MappingTrainer instance).

static create_subgraph(graph, node)[source]

Returns a subgraph containing all child nodes for a given node, including the given node.

Parameters:

graph (networkx.Graph) – The graph from which to extract the subgraph.
node (str) – The node for which to create the subgraph.

Returns:

A subgraph of graph containing all child nodes of node, including node.

Return type:

networkx.Graph

exhaustive_search(concept, layer_names, mapping_neurons)[source]

Trains and evaluates mapping networks based on the activations of each of the specified layers of the network.

Parameters:

concept (str) – The concept for which to perform the search.
layer_names (list) – A list of layer names to consider for training and evaluation.
mapping_neurons (list[int]) – The number of neurons in the mapping network.

Returns:

A dict containing the best layer name and the corresponding evaluation value.

Return type:

dict

heuristic_search(target_concept, top_layer_num, patience_layers, mapping_neurons)[source]

Due to the heuristic reduction of the set of specified layers, mapping networks are not trained for every combination of layer-concept. Uses linear search.

Parameters:

target_concept (str) – The target concept that should be obtained by ontological inference. Mapping networks are trained to extract concepts relevant to the target concept.
top_layer_num (int) – The starting layer number for training and evaluation.
patience_layers (int) – The number of layers to tolerate deterioration in the quality metric.
mapping_neurons (list[int]) – The number of neurons in the mapping network.

Returns:

A dictionary containing the best layer number and evaluation value for each concept in the subgraph.

Return type:

dict

linear_search(concept, top_layer_num, patience_layers, mapping_neurons)[source]

Trains and evaluates mapping networks based on the activations of each of the layers starting from the specified one, until the value of the quality metric deteriorates over several layers (the value of patience).

Parameters:

concept (str) – The concept for which to perform the search.
top_layer_num (int) – The starting layer number for training and evaluation.
patience_layers (int) – The number of layers to tolerate deterioration in the quality metric.
mapping_neurons (list[int]) – The number of neurons in the mapping network.

Returns:

A tuple containing the best layer number and the corresponding evaluation value.

Return type:

tuple

order_concepts(target_concept, ontology)[source]

Performs topological sorting of a subgraph formed by a given parent node (target concept).

Parameters:

target_concept (str) – The target concept node for which to perform topological sorting.
ontology (nxontology.NXOntology) – The ontology graph.

Returns:

A list of concepts in topologically sorted order within the subgraph.

Return type:

list

simultaneous_extraction(target_concept, decoder_channels, num_shared_neurons, num_output_neurons, sdd_path=None, vtree_path=None, sem_loss_weight=None, unlabeled_samples=None)[source]

Parameters:

target_concept (str) – The target concept that should be obtained by ontological inference. Mapping networks are trained to extract concepts relevant to the target concept.
decoder_channels (int) – The number of decoder channels. The output number of channels of the convolutional layer of the decoder or the output number of neurons of the decoder of the fully connected layer.
num_shared_neurons (list[int]) – The number of neurons in consecutive fully connected layers of the common part of the network (internal representation of the simultaneous extraction network).
num_output_neurons (list[int]) – The number of neurons in consecutive fully connected layers of each of the concept blocks.
sdd_path (str) – The path to the .sdd file.
vtree_path (str) – The path to the .vtree file.
sem_loss_weight (float) – The contribution of semantic loss to the overall loss function.
unlabeled_samples (int or float) – The number of unlabeled samples to include. If float, it represents the fraction of unlabeled samples.

Returns:

concepts_auc (list[float]) – ROC AUC values for each of the concepts.
all_auc (float) – ROC AUC value for all labels of a simultaneous mapping network.

datasets

class revelionn.datasets.MultiLabeledImagesDataset(annotations_file, img_dir, name_column, target_columns, transform=None)[source]

A PyTorch dataset class for multi-labeled image data.

img_labels

A pandas DataFrame containing the image annotations.

Type:: pd.DataFrame

img_dir

The directory path containing the images.

Type:: str

transform

A transform to apply to the image data.

Type:: torchvision.transforms

__len__()[source]: Returns the total number of samples in the dataset.

__getitem__(idx)[source]: Returns the image and corresponding labels at the given index.

labels()[source]: Returns a list of the target labels.

labels()[source]

Return a list of the target labels.

Returns:: A list of target labels.
Return type:: list

class revelionn.datasets.SemiSupervisedImagesDataset(annotations_file, img_dir, name_column, target_columns, unlabeled_samples, transform=None)[source]

A PyTorch dataset class for semi-supervised multi-labeled image data, inheriting from MultiLabeledImagesDataset.

img_labels

A pandas DataFrame containing the image annotations.

Type:: pd.DataFrame

img_dir

The directory path containing the images.

Type:: str

transform

A transform to apply to the image data.

Type:: torchvision.transforms

unlabeled_idx

An array containing the indices of unlabeled samples.

Type:: numpy.ndarray

__init__(annotations_file, img_dir, name_column, target_columns, unlabeled_samples, transform=None)[source]: Initialize the SemiSupervisedImagesDataset.

__getitem__(idx)[source]: Get the image, corresponding labels, and unlabeled flag at the given index.

separate_unlabeled(x_raw, y_raw, is_unlabeled)[source]: Separate the labeled and unlabeled samples from the given data.

static separate_unlabeled(x_raw, y_raw, is_unlabeled)[source]

Separate the labeled and unlabeled samples from the given data.

Parameters:

x_raw (torch.Tensor) – The input data.
y_raw (torch.Tensor) – The target labels.
is_unlabeled (torch.Tensor) – The unlabeled flags indicating whether a sample is labeled (0) or unlabeled (1).

Returns:

A tuple containing the labeled data, labeled target labels, unlabeled data, and unlabeled target labels.

Return type:

tuple

revelionn.datasets.create_dataloader(path_to_csv, path_to_images, image_names_column, target_columns, batch_size, num_workers, transformation, unlabeled_samples=None)[source]

Create a PyTorch DataLoader for loading the multi-labeled image dataset.

Parameters:

path_to_csv (str) – The file path to the annotations file in CSV format.
path_to_images (str) – The directory path containing the images.
image_names_column (str) – The name of the column in the annotations file that contains the image names.
target_columns (str or list[str]) – The column name(s) of the target labels in the annotations file.
batch_size (int) – The batch size for the DataLoader.
num_workers (int) – The number of worker processes to use for data loading.
transformation (torchvision.transforms) – A transform to apply to the image data.
unlabeled_samples (int or float, optional) – The number of unlabeled samples to include. If float, it represents the fraction of unlabeled samples. Default is None.

Returns:

A PyTorch DataLoader for the multi-labeled image dataset.

Return type:

torch.utils.data.DataLoader

Raises:

ValueError – If the value of the parameter ‘unlabeled_samples’ is invalid.

early_stopping

class revelionn.early_stopping.EarlyStopping(patience=7, verbose=False, delta=0, trace_func=<built-in function print>)[source]

Early stopping class to stop training when validation loss stops improving.

patience

Number of epochs to wait for improvement before stopping.

Type:: int

verbose

If True, prints a message when validation loss decreases and the model is saved.

Type:: bool

counter

Counter to track the number of epochs without improvement.

Type:: int

best_score

Best score (negative validation loss) obtained so far.

Type:: float or None

early_stop

Flag indicating whether to stop the training.

Type:: bool

val_loss_min

Minimum validation loss observed so far.

Type:: float

delta

Minimum change in the monitored quantity to qualify as an improvement.

Type:: float

trace_func

A function used to trace the output message.

Type:: function

__call__(val_loss)[source]: Call the early stopping class and determine whether to stop the training.

save_checkpoint(val_loss)[source]: Save the model checkpoint when the validation loss decreases.

save_checkpoint(val_loss)[source]

Save the model checkpoint when the validation loss decreases.

Parameters:: val_loss (float) – The current validation loss value.
Returns:: A message indicating that the validation loss decreased and the model is saved, or None.
Return type:: str

occlusion

class revelionn.occlusion.MultiLabelClassifier(main_module, mapping_module, activation_extractor, img_transformation)[source]

classify_images(image_iter)[source]

Classify an input iterable of images, in the form of np.ndarray matricies into a parallel iterable of label-to-confidence mappings (dictionaries).

We expect input image matrices to come in either the [H, W] or [H, W, C] dimension formats.

Each classification mapping should contain confidence values for each label the configured model contains. Implementations may act in a discrete manner whereby only one label is marked with a 1 value (others being 0), or in a continuous manner whereby each label is given a confidence-like value in the [0, 1] range.

Parameters:: array_iter – Iterable of images, as numpy arrays, to be classified.
Raises:: ValueError – Input arrays were not all of consistent dimensionality.
Returns:: Iterator of dictionaries, parallel in association to the input images. Each dictionary should map labels to associated confidence values.

get_config()[source]

Return a JSON-compliant dictionary that could be passed to this class’s from_config method to produce an instance with identical configuration.

In the most cases, this involves naming the keys of the dictionary based on the initialization argument names as if it were to be passed to the constructor via dictionary expansion. In some cases, where it doesn’t make sense to store some object constructor parameters are expected to be supplied at as configuration values (i.e. must be supplied at runtime), this method’s returned dictionary may leave those parameters out. In such cases, the object’s from_config class-method would also take additional positional arguments to fill in for the parameters that this returned configuration lacks.

Returns:: JSON type compliant configuration dictionary.
Return type:: dict

get_labels()[source]

Get the sequence of class labels that this classifier can classify images into. This includes the negative or background label if the classifier embodies such a concept.

Returns:: Sequence of possible classifier labels.
Raises:: RuntimeError – No model loaded.

revelionn.occlusion.perform_occlusion(main_module, mapping_module, activation_extractor, transformation, img_size, path_to_img, window_size, stride, threads)[source]

Highlights concepts extracted by the mapping network in the image by occlusion.

Parameters:

main_module (MainModelProcessing) – Class for training, evaluation and processing the main network model.
mapping_module (MappingModelProcessing) – Class for training, evaluation and processing the mapping network model.
activation_extractor (ActivationExtractor) – Class for identifying layers of a convolutional neural network and for extracting activations produced during network inference.
transformation (torchvision.transforms) – A transform to apply to the image.
img_size (int) – The size of the image side.
path_to_img (str) – Image file path.
window_size (int) – The block window size.
stride (int) – The sliding window striding step.
threads (int) – Optional number threads to use to enable parallelism in applying perturbation masks to an input image. If 0, a negative value, or None, work will be performed on the main-thread in-line.

Returns:

plt

Return type:

matplotlib.pyplot

utils.explanation

revelionn.utils.explanation.explain_target_concept(extracted_concepts, mapping_probabilities, concepts_map, target_concept, ontology_filepath, path_to_temp_files)[source]

Parameters:

extracted_concepts (list[str]) – Concepts relevant to the target concept, which are extracted by the mapping network.
mapping_probabilities (list[float]) – The probabilities, obtained from the output of the sigmoid, of each of the extracted concepts relevant to the target concept.
concepts_map (dict) – Dictionary whose keys are the names of the attributes of the dataset, and the values are the corresponding concepts of the ontology.
target_concept (str) – The concept of ontology, which should be obtained by ontological inference from the extracted concepts.
ontology_filepath (str) – Path to the OWL ontology file.
path_to_temp_files – Temporary files directory for storing observations and explanations.

Returns:

justifications – A set of obtained justifications of the target class.

Return type:

str

revelionn.utils.explanation.extract_concepts_from_img(main_module, mapping_module, img, transformation)[source]

Extracts a set of concepts present in a given image.

Parameters:

main_module (MainModelProcessing) – Class for training, evaluation and processing the main network model.
mapping_module (MappingModelProcessing) – Class for training, evaluation and processing the mapping network model.
img (PIL.Image) – Class that represents a PIL image.
transformation (torchvision.transforms) – A transform to apply to the image.

Returns:

main_concept (list[str]) – Target concept extracted by the convolutional network.
extracted_concepts (list[str]) – Concepts relevant to the target concept, which are extracted by the mapping network.
mapping_probabilities (list[float]) – The probabilities, obtained from the output of the sigmoid, of each of the extracted concepts relevant to the target concept.

revelionn.utils.explanation.to_main_observation(concept)[source]

Formats a string from the name of the target concept to be parsed by the justifier.

Parameters:: concept (str) – Name of the target concept.
Returns:: String from the name of the target concept to be parsed by the justifier.
Return type:: str

revelionn.utils.explanation.to_mapping_observation(concept, probability)[source]

Formats a string from the name of the concept relevant to the target concept, which will be parsed by the justifier.

Parameters:

concept (str) – Name of the concept relevant to the target concept.
probability (float) – The probability of the concept obtained at the output of the sigmoid.

Returns:

String from the name of the concept relevant to the target concept, which will be parsed by the justifier.

Return type:

str

utils.model

revelionn.utils.model.convert_to_rvl_format(main_model, filename, class_label, module_name, main_net_class, transformation_name, img_size, num_channels)[source]

Converts the pre-trained main network model to RevelioNN format. Creates the converted model as an RVL file.

Parameters:

main_model (torch.nn.Module) – Main network model with loaded weights.
filename (str) – Filename (path) to save the converted model.
class_label (str) – Name of the output class label of the main network.
module_name (str) – Name of the module (.py file name) containing the class of the main network.
main_net_class (str) – Name of the main network class.
transformation_name (str) – Name of the variable storing transformations.
img_size (int) – Size of the image side.
num_channels (int) – Number of image channels.

revelionn.utils.model.load_main_model(main_model_filepath, main_net_modules_directory, device)[source]

Loads the main network model in RevelioNN format from a file. Initializes and returns a class to work with the main net, as well as a transformation object and image size.

Parameters:

main_model_filepath (str) – File path containing the parameters of the main network model.
main_net_modules_directory (str) – Directory containing .py files with classes of the main networks.
device (torch.device) – Tensor processing device.

Returns:

main_module (MainModelProcessing) – Class for training, evaluation and processing the main network model.
transformation (torchvision.transforms) – A transform to apply to the images.
img_size (int) – Size of the image side.

revelionn.utils.model.load_mapping_model(mapping_model_filepath, main_models_directory, main_net_modules_directory, device)[source]

Loads the mapping network model from a file. Initializes and returns a class to work with the main net, as well as a transformation object and image size.

Parameters:

mapping_model_filepath (str) – File path containing the parameters of the mapping network model.
main_models_directory (str) – Directory containing files with parameters of the main network models.
main_net_modules_directory (str) – Directory containing .py files with classes of the main networks.
device (torch.device) – Tensor processing device.

Returns:

main_module (MainModelProcessing) – Class for training, evaluation and processing the main network model.
mapping_module (MappingModelProcessing) – Class for training, evaluation and processing the mapping network model.
activation_extractor (ActivationExtractor) – Class for identifying layers of the main network and for extracting activations produced during network inference.
transformation (torchvision.transforms) – A transform to apply to the images.
img_size (int) – Size of the image side.