utils
Datasets
HDF5Dataset
- class indexedconv.utils.HDF5Dataset(path, transform=None, target_transform=None)
Loads data in a Dataset from a HDF5 file.
- Parameters:
path (str) – The path to the HDF5 file.
transform (callable, optional) – A callable or a composition of callable to be applied to the data.
target_transform (callable, optional) – A callable or a composition of callable to be applied to the labels.
NumpyDataset
- class indexedconv.utils.NumpyDataset(data, labels, transform=None, target_transform=None)
Loads data in a Dataset from
numpy.array
.- Parameters:
data (numpy.array) – The data to load.
labels (numpy.array) – The labels of the data.
transform (callable, optional) – A callable or a composition of callable to be applied to the data.
target_transform (callable, optional) – A callable or a composition of callable to be applied to the labels.
Transforms
NumpyToTensor
- class indexedconv.utils.NumpyToTensor
Converts a numpy array to a tensor.
SquareToHexa
- class indexedconv.utils.SquareToHexa
Converts an image with a square grid to an image with a hexagonal one.
Image processing
- class indexedconv.utils.PCA(D, n_components)
Credit E. Hoogeboom http://github.com/ehoogeboom/hexaconv
- fit(D, n_components)
The computation works as follows:
The covariance is
C = 1/(n-1) * D * D.T
The eigendecomp of C is:
C = V Sigma V.T
Let
Y = 1/sqrt(n-1) * D
Let
U S V = svd(Y)
,Then the columns of U are the eigenvectors of
Y * Y.T = C
And the singular values S are the sqrts of the eigenvalues of C
We can apply PCA by multiplying by U.T
- transform(D, whiten=False, ZCA=False, regularizer=1e-05)
We want to whiten, which can be done by multiplying by \(\sigma^{-1/2} U.T\)
Any orthogonal transformation of this is also white, and when
ZCA=True
we choose \(U \sigma^{-1/2} U.T\)
- indexedconv.utils.normalize(images)
Normalizes images.
- Parameters:
images – image tensor of shape (c, n, m)
- indexedconv.utils.build_hexagonal_position(index_matrix)
Computes the position of the pixels in the hexagonal grid from the index matrix.
- Parameters:
index_matrix (tensor) – The index matrix representing the index of each pixel in the axial addressing system.
- indexedconv.utils.square_to_hexagonal(image)
Rough sampling of square images to hexagonal grid
- Parameters:
image – image tensor of shape (c, n, m)
- indexedconv.utils.square_to_hexagonal_index_matrix(image)
Creates the index matrix of square images in a hexagonal grid (axial addressing system).
- Parameters:
image – input tensor of shape (c, n, m)
Indexed functions
- indexedconv.utils.build_kernel(kernel_type, radius=1, dilation=1)
Builds the convolution kernel or mask. (Following the suggestion of Miguel Lallena).
- Parameters:
kernel_type (str) – The type of kernel. Can be hexagonal (‘Hex’) or square (‘Square’).
radius (int) – The radius of the kernel.
dilation (int) – The dilation. A dilation of 1 means no dilation.
- Returns:
A
np.array
.
- indexedconv.utils.neighbours_extraction(index_matrix, kernel_type='Hex', radius=1, stride=1, dilation=1, retina=False)
Builds the matrix of indices from an index matrix based on a kernel.
The matrix of indices contains for each pixel of interest its neighbours, including itself.
- Parameters:
index_matrix (torch.Tensor) – Matrix of index for the images, shape(1, 1, matrix.size).
kernel_type (str) – The kernel shape, Hex for hexagonal Square for a square and Pool for a square of size 2.
radius (int) – The radius of the kernel.
stride (int) – The stride.
dilation (int) – The dilation. A dilation of 1 means no dilation.
retina (bool) – Whether to build a retina like kernel. If True, dilation must be 1.
- Returns:
A torch.Tensor - the matrix of the neighbours.
Example
>>> index_matrix = [[0, 1, -1], [2, 3, 4], [-1, 5, 6]] [[0, 1, -1], [2, 3, 4], [-1, 5, 6]] >>> kernel_type = 'Hex' >>> radius = 1 >>> kernel [[1, 1, 0], [ 1, 1, 1], [ 0, 1, 1]] >>> stride = 1 >>> neighbours = neighbours_extraction(index_matrix, kernel_type, radius, stride) [[-1, -1, -1, 0, 1, 2, 3], [ -1, -1, 0, 1, -1, 3, 4], [ -1, 0, -1, 2, 3, -1, 5], [ 0, 1, 2, 3, 4, 5, 6], [ 1, -1, 3, 4, -1, 6, -1], [ 2, 3, -1, 5, 6, -1, -1], [ 3, 4, 5, 6, -1, -1, -1]]
- indexedconv.utils.create_index_matrix(nbRow, nbCol, injTable)
Creates the matrix of index of the pixels of the images of any shape stored as vectors.
- Parameters:
nbRow (int) – The number of rows of the index matrix.
nbCol (int) – The number of cols of the index matrix.
injTable (numpy.array) – The injunction table, i.e. the list of the position of every pixels of the vector image in a vectorized square image.
- Returns:
A torch.Tensor containing the index of each pixel represented in a matrix.
Example
>>> image = [0, 1, 2, 3, 4, 5, 6] # hexagonal image stored as a vector >>> # in the hexagonal space 0 1 >>> # 2 3 4 >>> # 5 6 >>> # injunction table of the pixel position of a hexagonal image represented in the axial addressing system >>> injTable = [0, 1, 3, 4, 5, 7, 8] >>> index_matrix = [[0, 1, -1], [2, 3, 4], [-1, 5, 6]] [[0, 1, -1], [2, 3, 4], [-1, 5, 6]]
- indexedconv.utils.pool_index_matrix(index_matrix, kernel_type='Pool', stride=2)
Pools an index matrix.
- Parameters:
index_matrix (torch.Tensor) – The index matrix for the images, shape(1, 1, matrix.size).
kernel_type (str) – The kernel shape, Hex for hexagonal, Square for a square of size 3 and Pool for a square of size 2.
stride (int) – The stride.
- Returns:
A torch.Tensor containing the pooled matrix.
- indexedconv.utils.prepare_mask(indices)
Prepares the indices and the mask for the GEMM im2col operation.
- Parameters:
indices (torch.Tensor) – The matrix of indices containing the neighbours of each pixel of interest.
- indexedconv.utils.img2mat(input_images, index_matrix)
Transforms a batch of features of vector images in a batch of features of matrix images.
- Parameters:
input_images (torch.Tensor) – The images with shape (batch, features, image).
index_matrix (torch.Tensor) – The index matrix containing the index of the pixels of the images. represented in a matrix
- Returns:
A torch.Tensor
- indexedconv.utils.mat2img(input_matrix, index_matrix)
Transforms a batch of features of matrix images in a batch of features of vector images.
- Parameters:
input_matrix (torch.Tensor) – The images with shape (batch, features, matrix.size).
index_matrix (torch.Tensor) – The index matrix for the images, shape(1, 1, matrix.size).
Utilities
- indexedconv.utils.get_gpu_usage_map(device_id)
Get the current gpu usage. Inspired from gpu usage from pytorch
- Parameters:
device_id (int) – the GPU id as GPU/Unit’s 0-based index in the natural enumeration returned by the driver
- Returns
- dict - usage
Keys are device ids as integers. Values are memory usage as integers in MB, total memory usage as integers in MB, gpu utilization in %.
- indexedconv.utils.compute_total_parameter_number(net)
Computes the total number of parameters of a network.
- Parameters:
net (
nn.Module
) – The network.- Returns:
A int