tensorstate.Layers¶

The Layers module contains classes that interact with neural network layers.

class AbstractStateCapture(name, disk_path=None, **kwargs)¶

Bases: abc.ABC

Base class for capturing state space information in a neural network.

This class implements the infrastructure used to capture, quantize, and process state space information. For Tensorflow, a subclass is constructed to inherit these methods as a layer to be inserted into the network. For PyTorch, a subclass is constructed to implement these methods as layer hooks.

This class captures state information and quantizes layer outputs as firing or not firing based on whether the values are >0 or <=0 respectively. Although this layer is intended to be attached before or after a neural layer, this can actually be attached to any layer type. After recording the firing state of all neurons, the original input is returned unaltered. Thus, this layer can be thought of as a “probe”, since it does not add or subtract from the function of a network.

Layer states are stored in a zarr array, which permits compressed storage of data in memory or on disk. Only blosc compression is used to ensure fast compression/decompression speeds. By default, data is stored in memory, but data can be stored on disk to reduce memory consumption by using the disk_path keyword.

NOTE: This layer currently only works within Tensorflow Keras models and PyTorch models.

counts()¶

Layer state counts

This method returns a numpy.array of integers, where each integer is the number of times a state is observed. The identity of the states can be obtained by calling the state_ids method.

NOTE: The list only contains counts for observed states, so all values will be >0

Returns:	Counts of stat occurences
Return type:	[list of `int`]

efficiency(alpha1=1, alpha2=None)¶

Calculate the efficiency of the layer

This method returns the efficiency of the layer. Originally, the efficiency was defined as the ratio of Shannon’s entropy to the theoretical maximum entropy based on the number of neurons in the layer. This method with no inputs will return that value. However, this method will also now permit defining the alpha value for the Renyi entropy, so that the efficiency will be calculated as the Renyi entropy of order alpha1 divided by the maximum theoretical entropy.

Parameters:	alpha1 ([float, int], optional) – Order of Renyi entropy in numerator alpha2 ([float, int, None], optional) – Order of Renyi entropy in denominator
Returns:	The efficiency of the layer
Return type:	[float]

entropy(alpha=1)¶

Calculate the entropy of the layer

Calculate the entropy from the observed states. The alpha value is the order of entropy calculated using the formula for Renyi entropy. When alpha=1, this returns Shannon’s entropy.

Parameters:	alpha (int, None) – Order of entropy to calculate. If `None`, then use `max_entropy()`
Returns:	The entropy of the layer
Return type:	[float]

max_entropy()¶

Theoretical maximum entropy for the layer

The maximum entropy for the layer is equal to the number of neurons in the layer. This is different than the maximum entropy value that would be returned from the TensorState.entropy method with alpha=0, which is a count of the observed states.

Returns:	Theoretical maximum entropy value
Return type:	[float]

raw_states¶: Raw state data as stored in memory, bit compressed

reset_states(input_shape=None)¶

Initialize the state space

This method initializes the layer and resets any previously held data. The zarr array is initialized in this method.

Parameters:	input_shape (TensorShape,tuple, list) – Shape of the input.

state_count¶: The total number of observed states, including repeats.

state_ids()¶

Identity of observed states

This method returns a list of byte arrays. Each byte array corresponds to a unique observed state, where each bit in the byte array corresponds to a neuron. The list returned by this method matches the list returned by counts, so that the value in state_ids at position i is associated with the counts value at position i.

For example, if the StateCapture layer is attached to a convolutional layer with 8 neurons, then each item in the list will be a byte array of length 1. If one of the bytes is \x00 (a null byte), then the state has no firing neurons.

NOTE: Only observed states are contained in the list.

Returns:	Unique states observed by the layer
Return type:	[list of Bytes]

states¶: Decompressed state data

class StateCapture(name, disk_path=None, **kwargs)¶

Bases: TensorState.Layers.AbstractStateCapture

Tensorflow keras layer to capture states in keras models

This class is designed to be used in a Tensorflow keras model to automate the capturing of neurons states as data is passed through the network.

build(input_shape)¶

Build the StateCapture Keras Layer

This method initializes the layer and resets any previously held data. The zarr array is initialized in this method.

Parameters:	input_shape (TensorShape) – Either a TensorShape or list of TensorShape instances.

call(inputs)¶

class StateCaptureHook(name, disk_path=None, **kwargs)¶

Bases: TensorState.Layers.AbstractStateCapture

StateCapture hook for PyTorch

This class implements all methods in AbstractStateCapture, but is designed to be a pre or post hook for a layer.