Welcome to SPAIC’s documentation!

SPAIC (spike based artificial intelligence computing) is a brain-inspired computing framework for combining neuroscience with machine learning.

• 中文首页(Chinese homepage)

I. How to install

Install the latest version from source code:

From GitHub:

git clone https://github.com/ZhejianglabNCRC/SPAIC.git
cd SPAIC
python setup.py install

II. How to build a spiking neural network

In order to facilitate users to understand how to use SPAIC to carry out their own research work, we will use STCA learning algorithm [1] to train the network to recognize MNIST data set as an example to build a SNN training network.

1. Construct a network class

Network is the most important part of SPAIC, like the framework of the whole neural network, so we need to build a network class first, and then fill this network with other elements, such as neurons and connections. Inherit spaic.Network To recreate and instantiate a network class:

class SampleNet(spaic.Network):
   def __init__(self):
      super(SampleNet, self).__init__()
         ......

Net = SampleNet()

2. Add Network Components

When building the framework of Network, we need to add neurons and connect these components in it, so that the Network will not be an empty framework. The components that can be added include the input and output parts: Node; the neurongroup NeuronGroups; the synapse connection connection; the monitor monitor ;the learning algorithm: learner. Also, we can add some special components when building some large and complex networks, Assembly and Projection , which used to let the complex structures more clearly.

2.1 Create Node and NeuronGroups

For a network that uses STCA algorithm and recognize MNIST dataset, the node we need is a Node.Encoder layer as input to encode the input data, a clif NeuronGroup layer for training and a Node.Decoder layer as output to decode the output data. So, what we need to do is add:

self.input = spaic.Encoder(num=784, coding_method='poisson')
self.layer1 = spaic.NeuronGroup(num=10, model='clif')
self.output = spaic.Decoder(num=10, dec_target=self.layer1)

Note

To be mentioned, the number of the neuron in output need to be same as the target layer of dec_target .

2.2 Construct connections

In this example, since the network structure is fairly simple, all required is a simple full connection that connecting the input layer to the training layer.

self.connection1 = spaic.Connection(self.input, self.layer1, link_type='full')

2.3 Add learning algorithm and optimization algorithm

In this example, we use STCA algorithm [1], it is a BPTT algorithm that use surrogate gradient strategy. And we choose Adam as our optimizer and set

self.learner = spaic.Learner(trainable=self, algorithm='STCA')
self.learner.set_optimizer('Adam', 0.001)

2.4 Add monitor

In this example, although it is not necessary to add a monitor, we can monitor the voltage and spike output of layer1 for teaching purposes, i.e.

self.mon_V = spaic.StateMonitor(self.layer1, 'V')
self.spk_O = spaic.SpikeMonitor(self.layer1, 'O')

2.5 Add backend

Backend is an extremely important part of SPAIC, responsible for the actual simulation of the backend network. backend.dt is used to set the time step for network simulation, which needs to be set in advance before establishing the network. The selection of different backends and devices also needs to be set up before building the network. In this example, we use PyTorch as the backend simulator and build the network with cuda. Use 0.1ms as the time step

# Method 1:
if torch.cuda.is_available():
    device = 'cuda'
else:
    device = 'cpu'
backend = spaic.Torch_Backend(device)
backend.dt = 0.1
self.set_backend(backend)

# Method 2:
self.set_backend('PyTorch', 'cuda')
self.set_backend_dt(0.1)

2.6 Overall network structure

import spaic
import torch

class TestNet(spaic.Network):
    def __init__(self):
        super(TestNet, self).__init__()

        # Encoding
        self.input = spaic.Encoder(num=784, coding_method='poisson')

        # NeuronGroup
        self.layer1 = spaic.NeuronGroup(num=10, model='clif')

        # Decoding
        self.output = spaic.Decoder(num=10, dec_target=self.layer1, coding_method='spike_counts')

        # Connection
        self.connection1 = spaic.Connection(pre=self.input, post=self.layer1, link_type='full')

        # Monitor
        self.mon_V = spaic.StateMonitor(self.layer1, 'V')
        self.spk_O = spaic.SpikeMonitor(self.layer1, 'O')

        # Learner
        self.learner = spaic.Learner(trainable=self, algorithm='STCA')
        self.learner.set_optimizer('Adam', 0.001)

        if torch.cuda.is_available():
            device = 'cuda'
        else:
            device = 'cpu'
        backend = spaic.Torch_Backend(device)
        backend.dt = 0.1

        self.set_backend(backend)

# Network instantiation
Net = TestNet()

3. Start training

3.1 Load the dataset

from tqdm import tqdm
import torch.nn.functional as F
from spaic.IO.Dataset import MNIST as dataset
# Create the training data set
root = './spaic/Datasets/MNIST'
train_set = dataset(root, is_train=True)
test_set = dataset(root, is_train=False)

# Set the run time and batch size
run_time = 50
bat_size = 100

# Create the DataLoader iterator
train_loader = spaic.Dataloader(train_set, batch_size=bat_size, shuffle=True, drop_last=False)
test_loader = spaic.Dataloader(test_set, batch_size=bat_size, shuffle=False)

3.2 Run the network

eval_losses = []
eval_acces = []
losses = []
acces = []
num_correct = 0
num_sample = 0
for epoch in range(100):

    # Train
    print("Start training")
    train_loss = 0
    train_acc = 0
    pbar = tqdm(total=len(train_loader))
    for i, item in enumerate(train_loader):
        # forward propagation
        data, label = item
        Net.input(data)
        Net.output(label)
        Net.run(run_time)
        output = Net.output.predict
        output = (output - torch.mean(output).detach()) / (torch.std(output).detach() + 0.1)
        label = torch.tensor(label, device=device)
        batch_loss = F.cross_entropy(output, label)

        # Back propagation
        Net.learner.optim_zero_grad()
        batch_loss.backward(retain_graph=False)
        Net.learner.optim_step()

        # Record the error
        train_loss += batch_loss.item()
        predict_labels = torch.argmax(output, 1)
        num_correct = (predict_labels == label).sum().item()  # Record the number of correct tags
        acc = num_correct / data.shape[0]
        train_acc += acc

        pbar.set_description_str("[loss:%f]Batch progress: " % batch_loss.item())
        pbar.update()
    pbar.close()
    losses.append(train_loss / len(train_loader))
    acces.append(train_acc / len(train_loader))
    print('epoch:{},Train Loss:{:.4f},Train Acc:{:.4f}'.format(epoch, train_loss / len(train_loader), train_acc / len(train_loader)))

    # Test
    eval_loss = 0
    eval_acc = 0
    print("Start testing")
    pbarTest = tqdm(total=len(test_loader))
    with torch.no_grad():
        for i, item in enumerate(test_loader):
            data, label = item
            Net.input(data)
            Net.run(run_time)
            output = Net.output.predict
            output = (output - torch.mean(output).detach()) / (torch.std(output).detach() + 0.1)
            label = torch.tensor(label, device=device)
            batch_loss = F.cross_entropy(output, label)
            eval_loss += batch_loss.item()

            _, pred = output.max(1)
            num_correct = (pred == label).sum().item()
            acc = num_correct / data.shape[0]
            eval_acc += acc
            pbarTest.set_description_str("[loss:%f]Batch progress: " % batch_loss.item())
            pbarTest.update()
        eval_losses.append(eval_loss / len(test_loader))
        eval_acces.append(eval_acc / len(test_loader))
    pbarTest.close()
    print('epoch:{},Test Loss:{:.4f},Test Acc:{:.4f}'.format(epoch,eval_loss / len(test_loader), eval_acc / len(test_loader)))

4. Training results

After training and testing 100 epochs, we get the following accuracy curve through matplotlib

from matplotlib import pyplot as plt
plt.subplot(2, 1, 1)
plt.plot(acces)
plt.title('Train Accuracy')
plt.ylabel('Acc')
plt.xlabel('epoch')

plt.subplot(2, 1, 2)
plt.plot(test_accuracy)
plt.title('Test Accuracy')
plt.ylabel('Acc')
plt.xlabel('epoch')

plt.show()

5. Save model

After the training is completed, we can store the weight information through the built-in function Network.save_state , or use spaic.Network_saver.network_save to store the overall network structure and weight.

Method 1: (only store weights)

save_file = Net.save_state("TestNetwork")

Method 2: (store network structure and weights at the same time)

save_file = network_save(Net, "TestNetwork", trans_format='json')

Note

In method 2, the format of the network structure storage can be json or yaml , both of which can be read directly without transmitted. The weights of the first and second methods are stored in the tensor format of Pytorch recently.

Additional: Other ways of constructing networks

1. the form using ‘with’

# Initializes an object of the base network class
Net = spaic.Network()

# The network structure is established by defining network units in with
with Net:
    # Create an input node and select the input encoding form
    input1 = spaic.Encoder(784, encoding='poisson')

    # Establish the neuron cluster, select the neuron type, and set the neuron
    # parameters such as discharge threshold and membrane voltage time constant
    layer1 = spaic.NeuronGroup(10, model='clif')

    # Establish connections between nerve clusters
    connection1 = spaic.Connection(input1, layer1, link_type='full')

    # Set up the output node and select the output decoding form
    output = spaic.Decoder(num=10, dec_target=self.layer1,
                                       coding_method='spike_counts')


    # establish state detector, it can monitor the state of neurons, input and
    # output nodes, connections and other units
    monitor1 = spaic.StateMonitor(layer1, 'V')

    # Add the learning algorithm and select the network structure to be trained,
    # (self stands for the whole ExampleNet structures)
    learner = spaic.Learner(trainable=self, algorithm='STCA')

    # Add optimization algorithm
    learner.set_optimizer('Adam', 0.001)

2. Import and modify the base model to construct a network model

from spaic.Library import ExampleNet
Net = ExampleNet()
# Neuron parameter
neuron_param = {
    'tau_m': 8.0,
    'V_th': 1.5,
}
# Creating new neuron clusters
layer3 = spaic.NeuronGroup(100, model='lif', param=neuron_param)
layer4 = spaic.NeuronGroup(100, model='lif', param=neuron_param)

# Add new set members to a neural set
Net.add_assembly('layer3', layer3)
# Delete an existing set member from the neural set
Net.del_assembly(Net.layer3)
# Copy an existing Assembly structure and add the new assembly to the neural set
Net.copy_assembly('net_layer', ExampleNet())
# Replace an existing neural set within the set with a new neural set
Net.replace_assembly(Net.layer1, layer3)
# Combine this neural with another neural set to obtain a new neural set
Net2 = ExampleNet()
Net.merge_assembly(Net2)
# Connect two nerve clusters within a nerve set
con = spaic.Connection(Net.layer2, Net.net_layer, link_type='full')
Net.add_connection('con3', con)
# Take out some members of this nerve set and the connections between them
# to form a new nerve set, and the original nerve set remains unchanged
Net3 = Net.select_assembly([Net.layer2, net_layer])

User manual:

Basic Structure

Basic components

Assembly is the most important basic class in SPAIC . spaic.Assembly contains three part: Network ,NeuronGroup and Node . spaic.Network is the basic class of the whole model. spaic.NeuronGroup contains neurons. spaic.Node is the basic class of the input and output nodes.

Front-end structure：

Assembly

Assembly is an abstract class of neural network structure, representing any network structure, and other network modules are subclasses of the Assembly . Assembly has two properties named _groups and_connections that save the neurons and connections. As the main interface for network model, it contains the following main functions:

• add_assembly(name, assembly) – Add new assembly members into the neuron assembly

• del_assembly(assembly=None, name=None) – Delete exist members from the neuron assembly

• copy_assembly(name, assembly) – copy an exist assembly and add it into this assemlby

• replace_assembly(old_assembly, new_assembly) – replace an exist member with a new assembly

• merge_assembly(assembly) – merge the given assembly into another assembly

• select_assembly(assemblies, name=None) – choose part of the target assembly as a new assembly

• add_connection(name, connection) – add a new connection into the assembly

• del_connection(connection=None, name=None) – delete the target connection

• assembly_hide() – hide this assembly

• assembly_show() – show this assembly

NeuronGroup

spaic.NeuronGroup is the class with some neurons, usually, we call it a layer of neuron with same neuron model and connection ways. For more details, please look up Neuron

Node

spaic.Node is the transform node of model, it contains a lot of encoding and decoding methods. Encoder , Decoder , Generator , Action and Reward are all inherited from Node. For more details, please look up Encoder & Decoder

Network

spaic.Network is the most top structure in SPAIC , all modules like NeuronGroup or Connection should be contained in it. spaic.Network also controls the training, simulation and some data interaction process. spaic.Network supports some useful interface as follow:

• run(run_time) – run the model, run_time is the time window

• save_state – save the weight of model

• state_from_dict – load weight of model

Projection

spaic.Projection is a high-level abstract class of topology, instead of Connection , Project represent the connections between Assembly s. Connection is subclass of Projection , when user build Projection between Assembly s, the construct function will build the corresponding connections according to the policies of Projection

Connection

spaic.Connection is used to build connections between NeuronGroup s, it also used to construct different synapses. For more details, please look up Connection

Backend

The core of backend that construct variables and generate computational graph. For more details, please look up Backend 。

More details

Neuron

This chapter introduces how to choose neuron model and change some important parameters of the model.

neuron model

Neuron model is one of the most important component of the model. Different neuron model will have different neuron dynamics. In spiking neuron network, people always convert the change of membrane potential of neuron model into different equation and approximate it by difference equation. Finally, obtain the differential neuron model that can be computed by computer. In SPAIC , we contains most of the common neuron models:

• IF - Integrate-and-Fire model

• LIF - Leaky Integrate-and-Fire model

• CLIF - Current Leaky Integrate-and-Fire model

• GLIF - Generalized Leaky Integrate-and-Fire model

• aEIF - Adaptive Exponential Integrate-and-Fire model

• IZH - Izhikevich model

• HH - Hodgkin-Huxley model

In SPAIC , NeuronGroup is like nodes of the network model. Like layers in PyTorch , in SPAIC , NeuronGroup is the layer. Users need to specify the neuron numbers, neuron model or other related paramters.

from spaic import NeuronGroup

LIF neuron model

LIF(Leaky Integrated-and-Fire Model) neuron formula and parameters:

\[\begin{split}V & = tua\_m * V + I \\ O & = spike\_func(V^n)\end{split}\]

For example, we build a layer with 100 LIF neurons:

self.layer1 = NeuronGroup(num=100, model='lif')

A layer with 100 standard LIF neurons has been constructed. While, sometimes we need to specify the LIF neuron to get different neuron dynamics, that we will need to specify some parameters:

• tau_m - time constant of neuron membrane potential, default as 6.0

• v_th - the threshold voltage of a neuron, default as 1.0

• v_reset - the reset voltage of the neuron, which defaults to 0.0

If users need to change these parameters, they can enter the parameters when construct NeuronGroups .

self.layer2 = NeuronGroup(num=100, model='lif',
                tau_m=10.0, v_th=10, v_reset=0.2)

CLIF neuron model

CLIF(Current Leaky Integrated-and-Fire Model) neuron formula and parameters:

\[\begin{split}V(t) & = M(t) - S(t) - E(t) \\ I & = V0 * I \\ M & = tau\_p * M + I \\ S & = tau\_q * S + I \\ E & = tau\_p * E + Vth * O \\ O & = spike\_func(V)\end{split}\]

• tau_p, tau_q - time constants of synapse, default as 12.0 and 8.0

• tau_m - time constant of neuron membrane potential, default as 20.0

• v_th - the threshold voltage of a neuron, default as 1.0

GLIF neuron model

GLIF(Generalized Leaky Integrate-and-Fire Model) [1] neuron parameters:

• R, C, E_L

• Theta_inf

• f_v

• delta_v

• b_s

• delta_Theta_s

• k_1, k_2

• delta_I1, delta_I2

• a_v, b_v

aEIF neuron model

aEIF(Adaptive Exponential Integrated-and-Fire Model) [2] neuron model and parameters:

\[\begin{split}V & = V + dt / C * (gL * (EL - V + EXP) - w + I^n[t]) \\ w & = w + dt / tau\_w * (a * (V - EL) - w) \\ EXP & = delta\_t * exp(dv\_th/delta\_t) \\ dv & = V - EL \\ dv\_th & = V - Vth \\ O & = spike\_func(V^n) \\ if\quad V & > 20: \\ then\quad V & = EL, w = w + b\end{split}\]

• C, gL - membrane capacitance and leak conductance

• tau_w - adaptation time constant

• a. - subthreshold adaptation

• b. - spike-triggered adaptation

• delta_t - slope factor

• EL - leak reversal potential

IZH neuron model

IZH(Izhikevich Model) [3] neuron model and parameters:

\[\begin{split}V &= V + dt / tau\_M * (C1 * V * V + C2 * V + C3 - U + I) \\ V &= V + dt / tau\_M * (V* (C1 * V + C2) + C3 - U + I) \\ U &= U + a. * (b. * V - U) \\ O &= spike\_func(V^n) \\ if\quad V &> Vth, \\ then\quad V &= Vreset, U = U + d\end{split}\]

• tau_m

• C1, C2, C3

• a, b, d

• Vreset - Voltage Reset

HH neuron model

HH(Hodgkin-Huxley Model) [4] neuron model and parameters:

\[\begin{split}V & = V + dt/tau\_v * (I - Ik) \\ Ik & = NA + K + L \\ NA & = g\_NA * m^3 * h * (V - V_NA) \\ K & = g\_K * n^4 * (V - V_K) \\ L & = g\_L * (V - V_L) \\ K\quad activation: \\ n & = n + dt/tau\_n * (alpha\_n * (1-n) - beta\_n * n) \\ Na\quad activation: \\ m & = m + dt/tau\_m * (alpha\_m * (1-m) - beta\_m * m) \\ Na\quad inactivation: \\ h & = h + dt/tau\_h * (alpha\_h * (1-h) - beta\_h * h) \\ alpha\_m & = 0.1 * (-V + 25) / (exp((-V+25)/10) - 1) \\ beta\_m & = 4 * exp(-V/18) \\ alpha\_n & = 0.01 * (-V + 10) / (exp((-V+10)/10) - 1) \\ beta\_n & = 0.125 * exp(-V/80) \\ alpha\_h & = 0.07 * exp(-V/20) \\ beta\_h & = 1/(exp((-V+30)/10) + 1) \\ O & = spike\_func(V)\end{split}\]

• dt

• g_NA, g_K, g_L

• E_NA, E_K, E_L

• alpha_m1, alpha_m2, alpha_m3

• beta_m1, beta_m2, beta_m3

• alpha_n1, alpha_n2, alpha_n3

• beta_n1, beta_n2, beta_n3

• alpha_h1, alpha_h2, alpha_h3

• beta_1, beta_h2, beta_h3

• Vreset

• m, n, h

• V, v_th

customize

In the following chapter called Custom Neuron Model , we will talke about how to add custom neuron model into SPAIC with more details.

[1]

GLIF model : Teeter, C., Iyer, R., Menon, V., Gouwens, N., Feng, D., Berg, J., … & Mihalas, S. (2018). Generalized leaky integrate-and-fire models classify multiple neuron types. Nature communications, 9(1), 1-15.

[2]

AEIF model : Brette, Romain & Gerstner, Wulfram. (2005). Adaptive Exponential Integrate-And-Fire Model As An Effective Description Of Neuronal Activity. Journal of neurophysiology. 94. 3637-42.` doi:10.1152/jn.00686.2005. <https://doi.org/10.1152/jn.00686.2005>`_

[3]

IZH model : Izhikevich, E. M. (2003). Simple model of spiking neurons. IEEE Transactions on neural networks, 14(6), 1569-1572.

[4]

HH model : Hodgkin, A. L., & Huxley, A. F. (1952). A quantitative description of membrane current and its application to conduction and excitation in nerve. The Journal of physiology, 117(4), 500.

Connection

This chapter will introduce spaic.Connection and how to use different connections in SPAIC . As the most basic component of spiking neuron network, spaic.Connection contains the most important weight information of the model. At the same time, as a brain-inspired platform, SPAIC supports bionic link which means supports feedback connections, synaptic delay or other connections with physiological properties.

Connect Parameters

def __init__(self, pre: Assembly, post: Assembly, name=None,
        link_type=('full', 'sparse_connect', 'conv', '...'), syn_type=['basic'],
        max_delay=0, sparse_with_mask=False, pre_var_name='O', post_var_name='Isyn',
        syn_kwargs=None, **kwargs):

In the initial parameters of connection, we can see that when we construct connections, pre , post and link_type are requisite.

• pre - presynaptic neuron

• post - postsynaptic neuron

• name - name of the connection, make connections easier to distinguish

• link_type - link type, ‘full connection’, ‘sparse connection’ or ‘convolution connection’, etc.

• syn_type - synapse type, it will be further explanation in the synaptic section

• max_delay - the maximum sypatic delay

• sparse_with_mask - whether use mask in sparse connection

• pre_var_name - the signal variable name of presynaptic neuron, default as O , means output spike

• post_var_name - the signal variable name of postsynaptic neuron, default as Isyn , means synaptic current

• syn_kwargs - the custom parameters of synapse, it will be further explanation in the synaptic section

• **kwargs - some typical parameters are included in kwargs .

Despite these initial parameters, there are still some important parameters about weight:

• w_mean - mean value of weight

• w_std - standard deviation of weight

• w_max - maximum value of weight

• w_min - minimum value of weight

• weight - weight

SPAIC will generate weight randomly if users don’t provide weight. w_mean and w_std will be used to generate the weight. SPAIC will clamp the weight if w_min or w_max is offered.

For example, in conn1_example, the connection will generate weight with mean 1 and standard deviation of 5, and clip weights between 0 and 2.

self.conn1_example = spaic.Connection(self.layer1, self.layer2, link_type='full',
                                w_mean=1.0, w_std=5.0, w_min=0.0, w_max=2.0)

Full Connection

Full connection is one of the basic connection type.

self.conn1_full = spaic.Connection(self.layer1, self.layer2, link_type='full')

Important key parameters of full connection:

weight = kwargs.get('weight', None) # weight, if not given, it will generate randomly
self.w_std = kwargs.get('w_std', 0.05) # standard deviation of weight, used to generate weight
self.w_mean = kwargs.get('w_mean', 0.005) # mean value of weight, used to generate weight
self.w_max = kwargs.get('w_max', None) # maximum value of weight
self.w_min = kwargs.get('w_min', None) # minimum value of weight

bias = kwargs.get('bias', None) # If you want to use bias, you can pass in the Initializer object or custom value

One-to-one Connection

There are two kinds of one to one connection in SPAIC, the basic one_to_one and the sparse one_to_one_sparse

self.conn_1to1 = spaic.Connection(self.layer1, self.layer2, link_type='one_to_one')
self.conn_1to1s = spaic.Connection(self.layer1, self.layer2, link_type='one_to_one_sparse')

Important key parameters of one to one connection:

Convolution Connection

Common convolution connection, pooling method can choose avgpool or maxpool in synapse type.

Note

In order to provide better computational support, convolution connections need to be used with convolution synapses.

Main connection parameters in convolution connection:

self.out_channels = kwargs.get('out_channels', None)  # input channel
self.in_channels = kwargs.get('in_channels', None)    # output channel
self.kernel_size = kwargs.get('kernel_size', [3, 3]) # convolution kernel
self.w_std = kwargs.get('w_std', 0.05) # standard deviation of weight, used to generate weight
self.w_mean = kwargs.get('w_mean', 0.05) # mean value of weight, used to generate weight
weight = kwargs.get('weight', None) # weight, if not given, connection will generate randomly

self.stride = kwargs.get('stride', 1)
self.padding = kwargs.get('padding', 0)
self.dilation = kwargs.get('dilation', 1)
self.groups = kwargs.get('groups', 1)
self.upscale = kwargs.get('upscale', None)

bias = kwargs.get('bias', None) # If you want to use bias, you can pass in the Initializer object or custom value

Convolution connection example 1:

Convolution connection example 2:

self.conv2 = spaic.Connection(self.layer1, self.layer2, link_type='conv',
                                syn_type=['conv', 'dropout'], in_channels=128, out_channels=256,
                                kernel_size=(3, 3), stride=args.stride, padding=args.padding,
                                weight=kaiming_uniform(a=math.sqrt(5)),
                                bias=uniform(a=-math.sqrt(1 / 1152), b=math.sqrt(1 / 1152))
                                )
self.conv3 = spaic.Connection(self.layer2, self.layer3, link_type='conv',
                                syn_type=['conv', 'maxpool', 'dropout'], in_channels=256, out_channels=512,
                                kernel_size=(3, 3), stride=args.stride, padding=args.padding,
                                pool_stride=2, pool_padding=0,
                                weight=kaiming_uniform(a=math.sqrt(5)),
                                bias=uniform(a=-math.sqrt(1 / 2304), b=math.sqrt(1 / 2304))
                                )
self.conv4 = spaic.Connection(self.layer3, self.layer4, link_type='conv',
                                syn_type=['conv', 'maxpool', 'dropout'], in_channels=512, out_channels=1024,
                                kernel_size=(3, 3), stride=args.stride, padding=args.padding,
                                pool_stride=2, pool_padding=0,
                                weight=kaiming_uniform(a=math.sqrt(5)),
                                bias=uniform(a=-math.sqrt(1 / 4608), b=math.sqrt(1 / 4608))
                                syn_kwargs={'p': 0.6})

Sparse Connection

Common sparse connection, set the density of connection with parameter density .

Random Connection

Common random connection, set the connection probability with parameter probability .

Synapse

This chapter will introduce synaptic models in SPAIC .

Chemical Synapse

Chemical synapse is a common form of synapse, information transmitted between neurons by synaptic transmitters, which causes some concentration of certain ions to change. In computational neuroscience, we use weight and calculate form to simulate the physiological synapse that excitatory and inhibitory transmitters are simulated with plus or minus weight.

In SPAIC , by default, the synapse use chemical synapse and neurons use ‘Isyn’ as input. So, we call the basic chemical synapse as basic .

self.connection = spaic.Connection(self.layer1, self.layer2, link_type='full',
                                    syn_type=['basic'],
                                    w_std=0.0, w_mean=0.1)

Gap Junction

Gap junction , another common form of synapse. The presynaptic and postsynaptic neurons are so closely that charged ions exchange with each other. The characteristic of gap junction is that they are usually bidirection (i.e. the action of the synapse acts on both the presynaptic neuron and the postsynaptic neuron, bringing the voltage of the two neurons closer together).

The calculate form of gap junction: Igap = w_gap(Vpre - Vpost)

If users want to use gap junction, need to set the synapse_type as electrical .

self.connection = spaic.Connection(self.layer1, self.layer2,
                                          link_type='full', syn_type=['electrical'],
                                          w_std=0.0, w_mean=0.1,
                                          )

All Synapses

In Synapse , we also construct some other synapse, including pooling and flatten.

• Basic_synapse – basic

• conv_synapse – conv combine with convolution connection.

• DirectPass_synapse – directpass , choose this synapse will let output equal to the input, which means the output Isyn will equal to the output value of presynapse neurons.

• Dropout_synapse – dropout

• AvgPool_synapse – avgpool

• MaxPool_synapse – maxpool

• BatchNorm2d_synapse – batchnorm2d

• Flatten – flatten

• First_order_chemical_synapse – 1_order_synapse , first order attenuated synapses in chemical synapses

• Second_order_chemical_synapse – 2_order_synapse , second order attenuated synapses in chemical synapses

• Mix_order_chemical_synapse – mix_order_synapse , mix order attenuated synapses in chemical synapses

• Max pooling – maxpool

• Average pooling – avgpool

• Flatten – flatten

• Dropout – dropout

• Direct pass – directpass , choose this synapse will let output equal to the input, which means the output Isyn will equal to the output value of presynapse neurons.

Learner

This chapter will introduce the learner in SPAIC. Recently, SPAIC supports STCA, STBP, STDP and R-STDP algorithms. STCA and STBP use BPTT by surrogate gradient. STDP is the classical unsupervised algorithms that use synaptic plasticity. R-STDP has a reward on STDP that suitable for reinforcement learning.

Example 1

self.learner = spaic.Learner(trainable=self, algorithm='STCA', alpha=0.5)
self.learner.set_optimizer('Adam', 0.001)
self.learner.set_schedule('StepLR', 0.01)

In the sample code, we use STCA learning algorithm, users need to use trainable to specify the training target. self represent the whole network. If user doesn’t want to train the whole network, can specify the target such as self.layer1 or [self.layer1, self.layer2] . And the last alpha=0.5 is a parameters of STCA learning algorithm. In SPAIC, all the parameters of algorithms should be provided at the end of the function.

In the sample code, we also use Adam optimization algorithm and StepLR learning rate scheduler. SPAIC also has some other optimization algorithms:

‘Adam’, ‘AdamW’, ‘SparseAdam’, ‘Adamx’, ‘ASGD’, ‘LBFGS’, ‘RMSprop’, ‘Rpop’, ‘SGD’,‘Adadelta’, ‘Adagrad’

and learning rate schedulers:

‘LambdaLR’, ‘StepLR’, ‘MultiStepLR’, ‘ExponentialLR’, ‘CosineAnnealingLR’, ‘ReduceLROnPlateau’, ‘CyclicLR’, ‘CosineAnnealingWarmRestarts’

Example 2

#STDP learner
self._learner = spaic.Learner(trainable=self.connection1, algorithm='full_online_STDP', w_norm=3276.8)

In example 2, we use the STDP Algorithm. Users pass in stuff to be trained in trainable , usually a certain layer of the SNN. The final w_norm is the parameter of the STDP Algorithm. The parameters to pass in is determined by certain Algorithm.

Example 3

# global reward
self.reward = spaic.Reward(num=label_num, dec_target=self.layer3, coding_time=run_time,
                             coding_method='global_reward', pop_size=pop_size, dec_sample_step=time_step)
# RSTDP
self._learner = spaic.Learner(trainable=[self.connection3], algorithm='RSTDP',
                         lr=1, A_plus=1e-1, A_minus=-1e-2)

In Example 3, the RSTDP algorithm is used, which is a supervised algorithm.The user needs to use Reward to pass in reward as a penalty or reward signal. Parameters of Reward and RSTDP can be checked in spaic.Neuron.Rewards and spaic.Learning.STDP_Learner respectively. In the training process, different from other algorithms, users need to use Reward to pass the supervision signal and use the optim_step function to synchronize the parameters_dict and variables parameters of the backend, as follows.

Net.input(data)
Net.reward(label)
Net.run(run_time)
output = Net.output.predict
Net._learner.optim_step()

Note

To RSTDPET learning algorithm, the batch_size should be 1.

Encoder & Decoder

This chapter introduces five classes, including Encoder , Generator , Decoder , Reward and Action . Encoder and Generator are used to encode input data into spike trains. Decoder , Reward and Action are used to decode output spike trains to obtain predict label, reward signal and action.

Encoder

The class Encoder is a subclass of the class Node . The Encoder is mainly used to convert the input data into spike trains available in the spiking neural network. For the spiking neural network, the numerical input of the previous artificial neural network does not conform to the physiological characteristics, and the binary spike data is usually used as the input.

In SPAIC , we have built in some common encoding methods:

• SingleSpikeToBinary (‘sstb’)

• MultipleSpikeToBinary (‘mstb’)

• PoissonEncoding (‘poisson’)

• Latency (‘latency’)

• NullEncoder (‘null’)

When instantiating an encoded class, users need to specify the shape or num , coding_method and other related paramters.

SingleSpikeToBinary (‘sstb’)

If the input data is a vector of the firing times of neurons, and one neuron corresponds to one firing time. We can use the SingleSpikeToBinary encoding method to convert the firing time into a binary matrix. The firing time corresponds to the index of time window.

For example, converting firing times [0.9, 0.5, 0.2, 0.7, 0.1] of one input sample into a binary matrix.

self.input = spaic.Encoder(num=node_num, coding_method='sstb')
# or
self.input = spaic.Encoder(shape=[node_num], coding_method='sstb')

Note

The num of node is equal to the size of one input sample, namely 5, and the network runtime is equal to the maximum firing time of dataset plus dt, namely 1.0.

MultipleSpikeToBinary (‘mstb’)

If the input data contains the index of the firing neuron and its firing time, we can use the MultipleSpikeToBinary encoding method to convert the firing index and firing time into a binary matrix.

For example, converting [[[0.9, 0.5, 0.2, 0.7, 0.1], [1, 1, 3, 2, 6]]] into a binary matrix.

self.input = spaic.Encoder(num=node_num, shape=[2, node_num], coding_method='mstb')

Note

• Since one neuron corresponds to zero or multiple spike times, the size of each samples may be different.

• The num of node is equal to the maximum id of neuron plus 1, namely 7.

• The shape of one sample should be [2, node_num], where the first row is the vector of firing times, and the second row is the vector of neuron ids.

• For MultipleSpikeToBinary encoding method, initialization parameters num and shape need to be specified simultaneously.

PoissonEncoding (‘poisson’)

Poisson coding method coding method belongs to rate coding. The stronger the stimulus, the higher the firing rate. Taking the image input as an example, the pixel intensity is first mapped to the instantaneous firing rate of the input neuron. Then at each time step, an uniform random number between 0 and 1 is generated and compared with the instantaneous firing rate. If the random number is less than the instantaneous rate, a spike is generated.

The following code defines a PoissonEncoder object which transforms the input into poisson spike trains.

self.input = spaic.Encoder(num=node_num, coding_method='poisson')

Note

• For full connection , the initialization parameter shape may not be specified.

• For convolution connection , the initialization parameter shape should be specified as [channel, width, height]. In this case, the initialization parameter num may not be specified.

• For PoissonEncoding , sometimes we need to scale the input intensity, which can be done by specifying the unit_conversion parameters:

  unit_conversion - a constant parameter that scales the input rate, default as 1.0

Latency (‘latency’)

The stronger the external stimulus, the earlier the neurons fire. Taking the image input as an example, the larger the gray value in the image, the more important the information is and the earlier the firing time of the neuron.

The following code defines a Latency object which transforms the input into spike trains.

self.input = spaic.Encoder(num=node_num, coding_method='latency')

Note

• For full connection , the initialization parameter shape may not be specified.

• For convolution connection , the initialization parameter shape should be specified as [channel, width, height]. In this case, the initialization parameter num may not be specified.

NullEncoder (‘null’)

If no encoding method is required, we can use NullEncoder .

The following code defines a NullEncoder object.

self.input = spaic.Encoder(num=node_num, coding_method='null')

Note

• For full connection , the initialization parameter shape may not be specified.

• For convolution connection , the initialization parameter shape should be specified as [channel, width, height]. In this case, the initialization parameter num may not be specified.

• For full connection , the shape of external input should be [batch_size, time_step, node_num].

• For convolution connection , the shape of external input should be [batch_size, time_step, channel, width, height].

Generator

The Generator class is a subclass of the Node class. It is a special encoder that will generate spike trains or current without dataset. For example, in some computational neuroscience studies, users need special input like poisson spikes to model background cortical activities.

To meet requirements, some common pattern generators are provided in SPAIC .

• Poisson_Generator (‘poisson_generator’) – generate poisson spike trains according input rate

• CC_Generator (‘cc_generator’) – generate constant current input

When instantiating an encoded class, users need to specify the shape or num , coding_method and other related paramters.

Poisson_Generator (‘poisson_generator’)

Poisson_Generator method generate spike trains according to input rate. At each time step, an uniform random number between 0 and 1 is generated and compared with input rate. If the random number is less than input rate, a spike is generated.

The following code defines a Poisson_Generator object which transforms the input rate into poisson spike trains.

self.input = spaic.Generator(num=node_num, coding_method='poisson_generator')

Note

• For full connection , the initialization parameter shape may not be specified.

• For convolution connection , the initialization parameter shape should be specified as [channel, width, height].

• In this case, the initialization parameter num may not be specified.

• If external input is a constant value, the input rate is the same for all nodes by default.

• If each node needs a different input rate, you should pass in an input matrix corresponding to the shape of the node.

• Sometimes we need to scale the input rate, which can be done by specifying the unit_conversion parameters:

  unit_conversion - a constant parameter that scales the input rate, default as 1.0.

CC_Generator (‘cc_generator’)

CC_Generator can generate constant current input, which is helpful for users to observe and simulate various neuronal dynamics. The CC_Generator is used similarly to Poisson_Generator , with coding_method=’cc_generator’ .

The following code defines a CC_Generator object which transforms the input rate into spike trains.

self.input = spaic.Generator(num=node_num, coding_method='cc_generator')

Note

CC_Generator’s precautions are similar to Poisson_Generator’s.

Decoder

The Decoder class is a subclass of the Node class. The main usage of Decoder is to convert the output spikes or voltages to a numerical signal. In SPAIC , we have built in some common decoding methods:

• Spike_Counts (‘spike_counts’) – get the mean spike count of each neuron in the target layer.

• First_Spike (‘first_spike’) – get the first firing time of each neuron in the target layer.

• Final_Step_Voltage (‘final_step_voltage’) – get the final step voltage of each neuron in the target layer.

• Voltage_Sum (‘voltage_sum’) – get the voltage sum of each neuron in the target layer.

The Decoder class is mainly used in the output layer of the network. When instantiating an decoded class, users need to specify the num , dec_target , coding_method and related parameters.

For example, when decoding the spiking activity of a NeuronGroup object with 10 LIF neurons, we can create an instance of the Spike_Counts class:

self.target = spaic.NeuronGroup(num=10, model='lif')
self.output = spaic.Decoder(num=10, dec_target=self.target, coding_method='spike_counts')

Note

• The value of parameter dec_target is the layer to be decoded.

• The value of parameter num in Decoder class should be the same as the value of num in the target layer.

• If you want to instantiate other decoding classes, simply assign str name of corresponding class to coding_method parameter.

• The value of parameter coding_var_name is the variable to be decoded, such as ‘O’ or ‘V’. ‘O’ represents spike and ‘V’ represents voltage.

• For Spike_Counts and First_Spike , the default value of parameter coding_var_name is ‘O’.

• For Final_Step_Voltage and Voltage_Sum , the default value of parameter coding_var_name is ‘V’.

For Spike_Counts, we can specify pop_size parameter,

• pop_size - population size of decoded neurons, default as 1 (each category is represented by one neuron)

Reward

The Reward class is a subclass of the Node class. It can be seen as a different type of decoder. During the execution of a reinforcement learning task, Reward is needed to decode the activity of the target object according to the task purpose.

In SPAIC , we have built in some reward methods:

• Global_Reward (‘global_reward’) – get a global reward. For the classification task, the predict label is determined according to the number of spikes or the maximum membrane potential. If the predict label is the same as the expected one, the positive reward will be returned. On the contrary, negative rewards will be returned.

• XOR_Reward (‘xor_reward’) – get reward for xor task. When the expected result is 1, if the number of output spikes is greater than 0, a positive reward will be obtained. When the expected result is 0, if the number of output pulses is greater than 0, the penalty is obtained

• DA_Reward (‘da_reward’) – get rewards in the same dimension as neurons in the dec_target

• Environment_Reward (‘environment_reward’) – get reward from RL environment

The Reward class is mainly used in the output layer of the network. When instantiating an reward class, users need to specify the num , dec_target , coding_method and other related parameters.

For example, when decoding the spiking activity of a NeuronGroup object with 10 LIF neurons to obtain a global reward, we can create an instance of the Global_Reward class as follows:

self.target = spaic.NeuronGroup(num=10, model='lif')
self.reward = spaic.Reward(num=10, dec_target=self.target, coding_method='global_reward')

Note

• The value of parameter dec_target is the layer to be decoded.

• The value of parameter num in Reward class should be the same as the value of num in the target layer.

• If you want to instantiate other reward classes, simply assign str name of corresponding class to coding_method parameter.

• The value of parameter coding_var_name is the variable to be decoded, such as ‘O’ or ‘V’. ‘O’ represents spike and ‘V’ represents voltage.

• The default value is ‘O’.

For Global_Reward , XOR_Reward and DA_reward , we can specify some parameters:

• pop_size - population size of decoded neurons, default as 1 (each category is represented by one neuron)

• dec_sample_step - decoding sampling time step, default as 1 (get reward each time step)

• reward_signal - reward, default as 1.0

• punish_signal - punish, default as -1.0

Action

The Action class is a subclass of the Node class.It is also a special decoder that will transform the output to an action. The main usage of Action is to choose the next action according to the action selection mechanism of the target object during reinforcement learning tasks.

In SPAIC , we have built in some action methods:

• Softmax_Action (‘softmax_action’) – action sampled from softmax over spiking activity of target layer.

• PopulationRate_Action (‘pop_rate_action’) – take the label of the neuron group with largest spiking frequency as action.

• Highest_Spikes_Action (‘highest_spikes_action’) – action sampled from highest activities of target layer.

• Highest_Voltage_Action (‘highest_voltage_action’) – action sampled from highest voltage of target layer.

• First_Spike_Action (‘first_spike_action’) – action sampled from first spike of target layer.

• Random_Action (‘random_action’) – action sampled from action space randomly.

The Action class is mainly used in the output layer of the network. When instantiating an action class, users need to specify the num , dec_target , coding_method and other related paramters.

For example, when decoding the spiking activity of a NeuronGroup object with 10 LIF neurons to obtain next action, we can create an instance of the Softmax_Action class as follows:

self.target = spaic.NeuronGroup(num=10, model='lif')
self.reward = spaic.Action(num=10, dec_target=self.target, coding_method='softmax_action')

Note

• The value of parameter dec_target is the layer to be decoded.

• The value of parameter num in Action class should be the same as the value of num in the target layer.

• If you want to instantiate other action classes, simply assign str name of corresponding class to coding_method parameter.

• The value of parameter coding_var_name is the variable to be decoded, such as ‘O’ or ‘V’. ‘O’ represents spike and ‘V’ represents voltage.

For PopulationRate_Action, we can specify pop_size parameters:

• pop_size - population size of decoded neurons, default as 1 (each category is represented by one neuron)

Network

This section mainly introduces the methods for building and running networks based on the Network class in the SPAIC platform.

Network Construction

Model construction can use three methods: The first is similar to Pytorch’s module class inheritance, in the form of building in the _init_ function; the second is similar to Nengo’s method of construction using the with statement; the third way, new network modules can also be added to the existing network during the modeling process through the function interfaces.

Model Construction Method 1: Class Inheritance Form

class SampleNet(spaic.Network):
   def __init__(self):
      super(SampleNet, self).__init__()

      self.layer1 = spaic.NeuronGroup(100, neuron_model='clif')
      ......

Net = SampleNet()

Model Construction Method 2: with Form

Net = SampleNet()
with Net:

   layer1 = spaic.NeuronGroup(100, neuron_model='clif')
   ......

Model Construction Method 3: Build or Modify Network Through Function Interface

Net = SampleNet()
layer1 = spaic.NeuronGroup(100, neuron_model='clif')
....

Net.add_assembly('layer1', layer1)

The current Network provides function interfaces for constructing or modifying a network, including:

• add_assembly(name, assembly) - Add a neural assembly class (including Node, NeuronGroup, etc.), where the parameter name represents the variable name in the network, and assembly represents the neural assembly object to be added.

• copy_assembly(name, assembly) - Copy a neural assembly class (including Node, NeuronGroup, etc.). Unlike add_assembly, this interface clones the assembly object before adding it to the network.

• add_connection(name, connection) - Add a connection object, where the parameter name represents the variable name in the network, and connection represents the connection object to be added.

• add_projection(name, projection) - Add a topology projection object, where the parameter name represents the variable name in the network, and projection represents the topology mapping object to be added.

• add_learner(name, learner) - Add a learning algorithm, where the parameter name represents the variable name in the network, and learner represents the learning algorithm object to be added.

• add_moitor(name, moitor) - Add a monitor, where the parameter name represents the variable name in the network, and moitor represents the monitor object to be added.

Network Execution and Execution Parameter Settings

The Network object provides function interfaces for running and setting execution parameters, including:

• run(backend_time) - The function interface for running the network, where the backend_time parameter is the network runtime.

• run_continue(backend_time) - The function interface for continuing to run the network. Unlike run, run_continue does not reset the initial values of variables but continues to run based on the original initial values.

• set_backend(backend, device, partition) - Set the network runtime backend, where the parameters backend represents the backend object or backend name, device is the hardware used for backend computation, and partition represents whether to distribute the model across different devices.

• set_backend_dt(backend, dt) - Set the network runtime timestep, where dt is the timestep.

• set_random_seed(seed) - Set the network runtime random seed.

Input/Output

Dataloader

Dataloader is the interface of loading dataset, it is used to encapsulate the custom Dataset into an array according to the size of batch_size and whether it is shuffle, etc, for network training.

Dataloader is consists of dataset and sampler, and the initialization parameters are as follows:

• dataset(Dataset) – the dataset to be loaded

• batch_size(int, optional) – the number of samples in each batch, the default is 1

• shuffle(bool, optional) – whether reorder the data at the beginning of each epoch, the default is False

• sampler(Sampler, optional) – customize the strategy for taking samples from the dataset

• batch_sampler(Sampler, optional) – Similar to sampler, but only returns the index of one batch

• collate_fn(callable, optional) – A function that composes a list of samples into a mini-batch

• drop_last(bool, optional) – If set to True, for the last batch, if the number of samples is less than batch_size, it will be thrown away. For example, if batch_size is set to 64, and the dataset has only 100 samples, then the last 36 samples will be trained during training. will be thrown away. If False (default), normal execution will continue, but the final batch_size will be smaller.

Loading MNIST dataset as example：

root = './Datasets/MNIST' # root of data
train_set = dataset(root, is_train=True)   # Train set
test_set = dataset(root, is_train=False)   # Test set
bat_size = 20
# Create DataLoader
train_loader = spaic.Dataloader(train_set, batch_size=bat_size, shuffle=True)
test_loader = spaic.Dataloader(test_set, batch_size=bat_size, shuffle=False)

Note

To be mentioned:

1. If sampler has been specified when creating Dataloader, the shuffle must be False.

2. If batch_sampler has been specified, then, batch_size, shuffle, sampler and drop_last can no longer be specified.

Backend

Backend is the core component of the backend of the SPAIC platform and is responsible for the overall running simulation of the network. dt and runtime are the two most important parameters in backend, representing the time step and the length of the time window of the simulation, respectively. And time represents the moment that currently simulated, and n_time_step represents the time step that is currently simulated.

The functions available to users in Backend are:

• set_runtime – sets the length of the time window, or the length of the simulation

• add_variable – adds variables to the backend. When customizing the algorithm, you need to add new variables to the Backend

• add_operation – adds a new calculation formula to the backend for custom algorithms, neurons and other operations, and needs to follow a certain format

• register_standalone – registers independent operations, mainly used to add some operations that are not supported by the platform to the backend

• register_initial – registers the initialization operation. The calculation in the initialization operation will be calculated once at the beginning of each time window, instead of every dt as the time step runs.

add_variable

When using add_variable , the parameters that must be added are name 与 shape , and the optional parameters are value , is_parameter , is_sparse , init,

min and max .

name decodes the key when save on the backend，and shape decides the shape, value represent the value，init decides the value that everytime the variable initialized, is_parameter decides whether this variable is trainable or not.

add_operation

When using add_operation , the parameters that must be added is op . op is used to load operations that backend need to calculate. It will put the operation given by users into the computing graph. It is mainly used for custom computation.

save or load model

This section will describe two ways of saving network information in detail.

pre-defined function in Network

Use pre-defined functions save_state and state_from_dict of spaic.Network to save or load the weight of the model directly.

The optional parameters are filename , direct and save. If users use save_state without giving any parameters, the function will use default name autoname with random number as the direct name and save the weight into the './autoname/parameters/_parameters_dict.pt' . If given filename, or direct , it will save the weight into 'direct/filename/parameters/_parameters_dict.pt' . Parameter save is default as True , which means it will save the weight. If users choose False , this function will return the parameter_dict of the model directly.

The parameters of state_from_dict is same as save_state but have two more parameters: state and direct ,and save parameters is unneeded. If users provide state , this function will use given parameters to replace the parameter dict of the backend. If state is None, this function will decide the saving path according to filename and direct. The device will decide where to storage the parameters.

Net.save_state('Test1', True)
...
Net.state_from_dict(filename='Test1', device=device)

network_save and network_load

The network save module spaic.Network_saver.network_save and spaic.Network_loader.network_load in spaic.Library will save the whole network structure of the model and the weight information separately. This method requires a filename filename when used, and then the platform will create a new file ./filename/filename.json in the running directory of the current program to save the network structure. At the same time, when using network_save , users also can choose the save format between json or yaml .

network_dir = network_save(Net=Net, filename='TestNet',
                                        trans_format='json', combine=False, save=True)

# network_dir = 'TestNet'
Net2 = network_load(network_dir, device=device)

In network_save :

• Net – the specific network object in SPAIC

• filename – filename, network_save will save the Net with this name

• path – file storage path, a new folder will be created based on the filename if target path doesn’t have such folder

• trans_format – save format, can choose json or yaml , default as json

• combine – this parameters decides whether save the weight and network structure in one file, default as False

• save – this parameters decides whether save the structure locally, if choose True , this function will save locally and return the file name. If choose False , it will only return the structure as a dict.

• save_weight – this parameters decides whether save the backend information and weights of the model

During the process of storing the parameters of parts of the network, if the parameters of the neurons are passed in as Tensor, the names of these parameters are stored in the storage file and the actual parameters are stored in the diff_para_dict.pt file in the same directory as the weights.

Then, I will give some example to explain the meaning of saved file:

# information about Nodes
-   input:
        _class_label: <nod> # Indicate this object is node
        _dt: 0.1 # Length of every time step
        _time: null #
        coding_method: poisson # Encode method
        coding_var_name: O # Output target of this node
        dec_target: null # Decode target of this node, since this is input node, it doesn't have decode target
        name: input # name of this node
        num: 784 # element number of this node
        shape: # shape
        - 784

# information about NeuronGroups
-   layer1:
        _class_label: <neg> # Indicate this object is NeuronGroup
        id: autoname1<net>_layer1<neg> # ID of this NeuronGroup, it is NeuronGroup 'layer1' of the network 'autoname1'
        model_name: clif # neuron model of this NeuronGroup, it's CLIF
        name: layer1 # name of this NeuronGroup
        num: 10 # neuron number of this NeuronGroup
        parameters: {} # parameters of kwargs, like some parameters of neuron model
        shape: # shape
        - 10
        type: null # type of this NeuronGroup, it is just like a label for Projection

-   layer3:
    -   layer1:
            _class_label: <neg> # Indicate this object is NeuronGroup
            id: autoname1<net>_layer3<asb>_layer1<neg>  # ID of this NeuronGroup，it is NeuronGroup 'layer1' of the Assembly 'layer3' of the network 'autoname1'
            model_name: clif # neuron model of this NeuronGroup, it's CLIF
            name: layer1 # name of this NeuronGroup
            num: 10 # neuron number of this NeuronGroup
            parameters: {} # parameters of kwargs, like some parameters of neuron model
            shape: # shape
            - 10
            type: null # type of this NeuronGroup, it is just like a label for Projection

    -   connection0:
            _class_label: <con> # Indicate this object is Connection
            link_type: full # link type of this Connection, it is full connection
            max_delay: 0 # the maximum delay step of this Connection
            name: connection0 # name of this Connection
            parameters: {} # parameters of kwargs, like some parameters of convolution connection
            post: layer3   # postsynaptic neuron, here is point to Assembly layer3
            post_var_name: Isyn   # the output of this synapse, here is 'Isyn', a default value
            pre: layer2    # presynaptic neuron, here is point to layer2
            pre_var_name: O         # input of this synapse, here is 'O', a default value
            sparse_with_mask: false # whether use mask, details will be explained in chapter 'Basic Structure.Connection'
            weight: # weight matrix
                autoname1<net>_layer3<asb>_connection0<con>:autoname1<net>_layer3<asb>_layer3<neg><-autoname1<net>_layer3<asb>_layer2<neg>:{weight}: # here is the ID of this weight
                -   - 0.05063159018754959

# information about Connections
-   connection1:
        _class_label: <con> # Indicate this object is Connection
        link_type: full # link type of this Connection, it is full connection
        max_delay: 0 # the maximum delay step of this Connection
        name: connection1 # name of this Connection
        parameters:  # parameters of kwargs, like some parameters of convolution connection, here is the parameter for randomly initializing the weight
            w_mean: 0.02
            w_std: 0.05
        post: layer1   # postsynaptic neuron, here is point to layer1
        post_var_name: Isyn   # the output of this synapse, here is 'Isyn', a default value
        pre: input     # presynaptic neuron, here is point to input node
        pre_var_name: O         # input of this synapse, here is 'O', a default value
        sparse_with_mask: false # whether use mask, details will be explained in chapter 'Basic Structure.Connection'
        weight: # weight matrix
            autoname1<net>_connection1<con>:autoname1<net>_layer1<neg><-autoname1<net>_input<nod>:{weight}:
            -   - 0.05063159018754959
                ......

# information about Learners
-   learner2:
        _class_label: <learner> # Indicate this object is Learner
        algorithm: full_online_STDP # the algorithms of this Learner, here is full_online_STDP
        lr_schedule_name: null # the learning rate scheduler of this Learner, here is unused
        name: _learner2 # name of this Learner
        optim_name: null # the optimizer of this Learner, here is unused
        parameters: {} # parameters of kwargs
        trainable: # the training target of this Learner
        - connection1
        - connection2

Custom

Although the platform already contains various kinds of models, and the commonly used codec, neuron and connection forms have been realized, we often have more and different requirements in the process of research. In this chapter, we will introduce how to add various types of modules on the SPAIC platform.

Custom encoding or decoding

This chapter will introduce how to customize Encoder , Generator , Decoder , Action and Reward .

Customize Encoder

Encoder is used to transmit the input data to temporal spiking data. It is one of the important step in building spiking neural network. Different encoding method will generate different data. To meet most of the application situation, SPAIC has already provided some common encoding methods. And customize encoding method can add as the format in spaic.Neuron.Encoders .

Initialize Encoder

The user-defined encoding method should inherit the class Encoder , and the parameter name in the initialization method should be the same as that of the class Encoder . Other parameters can be passed in by kwargs . Take PoissonEncoding class initialization function as an example:

def __init__(self, shape=None, num=None, dec_target=None, dt=None, coding_method='poisson',
         coding_var_name='O', node_type=('excitatory', 'inhibitory', 'pyramidal', '...'), **kwargs):
    super(PoissonEncoding, self).__init__(shape, num, dec_target, dt, coding_method, coding_var_name, node_type,
                                      **kwargs)
    self.unit_conversion = kwargs.get('unit_conversion', 1.0)

In this initialization method, unit_conversion is the required parameter for the PoissonEncoding class, which can get from kwargs .

Define Encoder Function

The encoding function is the implementation part of the encoding method. Because the platform supports multiple backends ( Pytorch , TensorFlow etc.), different backends support different data types and data operations. Therefore, the corresponding coding function needs to be implemented in the front-end coding method for different computing back-end. We take the implementation of torch_coding for PoissonEncoding as an example:

def torch_coding(self, source, device):
    # Source is raw real value data.
    # For full connection, the shape of source is [batch_size, num]
    # For convolution connection, the shape of source is [batch_size] + shape
    if source.__class__.__name__ == 'ndarray':
        source = torch.tensor(source, device=device, dtype=self._backend.data_type)
    # The shape of the encoded spike trains.
    spk_shape = [self.time_step] + list(self.shape)
    spikes = torch.rand(spk_shape, device=device).le(source * self.unit_conversion*self.dt).float()
    return spikes

At the end of this code, don’t forget add Encoder.register("poisson", PoissonEncoding) to add the usage linked to this function.

Customize Generator

Generator can be used to generate specific distributed spike trains or some special current mode. SPAIC has already provided some common generating methods. And customize generating method can add as the format in spaic.Neuron.Generators file.

Initialize Generator

The user-defined generating method should inherit the class Generator , and the parameter name in the initialization method should be the same as that of the class Generator . Other parameters can be passed in by kwargs . Take CC_Generator class initialization function as an example:

def __init__(self, shape=None, num=None, dec_target=None,  dt=None,
             coding_method='cc_generator', coding_var_name='O', node_type=('excitatory', 'inhibitory', 'pyramidal', '...'), **kwargs):
    super(CC_Generator, self).__init__(shape, num, dec_target, dt, coding_method, coding_var_name, node_type,
                                   **kwargs)

Define Generator Function

A coding function is the implementation part of a generating method. Because the platform supports multiple backends ( Pytorch , TensorFlow etc.), different backends support different data types and data operations. Therefore, the corresponding coding function needs to be implemented in the front-end coding method for different computing back-end. We take the implementation of torch_coding for CC_Generator as an example:

def torch_coding(self, source, device):

  if not (source >= 0).all():
      import warnings
      warnings.warn('Input current shall be non-negative')
  if source.__class__.__name__ == 'ndarray':
      source = torch.tensor(source, dtype=self._backend.data_type, device=device)

  spk_shape = [self.time_step] + list(self.shape)
  spikes = source * torch.ones(spk_shape, device=device)
  return spikes

Generator.register('cc_generator', CC_Generator) also needed here for front-end use.

Customize Decoder

Decoder is used to convert the output spikes or voltages to a numerical signal. SPAIC has already provided some common decoding methods. And decoding method can add as the format in spaic.Neuron.Decoders file.

Initialize Decoder

The user-defined decoding method should inherit the class Decoder, and the parameter name in the initialization method should be the same as that of the class Decoder. Other parameters can be passed in by kwargs . Take Spike_Counts class initialization function as an example:

def __init__(self, shape=None, num=None, dec_target=None, dt=None, coding_method='spike_counts',
        coding_var_name='O', node_type=('excitatory', 'inhibitory', 'pyramidal', '...'), **kwargs):
    super(Spike_Counts, self).__init__(shape, num, dec_target, dt, coding_method, coding_var_name, node_type,
                                  **kwargs)
    self.pop_size = kwargs.get('pop_size', 1)

In this initialization method, pop_size is the required parameter for the Spike_Counts class, which can get from kwargs .

Define Decoder Function

A coding function is the implementation part of a decoding method. Because the platform supports multiple backends ( Pytorch , TensorFlow etc.), different backends support different data types and data operations. Therefore, the corresponding coding function needs to be implemented in the front-end coding method for different computing back-end. We take the implementation of torch_coding for Spike_Counts as an example:

def torch_coding(self, record, target, device):
    # record is the activity of the NeuronGroup to be decoded
    # the shape of record is (time_step, batch_size, n_neurons)
    # target is the label of the sample
    spike_rate = record.sum(0).to(device=device)
    pop_num = int(self.num / self.pop_size)
    pop_spikes_temp = (
        [
            spike_rate[:, (i * self.pop_size): (i * self.pop_size) + self.pop_size].sum(dim=1)
            for i in range(pop_num)
        ]
    )
    pop_spikes = torch.stack(pop_spikes_temp, dim=-1)
    return pop_spikes

Decoder.register('spike_counts', Spike_Counts) also needed here for front-end use.

Customize Reward

Reward is used to convert the activity of the target object into reward signal. SPAIC has already provided some common reward methods. And reward method can add as the format in spaic.Neuron.Rewards file.

Initialize Reward

The user-defined reward method should inherit the class Reward , and the parameter name in the initialization method should be the same as that of the class Reward . Other parameters can be passed in by kwargs . Take Global_Reward class initialization function as an example:

def __init__(self,shape=None, num=None, dec_target=None, dt=None, coding_method='global_reward', coding_var_name='O', node_type=('excitatory', 'inhibitory', 'pyramidal', '...'), **kwargs):
    super(Global_Reward, self).__init__(shape, num, dec_target, dt, coding_method, coding_var_name, node_type, **kwargs)
    self.pop_size = kwargs.get('pop_size', 1)
    self.reward_signal = kwargs.get('reward_signal', 1)
    self.punish_signal = kwargs.get('punish_signal', -1)

In this initialization method, pop_size , reward_signal , punish_signal are required parameters for the Global_Reward class, which can get from kwargs .

Define Reward Function

A coding function is the implementation part of a reward method. Because the platform supports multiple backends ( Pytorch , TensorFlow etc.), different backends support different data types and data operations. Therefore, the corresponding coding function needs to be implemented in the front-end coding method for different computing back-end. We take the implementation of torch_coding for Global_Reward as an example:

def torch_coding(self, record, target, device):
    # the shape of record is (time_step, batch_size, n_neurons)
    spike_rate = record.sum(0)
    pop_num = int(self.num / self.pop_size)
    pop_spikes_temp = (
        [
            spike_rate[:, (i * self.pop_size): (i * self.pop_size) + self.pop_size].sum(dim=1)
            for i in range(pop_num)
        ]
    )
    pop_spikes = torch.stack(pop_spikes_temp, dim=-1)
    predict = torch.argmax(pop_spikes, dim=1)  # return the indices of the maximum values of a tensor across columns.
    reward = self.punish_signal * torch.ones(predict.shape, device=device)
    flag = torch.tensor([predict[i] == target[i] for i in range(predict.size(0))])
    reward[flag] = self.reward_signal
    if len(reward) > 1:
        reward = reward.mean()
    return reward

Reward.register('global_reward', Global_Reward) also needed here for front-end use.

Customize Action

Action is used to convert the activity of the target object into next action. SPAIC has already provided some common action methods. And action method can add as the format in spaic.Neuron.Actions file.

Initialize Action

The user-defined action method should inherit the class Action , and the parameter name in the initialization method should be the same as that of the class Action . Other parameters can be passed in by kwargs . Take Softmax_Action class initialization function as an example:

def __init__(self, shape=None, num=None, dec_target=None, dt=None, coding_method='softmax_action', coding_var_name='O', node_type=('excitatory', 'inhibitory', 'pyramidal', '...'), **kwargs):
    super(Softmax_Action, self).__init__(shape, num, dec_target, dt, coding_method, coding_var_name, node_type, **kwargs)

Define Action Function

A coding function is the implementation part of a action method. Because the platform supports multiple backends ( Pytorch , TensorFlow etc.), different backends support different data types and data operations. Therefore, the corresponding coding function needs to be implemented in the front-end coding method for different computing back-end. We take the implementation of torch_coding for Softmax_Action as an example:

def torch_coding(self, record, target, device):
    # the shape of record is (time_step, batch_size, n_neurons)
    assert (
        record.shape[2] == self.num
    ), "Output layer size is not equal to the size of the action space."
    spikes = torch.sum(record, dim=0)
    probabilities = torch.softmax(spikes, dim=0)
    return torch.multinomial(probabilities, num_samples=1).item()

Action.register('softmax_action', Softmax_Action) also needed here for front-end use.

Custom Neuron Model

Neuron model is the most important part in neural dynamics simulation. Different models and parameters will produce different phenomena. SPAIC includes many common neuron models in order to meet the needs of different applications. However, SPAIC is sometimes out of reach and users need to add their own personalized neurons that are more appropriate for their experiments. The neuron definition step can follow the the format in spaic.Neuron .

Define Variables

At first, we need to introduce some typical variable type in SPAIC :

• _variables – normal variable

• _tau_variables – exponential decay time constant

• _membrane_variables – decay time constant

• _parameter_variables – parameters

• _constant_variables – constant

To _tau_variables , we will transmit it as tau_var = np.exp(-dt/tau_var) . To _membrane_variables , we will transmit it as membrane_tau_var = dt/membrane_tau_var ,

When defining variables, initial value also should be given, since all the neuron parameters will be reset to the initial value after each model run. Some parameters can be change based on the parameters received. We use lif neuron model as the example:

"""
LIF model:
# V(t) = tuaM * V^n[t-1] + Isyn[t]   # tauM: constant membrane time (tauM=RmCm)
O^n[t] = spike_func(V^n[t-1])
"""

In the formula of lif model, the original formula should be transmitted to the differential equation by users themselves. Then, tauM and the threshold v_th are changeable, so we get parameters from kwargs:

# The complete add code
self._variables['V'] = 0.0
self._variables['O'] = 0.0
self._variables['Isyn'] = 0.0

self._parameter_variables['Vth'] = kwargs.get('v_th', 1)
self._constant_variables['Vreset'] = kwargs.get('v_reset', 0.0)

self._tau_variables['tauM'] = kwargs.get('tau_m', 20.0)

Define Calculation

Compute operation is the most important part of Neuron Model. These operations decide the change of elements during simulation. When add compute operations, there are some rules to follow. At first, every operation can only do one compute process, so users need to decomposition formula to independent operations. The whole build-in calculate operator can be found in spaic.backend.backend , and here is the example about LIF model:

# Recently, [updated] represent that here it needs the updated value instead of the old value from last round of
# calculation. Temporary variable don't need [updated].
# Vtemp = V * tauM + I, to be mentioned, tauM is a '_tau_variables' variable, which means it was not the initial value.
self._operations.append(('Vtemp', 'var_linear', 'tauM', 'V', 'Isyn[updated]'))

# O = 1 if Vtemp >= Vth else 0， 'threshold' used to check whether 'Vtemp' reaches the threshold 'Vth'
self._operations.append(('O', 'threshold', 'Vtemp', 'Vth'))

# Used to reset voltage after spike is sent
self._operations.append(('V', 'reset', 'Vtemp',  'O[updated]'))

Also, we need to add NeuronModel.register("lif", LIFModel) to combine the name with the model for front-end use.

Custom synapse or connection model

This chapter will introduce how to customize connections and synapse.

Customize connection

Connection is the basic structure of neuron network, it contains weight information. Different connection way will generate different spatially structure. To meet users’ requirements, SPAIC has constructed many common connection methods. If users want to add some personalize connection, can follow the document or the format in spaic.Network.Connection.

Initialize connection method

Custom connection method need to inherit Connection class and modify the corresponding parameters. Use FullConnection as example:

def __init__(self, pre, post, name=None, link_type=('full', 'sparse_connect', 'conv','...'),
             syn_type=['basic_synapse'], max_delay=0, sparse_with_mask=False, pre_var_name='O', post_var_name='Isyn',
             syn_kwargs=None, **kwargs):
    super(FullConnection, self).__init__(pre=pre, post=post, name=name,
                                         link_type=link_type, syn_type=syn_type, max_delay=max_delay,
                                         sparse_with_mask=sparse_with_mask,
                                         pre_var_name=pre_var_name, post_var_name=post_var_name, syn_kwargs=syn_kwargs, **kwargs)
    self.weight = kwargs.get('weight', None)
    self.w_std = kwargs.get('w_std', 0.05)
    self.w_mean = kwargs.get('w_mean', 0.005)
    self.w_max = kwargs.get('w_max', None)
    self.w_min = kwargs.get('w_min', None)

    self.is_parameter = kwargs.get('is_parameter', True) # is_parameter以及is_sparse为后端使用的参数，用于确认该连接是否为可训练的以及是否为稀疏化存储的
    self.is_sparse = kwargs.get('is_sparse', False)

In this initial way, the extra parameters should get from kwargs .

Customize synapse model

To meet users requirements, SPAIC has constructed some common synapse model. But if users want to add some personalized model, they need to define synapse model as the format of Network.Synapse .

Define parameters that can be obtained externally

In the initial part of defining the neuron model, we need to define some parameters that the neuron model can change, which can be changed by passing parameters. For example, in the first-order decay model of chemical synapses, the original formula can be obtained after transformation:

class First_order_chemical_synapse(SynapseModel):
    """
    .. math:: Isyn(t) = weight * e^{-t/tau}
    """

In this formula, self.tau is changeable, so we can change it by kwargs .

self._syn_tau_variables['tau[link]'] = kwargs.get('tau', 5.0)

Define variables

In the variable definition stage, we need to understand several variable forms of synapses:

• _syn_tau_constant_variables – Exponential decay constant

• _syn_variables – Normal variable

To _syn_tau_constant_variables , we will transmit it as value = np.exp(-self.dt / var) ,

When defining variables, initial values need to be set at the same time. After each run of the network, the parameters of neurons will be reset to the initial values set at this point.

self._syn_variables[I] = 0
self._syn_variables[WgtSum] = 0
self._syn_tau_constant_variables[tauP] = self.tau_p

Define calculation operation

The calculation operation is the most important part of the synaptic model. The calculation operation determines how the parameters will undergo some changes during the simulation.

There are a few rules to follow when adding computations. First, each row can only evaluate one specific operator, so you need to decompose the original formula into independent operators. The current built-in operator in the platform can be found in backend.basic_operation :

• add, minus, div

• var_mult, mat_mult, mat_mult_pre, sparse_mat_mult, reshape_mat_mult

• var_linear, mat_linear

• reduce_sum, mult_sum

• threshold

• cat

• exp

• stack

• conv_2d, conv_max_pool2d

Use the process of computing chemical current in chemical synapse as an example:

# Isyn = O * weight
# The first is the result, conn.post_var_name
# Compute operator `mat_mult_weight` at the second index
# The third is the factor of the calculation, input_name and weight[link]
# '[updated]' means the updated value of current calculation, temporary variables don't need
self._syn_operations.append(
    [conn.post_var_name + '[post]', 'mat_mult_weight', self.input_name,
     'weight[link]'])

Custom algorithm

Surrogate Gradient Algorithms

During backpropagation of a spiking neural network, the gradient of the spiking activation function \(output\_spike = sign(V>V_th)\) is:

\[\begin{split}\begin{array}{c} \delta (v) = \left\{\begin{matrix} +\infty, & v = v_{th} \\ 0, & v\ne v_{th} \end{matrix}\right. \end{array}\end{split}\]

Obviously, directly using the impulse activation function for gradient descent will make the training of the network extremely unstable, so we use the gradient surrogate algorithm to approximate the impulse activation function.The most import part of surrogate gradient algorithms is that use custom gradient function to replace the original backpropagation gradient. Here we use STCA and STBP as examples to show how to use custom gradient formula.

In STCA [1] learning algorithm, the graident function is:

\(h(V)=\frac{1}{\alpha}sign(|V-\theta|<\alpha)\)

In STBP [2] learning algorithm, the graident function is:

\(h_4(V)=\frac{1}{\sqrt{2\pi a_4}} e^{-\frac{(V-V_th)^2)}{2a_4}}\)

The comparison between the gradient surrogate function of the two gradient surrogate algorithms and the original activation function is shown in the figure：

The following code block shows the forward and backpropagation process using the gradient surrogate algorithms.

@staticmethod
def forward(
        ctx,
        input,
        thresh,
        alpha
):
    ctx.thresh = thresh
    ctx.alpha = alpha
    ctx.save_for_backward(input)
    output = input.gt(thresh).float()
    return output

@staticmethod
def backward(
        ctx,
        grad_output
    ):
    input, = ctx.saved_tensors
    grad_input = grad_output.clone()
    temp = abs(input - ctx.thresh) < ctx.alpha  # According to STCA learning algorithm
    # temp = torch.exp(-(input - ctx.thresh) ** 2 / (2 * ctx.alpha)) \  # According to STBP learning algorithm
    #                  / (2 * math.pi * ctx.alpha)
    result = grad_input * temp.float()
    return result, None, None

Synaptic Plasticity Algorithms

Hebbian Rule shows in the theory about synapse formation between neurons that the firing activity of a pair of neurons before and after a synapse will affect the strength of the synapse between them, The time difference between pre- and post-synaptic neuron firing determines the direction and magnitude of changes in synaptic weights. This weight adjustment method based on the time difference between pre- and post-synaptic neuron spike is called spike time-dependent plasticity (STDP), which is an unsupervised learning method We have constructed two kinds of STDP learning algorithm. The first one is based on the global synaptic plasticity, we call it full_online_STDP [3] ,another one is based on the nearest synaptic plasticity, we call it nearest_online_STDP [4] . The difference between the two algorithms lies in the update mechanism of the pre- and post-synaptic spike traces. Here we take the global synaptic plasticity STDP algorithm as an example.

Full Synaptic Plasticity STDP learning algorithm

The weight update formula and weight normalization formula of this algorithm [2] :

\[\begin{split}dw &= Apost * (output\_spike * input\_trace) – Apre * (output\_trace * input\_spike) \\ weight &= weight + dw \\ weight &= self.w\_norm * weight/sum(torch.abs(weight))\end{split}\]

Among them, the pre-synaptic and post-synaptic spike traces of the global synaptic plasticity STDP learning algorithm are：

\[\begin{split}input\_trace &= input\_spike + input\_trace * trace\_decay \\ output\_trace &= output\_spike + output\_trace * trace\_decay\end{split}\]

Differently, the STDP learning algorithm based on the plasticity of the nearest neighbors resets the corresponding trace to 1 when there is a spike, and decays at other times. The presynaptic and postsynaptic spike traces are：

\[\begin{split}input\_trace &= input\_spike + (1 - input\_spike) * input\_trace * trace\_decay \\ output\_trace &= output\_spike + (1 - output\_spike) * output\_trace * trace\_decay\end{split}\]

At first, get the presynaptic and postsynaptic NeuronGroups from trainable_connection :

preg = conn.pre
postg = conn.post

Then, get parameters ID, such as input spike, output spike and weight name:

pre_name = conn.get_input_name(preg, postg)
post_name = conn.get_group_name(postg, 'O')
weight_name = conn.get_link_name(preg, postg, 'weight')

Add necessary parameters to Backend :

self.variable_to_backend(input_trace_name, backend._variables[pre_name].shape, value=0.0)
self.variable_to_backend(output_trace_name, backend._variables[post_name].shape, value=0.0)
self.variable_to_backend(dw_name, backend._variables[weight_name].shape, value=0.0)

Append calculate formula to Backend :

self.op_to_backend('input_trace_temp', 'var_mult', [input_trace_name, 'trace_decay'])
self.op_to_backend(input_trace_name, 'add', [pre_name, 'input_trace_temp'])

self.op_to_backend('output_trace_temp', 'var_mult', [output_trace_name, 'trace_decay'])
self.op_to_backend(output_trace_name, 'add', [post_name, 'output_trace_temp'])

self.op_to_backend('pre_post_temp', 'mat_mult_pre', [post_name, input_trace_name+'[updated]'])
self.op_to_backend('pre_post', 'var_mult', ['Apost', 'pre_post_temp'])
self.op_to_backend('post_pre_temp', 'mat_mult_pre', [output_trace_name+'[updated]', pre_name])
self.op_to_backend('post_pre', 'var_mult', ['Apre', 'post_pre_temp'])
self.op_to_backend(dw_name, 'minus', ['pre_post', 'post_pre'])
self.op_to_backend(weight_name, self.full_online_stdp_weightupdate,[dw_name, weight_name])

Weight update part:

with torch.no_grad():
    weight.add_(dw)

Weight normalization part:

weight[...] = (self.w_norm * torch.div(weight, torch.sum(torch.abs(weight), 1, keepdim=True)))
weight.clamp_(0.0, 1.0)

[1]

Pengjie Gu et al. “STCA: Spatio-Temporal Credit Assignment with Delayed Feedback in Deep SpikingNeural Networks.” In:Proceedings of the Twenty-Eighth International Joint Conference on Artificial Intelligence, IJCAI-19. International Joint Conferences on Artificial Intelligence Organization, July 2019,pp. 1366–1372. doi:10.24963/ijcai.2019/189.

[2] (1,2)

Yujie Wu et al. “Spatio-Temporal Backpropagation for Training High-Performance Spiking Neural Networks” Front. Neurosci., 23 May 2018 | doi:10.3389/fnins.2018.00331.

[3]

Sjöström J, Gerstner W. Spike-timing dependent plasticity[J]. Spike-timing dependent plasticity, 2010, 35(0): 0-0._

[4]

Gerstner W, Kempter R, van Hemmen JL, Wagner H. A neuronal learning rule for sub-millisecond temporal coding. Nature. 1996 Sep 5;383(6595):76-81. doi: 10.1038/383076a0. PMID: 8779718.

Reward-Regulated Synaptic Plasticity Algorithm

The reward-regulated synaptic plasticity algorithm can be regarded as a STDP/Anti-STDP learning mechanism for correct or wrong decisions, respectively, that is, the reward or punishment signal generated by the behavioral results of the neural network is used to exert influence on the weight update of neurons. Two RSTDP learning algorithms are implemented on our platform, one is RSTDP learning algorithm based on eligibility trace[#f5]_, and the other is RSTDP learning algorithm based on surrogate gradient [6] . Let’s take the first algorithm as an example.

RSTDP Learning Algorithm Based on Eligibility Trace

The weight update equation of the algorithm：

\[\begin{split}\begin{gather*} dw = lr * reward * eligibility\_trace \\ weight = weight + dw \end{gather*}\end{split}\]

Among them, the eligibility trace update formula is：

\[\begin{split}input\_trace &= input\_trace * e^{-\frac{dt}{\tau_{+} } } \\ input\_trace &= input\_trace + input\_spike \\ output\_trace &= output\_trace * e^{-\frac{dt}{\tau_{-} } } \\ output\_trace &= output\_trace + output\_spike \\ eligibility\_trace &= Apre * input\_trace * output\_spike + Apost * output\_trace * input\_spike\end{split}\]

First get the pre-synaptic neuron group and post-synaptic neuron group trained by the learning algorithm from trainable_connection:

preg = conn.pre
postg = conn.post

Then get the backend name of the parameters required by the learning algorithm such as input pulse, output pulse and connection weight. We refer to the function of getting the name in Connection and define intermediate variable names, such as pre- and post-synaptic pulse traces and eligibility traces. .. code-block:: python

pre_name = conn.get_input_name(preg, postg) post_name = conn.get_group_name(postg, ‘O’) weight_name = conn.get_link_name(preg, postg, ‘weight’) p_plus_name = pre_name + ‘_{p_plus}’ p_minus_name = post_name + ‘_{p_minus}’ eligibility_name = weight_name + ‘_{eligibility}’

Then add the parameters needed by the algorithm to the backend

self.variable_to_backend(p_plus_name, pre_shape, value=0.0)
self.variable_to_backend(p_minus_name, backend._variables[post_name].shape, value=0.0)
self.variable_to_backend(eligibility_name, backend._variables[weight_name].shape, value=0.0)

Then add the formula to the backend

self.op_to_backend('p_plus_temp', 'var_mult', ['tau_plus', p_plus_name])
if len(pre_shape_temp) > 2 and len(pre_shape_temp) == 4:
    self.op_to_backend('pre_name_temp', 'feature_map_flatten', pre_name)
    self.op_to_backend(p_plus_name, 'var_linear', ['A_plus', 'pre_name_temp', 'p_plus_temp'])
else:
    self.op_to_backend(p_plus_name, 'var_linear', ['A_plus', pre_name, 'p_plus_temp'])

self.op_to_backend('p_minus_temp', 'var_mult', ['tau_minus', p_minus_name])
self.op_to_backend(p_minus_name, 'var_linear', ['A_minus', post_name, 'p_minus_temp'])

self.op_to_backend('post_permute', 'permute', [post_name, permute_name])
self.op_to_backend('pre_post', 'mat_mult', ['post_permute', p_plus_name + '[updated]'])

self.op_to_backend('p_minus_permute', 'permute', [p_minus_name + '[updated]', permute_name])
if len(pre_shape_temp) > 2 and len(pre_shape_temp) == 4:
    self.op_to_backend('post_pre', 'mat_mult', ['p_minus_permute', 'pre_name_temp'])
else:
    self.op_to_backend('post_pre', 'mat_mult', ['p_minus_permute', pre_name])
self.op_to_backend(eligibility_name, 'add', ['pre_post', 'post_pre'])
self.op_to_backend(weight_name, self.weight_update, [weight_name, eligibility_name, reward_name])

Weight update code：

with torch.no_grad():
    weight.add_(dw)

[5]

Răzvan V. Florian; Reinforcement Learning Through Modulation of Spike-Timing-Dependent Synaptic Plasticity. Neural Comput 2007; 19 (6): 1468–1502. doi: https://doi.org/10.1162/neco.2007.19.6.1468

[6]

11. Stewart, G. Orchard, S. B. Shrestha and E. Neftci, “On-chip Few-shot Learning with Surrogate Gradient Descent on a Neuromorphic Processor,” 2020 2nd IEEE International Conference on Artificial Intelligence Circuits and Systems (AICAS), Genova, Italy, 2020, pp. 223-227, doi: 10.1109/AICAS48895.2020.9073948.

Monitor

The main function of the monitor is to monitor the changes of various variables during the network operation. In SPAIC, we have built-in two forms of monitors, namely StateMonitor and SpikeMonitor.

spaic.StateMonitor is designed to be used for tracking the state of Neurons , Connections and Nodes . spaic.SpikeMonitor is designed to be used for tracking the spike states and calculate the firing frequency.

self.mon_V = spaic.StateMonitor(self.layer1, 'V')
self.mon_O = spaic.StateMonitor(self.input, 'O')
self.spk_O = spaic.SpikeMonitor(self.layer1, 'O')

To initialize the monitor, we can specify the following parameters:

• target – the object to be monitored. For StateMonitor, it can be any network module containing variables such as NeuronGroup and Connection . For SpikeMonitor, it is generally a module with pulse distribution such as NeuronGroup and Encoder.

• var_name – the name of the variable that needs to be monitored, it needs to be a variable that the monitoring object has, such as the neuron’s membrane voltage ‘V’

• index – the index value of the detection variable, for example, select a few neurons in a layer of neural clusters to record, you can use index=[1,3,4,…], the default is to record the entire variable

• dt – the sampling interval of the monitor, defaults to the same as the simulation step size

• get_grad – whether to record the gradient, True means the gradient is required, False means not required, the default is False

• nbatch – whether you need to record the data of multiple batches, True will save the data of multiple runs, False will overwrite the data each time run, the default is False

Common functions to users in both StateMonitor and SpikeMonitor are:

• monitor_on – Set the monitor to start the recording for current run. The monitor is set to be monitor_on by defualt.

• monitor_on – Set the monitor to stop recording for current run.

• clear – Clear all the recorded data in the monitor.

The difference between the two monitors is that StateMonitor has five property：

• nbatch_times – logging the time step information of all batches, the shape structure of the data is (number of batches, number of time steps)

• nbatch_values – logging the monitoring parameters of the target layer of all batches. The shape structure of the data is (batch, neuron, time step, sample in the batch)

• times – logging the time step information of the current batch, the shape structure of the data is (number of time steps)

• values – logging the monitoring variable of the target layer of the current batch. The shape structure of the data is (the number of samples in this batch, the number of neurons, the number of time steps)

• tensor_values – logging the original tensor variable of the target layer of the current batch. The shape structure of the data is (the number of samples in this batch, the number of neurons, the number of time steps)

• grad – logging the gradient of the target variable of the current batch, the shape of the data is the same as the shape of the values

And SpikeMonitor has another four property：

• spk_index – logging the number of the neuron firing the current batch

• spk_times – logging the time information of the current batch of pulses

• time – logging information about the time step of the current batch

• time_spk_rate – logging the instantaneous spike rate of the target layer for the current batch

• spk_rate – logging the average spike rate of the target layer for the current batch

• spk_count – logging each neuron’s spike count of the target layer for the current batch

Example code:

time_line = Net.mon_V.times  # Take the time indices of layer1
value_line = Net.mon_V.values[0][0]  # Take the voltage change records of the first neuron of layer 1 in this batch in the whole time window
input_line = Net.mon_O.values[0][0]  # Take the spike records of the first neuron of input layer in this batch in the whole time window

# Since nbatch setted False when initialized, only have one batch
output_line_index = Net.spk_O.spk_index[0] + 1.2  # Take the spike index of this layer, since only have one neuron, add 1.2 to beautify the visualization appearance
output_line_time = Net.spk_O.spk_times[0]  # Take the spike time index of this layer

plt.subplot(2, 1, 1)
plt.title('Monitor Example Appearance')
plt.plot(time_line, value_line, label='V')
plt.scatter(output_line_time, output_line_index, s=40, c='r', label='Spike')

plt.ylabel("Membrane potential")
plt.ylim((-0.1, 1.5))
plt.legend()
plt.subplot(2, 1, 2)
plt.plot(time_line, input_line, label='input spike')
plt.xlabel("time")
plt.ylabel("Current")
plt.legend()

Result:

Contact us:

Chaofei Hong: hongchf@zhejianglab.com

Mengwen Yuan: yuanmw@zhejianglab.com

Mengxiao Zhang: mxzhang@zhejianglab.com

欢迎来到SPAIC的文档网站!

SPAIC (spike based artificial intelligence computing) 是一个用于结合神经科学与机器学习的类脑计算框架。

• Homepage in English

I. 如何安装

通过源代码安装最新版本：

通过GitHub：

git clone https://github.com/ZhejianglabNCRC/SPAIC.git
cd SPAIC
python setup.py install

II. 如何从零开始构建一个脉冲神经网络

为了便于用户了解该如何使用 SPAIC 开展自己的研究工作，我们将以使用 STCA 学习算法 [1] 训练识别 **MNIST**数据集的网络作为例子，搭建一个SNN训练网络。

1. 建立一个网络

网络是 SPAIC 中最为重要的组成部分，如同整个神经网络的框架部分，所以我们需要首先先建立一个网络，再在这个网络中填充其他的元素，例如神经元和连接。继承 spaic.Network 来重新建立一个网络类并实例化：

class SampleNet(spaic.Network):
   def __init__(self):
      super(SampleNet, self).__init__()
         ......

Net = SampleNet()

2. 添加网络组件

在搭建 Network 这个框架时，我们需要在其内部添加神经元和连接这些组件，从而使得这个网络并不是单纯的一张白纸，能够添加的组件包括了输入输出部分的 Node 、神经元组的 NeuronGroups、突触连接 connections 、监视器 monitors 、学习算法 learners。或者，我们在构建大型复杂网络时可以添加一些特殊的部件, Assembly 以及 Projection，用于将复杂的结构更加清晰地分层构建。

2.1 创建并添加节点层与神经元组

对于一个使用 STCA 训练识别 MNIST 数据集的网络而言，我们需要的节点分别是一个input层用于编码并输入数据，一个 clif 神经元层用于训练以及一个输出层用于解码脉冲输出。所以我们所添加的就是：

self.input = spaic.Encoder(num=784, coding_method='poisson')
self.layer1 = spaic.NeuronGroup(num=10, model='clif')
self.output = spaic.Decoder(num=10, dec_target=self.layer1)

Note

这里较为需要注意的是，output 层的数量需要与其 dec_target 目标层的神经元数量一致。

2.2 建立连接

在本示例中，因为网络结构相当简单，所以需要的连接只是一个简单的全连接将输入层与训练层连接到一起。

self.connection1 = spaic.Connection(self.input, self.layer1, link_type='full')
# self.connection1 = spaic.Connection(pre=self.input, post=self.layer1, link_type='full')

2.3 添加学习算法与优化算法

在本示例中，我们采用了 STCA 学习算法， STCA 学习算法 [1] 是一种采用了替代梯度策略的BPTT类算法。优化器则选择 Adam 算法并设置学习率为0.001。

self.learner = spaic.Learner(trainable=self, algorithm='STCA')
self.learner.set_optimizer('Adam', 0.001)

2.4 添加监视器

在本示例中，虽然没有添加监视器的必要，但是作为教学，我们可以对 layer1 的电压与脉冲输出进行监控，即：

self.mon_V = spaic.StateMonitor(self.layer1, 'V')
self.spk_O = spaic.SpikeMonitor(self.layer1, 'O')
# self.mon_V = spaic.StateMonitor(target=self.layer1, var_name='V')

2.5 添加backend

Backend 是 SPAIC 中极为重要的一个部分，负责后端网络的实际模拟。 backend.dt 用于设置网络模拟的时间步长，需要在建立网络前提前进行设定。不同后端以及设备的选择也需要在搭建网络前设置完成。在本示例，我们采用 PyTorch 作为后端，将网络构建于 cuda 上，以 0.1ms 作为时间步长：

# 方式一：
   if torch.cuda.is_available():
       device = 'cuda'
   else:
       device = 'cpu'
   backend = spaic.Torch_Backend(device)
   backend.dt = 0.1
   self.set_backend(backend)

# 方式二：
   self.set_backend('PyTorch', 'cuda')
   self.set_backend_dt(0.1)
   # self.set_backend(backend='PyTorch', device='cuda')
   # self.set_backend_dt(dt=0.1)

2.6 整体网络结构

import spaic
import torch

class TestNet(spaic.Network):
    def __init__(self):
        super(TestNet, self).__init__()

        # Encoding
        self.input = spaic.Encoder(num=784, coding_method='poisson')

        # NeuronGroup
        self.layer1 = spaic.NeuronGroup(num=10, model='clif')

        # Decoding
        self.output = spaic.Decoder(num=10, dec_target=self.layer1, coding_method='spike_counts')

        # Connection
        self.connection1 = spaic.Connection(pre=self.input, post=self.layer1, link_type='full')

        # Monitor
        self.mon_V = spaic.StateMonitor(self.layer1, 'V')
        self.spk_O = spaic.SpikeMonitor(self.layer1, 'O')

        # Learner
        self.learner = spaic.Learner(trainable=self, algorithm='STCA')
        self.learner.set_optimizer('Adam', 0.001)

        if torch.cuda.is_available():
            device = 'cuda'
        else:
            device = 'cpu'
        backend = spaic.Torch_Backend(device)
        backend.dt = 0.1

        self.set_backend(backend)

# 网络实例化
Net = TestNet()

3. 开始训练

3.1 加载数据集

from tqdm import tqdm
import torch.nn.functional as F
from spaic.IO.Dataset import MNIST as dataset
# 创建训练数据集
root = './spaic/Datasets/MNIST'
train_set = dataset(root, is_train=True)
test_set = dataset(root, is_train=False)

# 设置运行时间与批处理规模
run_time = 50
bat_size = 100

# 创建DataLoader迭代器
train_loader = spaic.Dataloader(train_set, batch_size=bat_size, shuffle=True, drop_last=False)
test_loader = spaic.Dataloader(test_set, batch_size=bat_size, shuffle=False)

3.2 运行网络

eval_losses = []
eval_acces = []
losses = []
acces = []
num_correct = 0
num_sample = 0
for epoch in range(100):

    # 训练阶段
    print("Start training")
    train_loss = 0
    train_acc = 0
    pbar = tqdm(total=len(train_loader))
    for i, item in enumerate(train_loader):
        # 前向传播
        data, label = item
        Net.input(data)
        Net.output(label)
        Net.run(run_time)
        output = Net.output.predict
        output = (output - torch.mean(output).detach()) / (torch.std(output).detach() + 0.1)
        label = torch.tensor(label, device=device)
        batch_loss = F.cross_entropy(output, label)

        # 反向传播
        Net.learner.optim_zero_grad()
        batch_loss.backward(retain_graph=False)
        Net.learner.optim_step()

        # 记录误差
        train_loss += batch_loss.item()
        predict_labels = torch.argmax(output, 1)
        num_correct = (predict_labels == label).sum().item()  # 记录标签正确的个数
        acc = num_correct / data.shape[0]
        train_acc += acc

        pbar.set_description_str("[loss:%f]Batch progress: " % batch_loss.item())
        pbar.update()
    pbar.close()
    losses.append(train_loss / len(train_loader))
    acces.append(train_acc / len(train_loader))
    print('epoch:{},Train Loss:{:.4f},Train Acc:{:.4f}'.format(epoch, train_loss / len(train_loader), train_acc / len(train_loader)))

    # 测试阶段
    eval_loss = 0
    eval_acc = 0
    print("Start testing")
    pbarTest = tqdm(total=len(test_loader))
    with torch.no_grad():
        for i, item in enumerate(test_loader):
            data, label = item
            Net.input(data)
            Net.run(run_time)
            output = Net.output.predict
            output = (output - torch.mean(output).detach()) / (torch.std(output).detach() + 0.1)
            label = torch.tensor(label, device=device)
            batch_loss = F.cross_entropy(output, label)
            eval_loss += batch_loss.item()

            _, pred = output.max(1)
            num_correct = (pred == label).sum().item()
            acc = num_correct / data.shape[0]
            eval_acc += acc
            pbarTest.set_description_str("[loss:%f]Batch progress: " % batch_loss.item())
            pbarTest.update()
        eval_losses.append(eval_loss / len(test_loader))
        eval_acces.append(eval_acc / len(test_loader))
    pbarTest.close()
    print('epoch:{},Test Loss:{:.4f},Test Acc:{:.4f}'.format(epoch,eval_loss / len(test_loader), eval_acc / len(test_loader)))

4. 训练结果

在训练并测试共100个epoch后，通过 matplotlib 我们得到如下的准确率曲线：

from matplotlib import pyplot as plt
plt.subplot(2, 1, 1)
plt.plot(acces)
plt.title('Train Accuracy')
plt.ylabel('Acc')
plt.xlabel('epoch')

plt.subplot(2, 1, 2)
plt.plot(test_accuracy)
plt.title('Test Accuracy')
plt.ylabel('Acc')
plt.xlabel('epoch')

plt.show()

5. 保存网络

在训练完成之后，我们可以通过 Network 内置的 save_state 函数将权重信息存储下来，也可以通过 spaic.Network_saver.network_save 函数将整体网络的结构以及权重一同存储下来。

方式一：（只存储权重）

save_file = Net.save_state(filename="TestNetwork")

方式二：（同时存储网络结构以及权重）

save_file = network_save(Net, "TestNetwork", trans_format='json')
# save_file = network_save(Net=Net, filename="TestNetwork", trans_format='json')

Note

在方式二中，网络结构存储的格式可以为 json 亦或是 yaml 格式，这两种格式都是可以直接阅读不需要转译的。而方式一与方式二这两种方式的权重都以 Pytorch 的Tensor格式存储。

附：网络的其他构建方式

1. with形式

# 初始化基本网络类的对象
Net = spaic.Network()

# 通过把网络单元在with内定义，建立网络结构
with Net:
    # 建立输入节点并选择输入编码形式
    input1 = spaic.Encoder(784, encoding='poisson')


    # 建立神经元集群，选择神经元类型，并可以设置 放电阈值、膜电压时间常数等神经元参数值
    layer1 = spaic.NeuronGroup(10, model='clif')

    # 建立神经集群间的连接
    connection1 = spaic.Connection(input1, layer1, link_type='full')

    # 建立输出节点，并选择输出解码形式
    output = spaic.Decoder(num=10, dec_target=self.layer1, coding_method='spike_counts')

    # 建立状态检测器，可以Monitor神经元、输入输出节点、连接等多种单元的状态量
    monitor1 = spaic.StateMonitor(layer1, 'V')

    # 加入学习算法，并选择需要训练的网络结构，（self代表全体ExampleNet结构）
    learner = spaic.Learner(trainable=self, algorithm='STCA')

    # 加入优化算法
    learner.set_optimizer('Adam', 0.001)

2. 通过引入模型库模型并进行修改的方式构建网络

from spaic.Library import ExampleNet
Net = ExampleNet()
# 神经元参数
neuron_param = {
    'tau_m': 8.0,
    'V_th': 1.5,
}
# 新建神经元集群
layer3 = spaic.NeuronGroup(100, model='lif', param=neuron_param)
layer4 = spaic.NeuronGroup(100, model='lif', param=neuron_param)

# 向神经集合中加入新的集合成员
Net.add_assembly('layer3', layer3)
# 删除神经集合中已经存在集合成员
Net.del_assembly(Net.layer3)
# 复制一个已有的assembly结构，并将新建的assembly加入到此神经集合
Net.copy_assembly('net_layer', ExampleNet())
# 将集合内部已有的一个神经集合替换为一个新的神经集合
Net.replace_assembly(Net.layer1, layer3)
# 将此神经集合与另一个神经集合进行合并，得到一个新的神经集合
Net2 = ExampleNet()
Net.merge_assembly(Net2)
#连接神经集合内部两个神经集群
con = spaic.Connection(Net.layer2, Net.net_layer, link_type='full')
Net.add_connection('con3', con)
#将此神经集合中的部分集合成员以及它们间的连接取出来组成一个新的神经集合，原神经集合保持不变
Net3 = Net.select_assembly([Net.layer2, net_layer])

用户手册:

基础结构

基本组成

SPAIC 中最重要的基类是 Assembly ，一个个的 Assembly 节点以及节点间的连接 Connection 最终组成了一个网络。在 Assembly 中，包含了 Network 、 NeuronGroup 以及 Node 这三个部分， Network 类即为整个网络， NeuronGroup 类则包含了各层的神经元， Node 为输入输出的节点。

平台前端结构图：

Assembly (神经集合)

Assembly 是神经网络结构中最为上层的抽象类，代表任意网络结构，其它网络模块都是 Assembly 类的子类。 Assembly 对象具有 _groups ,_connections 的两个dict属性，用于保存神经集合内部的神经集群以及连接等。作为网络建模的主要接口，包含如下主要建模函数：

• add_assembly(name, assembly) – 向神经集合中加入新的集合成员

• del_assembly(assembly, name) – 删除神经集合中已经存在的某集合成员

• copy_assembly(name, assembly) – 复制一个已有的assembly结构，并将新建的assembly加入到此神经集合

• replace_assembly(old_assembly, new_assembly) – 将集合内部已有的一个神经集合替换为一个新的神经集合

• merge_assembly(assembly) – 将此神经集合与另一个神经集合进行合并，得到一个新的神经集合

• select_assembly(assemblies, name) – 将此神经集合中的部分集合成员以及它们间的连接取出来组成一个新的神经集合，原神经集合保持不变

• add_connection(name, connection) – 连接神经集合内部两个神经集群

• del_connection(connection, name) – 删除神经集合内部的某个连接

• assembly_hide() – 将此神经集合隐藏，不参与此次训练、仿真或展示。

• assembly_show() – 将此神经集合从隐藏状态转换为正常状态。

NeuronGroup (神经元集群)

spaic.NeuronGroup 是包含了一定数量的神经元的类，通常我们称其为一层具有相同神经元模型以及连接方式的神经元组。 更多细节参考 神经元

Node (节点)

spaic.Node 是神经网络输入输出的转换节点，包含编解码机制，将输入转化为放电或将放电转化为输出。 Encoder , Decoder , Generator , Action 以及 Reward 都继承自 Node. 更多细节参考 编码解码

Network (网络)

spaic.Network 在 SPAIC 中处于模型的最顶层，许多其他的模块，例如 NeuronGroup 以及 Connection 都需要被包含于 Network 中。 spaic.Network 也负责训练、模拟以及一些数据交互的过程。 spaic.Network 支持的一些常用的交互函数如下:

• run(rum_time) – 运行网络，run_time为网络仿真时间窗

• save_state – 保存网络权重

• state_from_dict – 读取网络权重

Projection (拓扑连接)

spaic.Projection 是拓扑结构中的一个高级抽象类，与 Connection 不同的是， Projection 代表了 Assembly 之间的连接。 当用户在 Assembly 之间构建 Projection 时， 网络的构建函数将会根据 Projection 的 policies 属性生成相对应的连接。

Connection (连接)

spaic.Connection 用于在神经元组之间构建连接，也用于构建不同类型的突触。更多具体介绍参考 连接 。

Backend (后端)

后端核心，负责实际构建变量以及生成计算图。更多具体的介绍参考 模拟后端 Backend 。

具体细节

神经元

本章节主要介绍在训练以及仿真中如何选择神经元模型，以及如何根据需求在原模型的基础上更改一些重要的参数。

神经元模型是脉冲神经网络中极为重要的一个组成部分，不同的神经元模型通常代表了对不同的神经元动力学的仿真与模拟。在脉冲神经网络中，我们通常将神经元模型关于电压的变化特征化为微分方程，再由差分方程来对其进行逼近，最后获得了计算机可以进行运算的神经元模型。在SPAIC 中，我们包含了大多数较为常见的神经元模型：

• IF - Integrate-and-Fire model

• LIF - Leaky Integrate-and-Fire model

• CLIF - Current Leaky Integrate-and-Fire model

• GLIF - Generalized Leaky Integrate-and-Fire model

• aEIF - Adaptive Exponential Integrate-and-Fire model

• IZH - Izhikevich model

• HH - Hodgkin-Huxley model

在 SPAIC 中， NeuronGroup 是作为网络节点的组成，如同 PyTorch 中的layer， SPAIC 中的每个layer都是一个 NeuronGroup ，用户需要根据自己的需要指定在这个 NeuronGroup 中所包含的神经元数量、神经元类型及与其类型相关的参数等。首先需要的就是导入 NeuronGroup 库：

from spaic import NeuronGroup

LIF神经元

LIF(Leaky Integrated-and-Fire Model) 神经元的公式以及参数：

\[\begin{split}V & = tua\_m * V + I \\ O & = spike\_func(V^n)\end{split}\]

以建立一层含有100个 LIF 神经元的layer为例:

self.layer1 = NeuronGroup(num=100, model='lif')

一个含有100个标准 LIF 神经元的layer就建立好了。然而许多时候我们需要按需定制不同的 LIF 神经元以获得不同的神经元的表现，这时候就需要在建立 NeuronGroup 时，指定一些参数：

• tau_m - 神经元膜电位的时间常量，默认为6.0

• v_th - 神经元的阈值电压，默认为1.0

• v_reset - 神经元的重置电压，默认为0.0

如果用户需要调整这些变量，可以在建立 NeuronGroup 的时候输入想改变的参数即可：

self.layer2 = NeuronGroup(num=100, model='lif',
                tau_m=10.0, v_th=10, v_reset=0.2)

这样，一个自定义参数的LIF神经元就建好了。

CLIF神经元

CLIF(Current Leaky Integrated-and-Fire Model) 神经元公式以及参数:

\[\begin{split}V(t) & = M(t) - S(t) - E(t) \\ I & = V0 * I \\ M & = tau\_p * M + I \\ S & = tau\_q * S + I \\ E & = tau\_p * E + Vth * O \\ O & = spike\_func(V)\end{split}\]

• tau_p, tau_q - 突触的时间常量，默认为12.0和8.0

• tau_m - 神经元膜电位的时间常量，默认为20.0

• v_th - 神经元的阈值电压，默认为1.0

GLIF神经元

GLIF(Generalized Leaky Integrate-and-Fire Model) [1] 神经元参数:

• R, C, E_L

• Theta_inf

• f_v

• delta_v

• b_s

• delta_Theta_s

• k_1, k_2

• delta_I1, delta_I2

• a_v, b_v

aEIF神经元

aEIF(Adaptive Exponential Integrated-and-Fire Model) [2] 神经元公式以及参数:

\[\begin{split}V & = V + dt / C * (gL * (EL - V + EXP) - w + I) \\ w & = w + dt / tau\_w * (a * (V - EL) - w) \\ EXP & = delta\_t * exp(dv\_th/delta\_t) \\ dv & = V - EL \\ dv\_th & = V - Vth \\ O & = spike\_func(V) \\ if\quad V & > 20: \\ then\quad V & = EL, w = w + b\end{split}\]

• C, gL - 膜电容与泄漏电导系数

• tau_w - 自适应时间常量

• a. - 阈下自适应系数

• b. - 脉冲激发自适应系数

• delta_t - 速率因子

• EL - 泄漏反转电位

IZH神经元

IZH(Izhikevich Model) [3] 神经元公式以及参数:

\[\begin{split}V &= V + dt / tau\_M * (C1 * V * V + C2 * V + C3 - U + I) \\ V &= V + dt / tau\_M * (V* (C1 * V + C2) + C3 - U + I) \\ U &= U + a. * (b. * V - U) \\ O &= spike\_func(V^n) \\ if\quad V &> Vth, \\ then\quad V &= Vreset, U = U + d\end{split}\]

• tau_m

• C1, C2, C3

• a, b, d

• Vreset - 电压重置位

HH神经元

HH(Hodgkin-Huxley Model) [4] 神经元模型及参数:

\[\begin{split}V & = V + dt/tau\_v * (I - Ik) \\ Ik & = NA + K + L \\ NA & = g\_NA * m^3 * h * (V - V_NA) \\ K & = g\_K * n^4 * (V - V_K) \\ L & = g\_L * (V - V_L) \\ K\quad activation: \\ n & = n + dt/tau\_n * (alpha\_n * (1-n) - beta\_n * n) \\ Na\quad activation: \\ m & = m + dt/tau\_m * (alpha\_m * (1-m) - beta\_m * m) \\ Na\quad inactivation: \\ h & = h + dt/tau\_h * (alpha\_h * (1-h) - beta\_h * h) \\ alpha\_m & = 0.1 * (-V + 25) / (exp((-V+25)/10) - 1) \\ beta\_m & = 4 * exp(-V/18) \\ alpha\_n & = 0.01 * (-V + 10) / (exp((-V+10)/10) - 1) \\ beta\_n & = 0.125 * exp(-V/80) \\ alpha\_h & = 0.07 * exp(-V/20) \\ beta\_h & = 1/(exp((-V+30)/10) + 1) \\ O & = spike\_func(V)\end{split}\]

• dt

• g_NA, g_K, g_L

• E_NA, E_K, E_L

• alpha_m1, alpha_m2, alpha_m3

• beta_m1, beta_m2, beta_m3

• alpha_n1, alpha_n2, alpha_n3

• beta_n1, beta_n2, beta_n3

• alpha_h1, alpha_h2, alpha_h3

• beta_1, beta_h2, beta_h3

• Vreset

• m, n, h

• V, v_th

自定义

在稍后的 神经元模型自定义 这一章节中，我们会更加详细具体地讲述该如何在我们平台上添加自定义的神经元模型。

[1]

GLIF model : Teeter, C., Iyer, R., Menon, V., Gouwens, N., Feng, D., Berg, J., … & Mihalas, S. (2018). Generalized leaky integrate-and-fire models classify multiple neuron types. Nature communications, 9(1), 1-15.

[2]

AEIF model : Brette, Romain & Gerstner, Wulfram. (2005). Adaptive Exponential Integrate-And-Fire Model As An Effective Description Of Neuronal Activity. Journal of neurophysiology. 94. 3637-42.` doi:10.1152/jn.00686.2005. <https://doi.org/10.1152/jn.00686.2005>`_

[3]

IZH model : Izhikevich, E. M. (2003). Simple model of spiking neurons. IEEE Transactions on neural networks, 14(6), 1569-1572.

[4]

HH model : Hodgkin, A. L., & Huxley, A. F. (1952). A quantitative description of membrane current and its application to conduction and excitation in nerve. The Journal of physiology, 117(4), 500.

连接

本章节主要介绍能够在 SPAIC 平台上使用的连接方式，包含了全连接，卷积连接，稀疏连接，一对一连接等。 作为脉冲神经网络最为基本的组成结构之一，连接中包含了网络最为重要的权重信息。与此同时，作为类脑计算平台， SPAIC 平台中的连接支持仿生连接的形式，即支持反馈连接与连接延迟以及突触连接等具有一定生理特征的连接方式。

连接参数

def __init__(self, pre: Assembly, post: Assembly, name=None,
        link_type=('full', 'sparse_connect', 'conv', '...'), syn_type=['basic'],
        max_delay=0, sparse_with_mask=False, pre_var_name='O', post_var_name='Isyn',
        syn_kwargs=None, **kwargs):

在连接的初始化参数中，我们可以看到，在建立连接时，必须给定的参数为 pre , post 以及 link_type 。

• pre - 突触前神经元，或是突触前神经元组，亦可视为连接的起点，上一层

• post - 突触后神经元，或是突触后神经元组，亦可视为连接的终点，下一层

• name - 连接的姓名，用于建立连接时更易区分，建议用户给定有意义的名称

• link_type - 连接类型，可选的有全连接、稀疏连接、卷积连接等

• syn_type - 突触类型，将会在突触部分进行更为详细的讲解

• max_delay - 突触延迟，即突触前神经元的信号将延迟几个时间步之后再传递给突触后神经元

• sparse_with_mask - 稀疏矩阵所用过滤器的开启与否

• pre_var_name - 突触前神经元对突触的输出，即该连接接收到的信号，默认接受到突触前神经元发放的‘output’脉冲信号，即默认为’O’

• post_var_name - 突触对突触后神经元的输出，即输出的信号，默认为突触电流’Isyn‘

• syn_kwargs - 突触的自定义参数，将在突触介绍部分做进一步讲解

• **kwargs - 在自定义参数中包含了某些连接所需要的特定参数，这些参数将在下文提及这些连接时谈到

除了这些参数以外，还有一些与权重相关的重要参数，例如:

• w_mean - 权重的平均值

• w_std - 权重的标准差

• w_max - 权重的最大值

• w_min - 权重的最小值

• weight - 权重值

在没有给定权重值，也就是用户没有传入 weight 的情况下，我们会进行权重的随机生成，这个时候就需要借用到 w_mean 与 w_std ，根据标准差与均值生成随机数之后，若用户设定了 w_min 与 w_max 则截取在 w_min 与 w_max 之间的值作为权重，否则直接将生成的随机数作为权重。

例如在连接 conn1_example 中，该连接在建立时将会根据均值为1，标准差为5生成随机权重，并且将小于0.0的权重归为0.0，将大于2.0的权重归为2.0。

self.conn1_example = spaic.Connection(self.layer1, self.layer2, link_type='full',
                                w_mean=1.0, w_std=5.0, w_min=0.0, w_max=2.0)

全连接

全连接是连接中最为基本的一种形式，

self.conn1_full = spaic.Connection(self.layer1, self.layer2, link_type='full')

全连接包含的重要关键字参数为：

weight = kwargs.get('weight', None) # 权重，如果不给定权重，连接将采取生成随机权重
self.w_std = kwargs.get('w_std', 0.05) # 权重的标准差，用于生成随机权重
self.w_mean = kwargs.get('w_mean', 0.005) # 权重的均值，用于生成随机权重
self.w_max = kwargs.get('w_max', None) # 权重的最大值，
self.w_min = kwargs.get('w_min', None) # 权重的最小值，

bias = kwargs.get('bias', None) # 默认不使用bias，如果想要使用，可以传入Initializer对象或者与输出通道同维自定义向量对bias进行初始化

一对一连接

一对一连接在 SPAIC 中分为两种，基本的 one_to_one 以及稀疏形式的 one_to_one_sparse ，

self.conn_1to1 = spaic.Connection(self.layer1, self.layer2, link_type='one_to_one')
self.conn_1to1s = spaic.Connection(self.layer1, self.layer2, link_type='one_to_one_sparse')

一对一连接主要包含的重要关键字参数为：

卷积连接

常见的卷积连接，池化方法可选择的有 avgpool 以及 maxpool ，这两个池化方法需要在突触类型中传入方可启用。

Note

为了更好地提供对计算的支持，目前卷积连接需要与卷积突触一同使用。

卷积连接中主要包含的连接参数有：

self.out_channels = kwargs.get('out_channels', None)  # 输出通道
self.in_channels = kwargs.get('in_channels', None)    # 输入通道
self.kernel_size = kwargs.get('kernel_size', [3, 3])# 卷积核
self.w_std = kwargs.get('w_std', 0.05) # 权重的标准差，用于生成随机权重
self.w_mean = kwargs.get('w_mean', 0.05) # 权重的均值，用于生成随机权重
weight = kwargs.get('weight', None) # 权重，如果不给定权重，连接将采取生成随机权重

self.stride = kwargs.get('stride', 1)
self.padding = kwargs.get('padding', 0)
self.dilation = kwargs.get('dilation', 1)
self.groups = kwargs.get('groups', 1)
self.upscale = kwargs.get('upscale', None)

bias = kwargs.get('bias', None) # 默认不使用bias，如果想要使用，可以传入Initializer对象或者与输出通道同维自定义向量对bias进行初始化

卷积连接的示例1：

# 通过Initializer对象初始化 weight 和 bias
self.connection1 = spaic.Connection(self.input, self.layer1, link_type='conv', syn_type=['conv'],
                                        in_channels=1, out_channels=4,
                                        kernel_size=(3, 3),
                                        weight=kaiming_uniform(a=math.sqrt(5)),
                                        bias=uniform(a=-math.sqrt(1 / 9), b=math.sqrt(1 / 9))
                                        )
# 传入自定义值初始化 weight 和 bias
self.connection2 = spaic.Connection(self.layer1, self.layer2, link_type='conv', syn_type=['conv'],
                                      in_channels=4, out_channels=8, kernel_size=(3, 3),
                                      weight=w_std * np.random.randn(out_channels, in_channels, kernel_size[0], kernel_size[1]) + self.w_mean,
                                      bias=np.empty(out_channels)
                                      )
# 根据默认的w_std和w_mean随机生成初始化权重
self.connection3 = spaic.Connection(self.layer2, self.layer3, link_type='conv', syn_type=['conv'],
                                      in_channels=8, out_channels=8, kernel_size=(3, 3)
                                      )
# 通过Initializer对象初始化 weight 和 bias
self.connection4 = spaic.Connection(self.layer3, self.layer4, link_type='full',
                                      syn_type=['flatten', 'basic'],
                                      weight=kaiming_uniform(a=math.sqrt(5)),
                                      bias=uniform(a=-math.sqrt(1 / layer3_num), b=math.sqrt(1 / layer3_num))
                                      )
# 传入自定义值初始化 weight 和 bias
self.connection5 = spaic.Connection(self.layer4, self.layer5, link_type='full',
                                      weight=w_std * np.random.randn(layer4_num, layer3_num) + self.w_mean,
                                      bias=np.empty(layer5_num)
                                      )

卷积连接的示例2：

self.conv2 = spaic.Connection(self.layer1, self.layer2, link_type='conv',
                                syn_type=['conv', 'dropout'], in_channels=128, out_channels=256,
                                kernel_size=(3, 3), stride=args.stride, padding=args.padding,
                                weight=kaiming_uniform(a=math.sqrt(5)),
                                bias=uniform(a=-math.sqrt(1 / 1152), b=math.sqrt(1 / 1152))
                                )
self.conv3 = spaic.Connection(self.layer2, self.layer3, link_type='conv',
                                syn_type=['conv', 'maxpool', 'dropout'], in_channels=256, out_channels=512,
                                kernel_size=(3, 3), stride=args.stride, padding=args.padding,
                                pool_stride=2, pool_padding=0,
                                weight=kaiming_uniform(a=math.sqrt(5)),
                                bias=uniform(a=-math.sqrt(1 / 2304), b=math.sqrt(1 / 2304))
                                )
self.conv4 = spaic.Connection(self.layer3, self.layer4, link_type='conv',
                                syn_type=['conv', 'maxpool', 'dropout'], in_channels=512, out_channels=1024,
                                kernel_size=(3, 3), stride=args.stride, padding=args.padding,
                                pool_stride=2, pool_padding=0,
                                weight=kaiming_uniform(a=math.sqrt(5)),
                                bias=uniform(a=-math.sqrt(1 / 4608), b=math.sqrt(1 / 4608))
                                syn_kwargs={'p': 0.6})

稀疏连接

常见的稀疏连接，通过传入参数 density 来设置稀疏连接的连接稠密程度

随机连接

常见的随机连接，通过传入参数 probability 来设置随机连接的连接概率

突触

本章节将会主要提及脉冲神经网络中使用的不同的突触模型，主要介绍最为常用的化学突触与电突触两种。突触类型主要在建立连接时传入 syn_type 进行设置，而突触的参数则由 syn_kwargs 传入。

化学突触

化学突触是突触的正常表现形式，神经元与神经元之间通过突触递质进行信息传递，从而导致了某些离子浓度的上升亦或是下降。在计算神经科学中我们将生理上的突触转化为权重，从而使得兴奋递质与抑制性递质转化为正负权值的形式来作用到神经元上。

在 SPAIC 中，我们的连接默认使用化学突触的形式进行连接，神经元模型中也默认接收到的电流来自于突触传递的电流，因此我们将化学突触称之为基础突触，即 basic 。

self.connection = spaic.Connection(self.layer1, self.layer2, link_type='full',
                                    syn_type=['basic'],
                                    w_std=0.0, w_mean=0.1)

电突触(Gap junction)

电突触，也就是我们常称之为 Gap junction 的一种突触形式，其原理是由于突触前与突触后神经元相隔距离尤其地近，以至于产生了带电的离子互相交换。电突触的特点在于通常情况下电突触是双向的（即突触的作用同时作用在突触前神经元以及突触后神经元，使得双方的电压不断接近）

电突触的数学公式为 Igap = w_gap(Vpre - Vpost)

若要使用电突触，则需要在连接的参数中将突触类型设置为电突触:

self.connection = spaic.Connection(self.layer1, self.layer2,
                                          link_type='full', syn_type=['electrical'],
                                          w_std=0.0, w_mean=0.1,
                                          )

全部突触

在突触中，我们还实现了一些其他种类的突触，并且将池化与展平操作都放入了突触中。

• Basic_synapse – 通过 basic 调用

• conv_synapse – 通过 conv 调用，配合卷积连接使用

• DirectPass_synapse – 通过 directpass 调用，该突触类型将会使得输出等同于输入部分，即输出的Isyn将会等同于连接的突触前神经元的输出的值

• Dropout_synapse – 通过 dropout 调用

• AvgPool_synapse – 通过 avgpool 调用

• MaxPool_synapse – 通过 maxpool 调用

• BatchNorm2d_synapse – 通过 batchnorm2d 调用

• Flatten – 通过 flatten 调用

• First_order_chemical_synapse – 通过 1_order_synapse 调用，化学突触中的一阶衰减突触

• Second_order_chemical_synapse – 通过 2_order_synapse 调用，化学突触中的二阶衰减突触

• Mix_order_chemical_synapse – 通过 mix_order_synapse 调用，化学突触中的混合衰减突触

算法

本章节主要介绍在 SPAIC 平台中内置的算法，目前我们已经在平台中添加了 STCA 、STBP 、STDP 与 R-STDP 算法。其中， STCA 与 STBP 都是采用了替代梯度的梯度反传算法，而 STDP 则是 SNN 中经典的无监督突触可塑性算法， R-STDP 在 STDP 的基础上添加了 reward 机制，更好的适用于强化学习。

示例一

#STCA learner
self.learner = spaic.Learner(trainable=self, algorithm='STCA', alpha=0.5)
self.learner.set_optimizer('Adam', 0.001)
self.learner.set_schedule('StepLR', 0.01)

在示例一中，采用了 STCA 算法，用户在 trainable 参数中传入需要训练的对象， self 代指整个网络。如果用户有针对性训练的需要，可以在 trainable 的地方传入指定的层，例如 self.layer1 等，若需要传入多个指定层，则采用列表的 方式: [self.layer1, self.layer2] 。如果用户制定了部分对象为可训练的，则需要启用 pathway 参数，用于辅助梯度在全局的传递。需要将剩下不需要训练的对象添加至 pathway 中，从而使其可以传递梯度。而最后的 alpha=0.5 则是传入 STCA 的一个参数。 在 SPAIC 中，算法自带参数都在末尾进行传参。

此处还使用了 Adam 优化算法与 StepLR 学习率调整机制，在平台中我们设置了诸多可供使用的优化算法:

‘Adam’, ‘AdamW’, ‘SparseAdam’, ‘Adamx’, ‘ASGD’, ‘LBFGS’, ‘RMSprop’, ‘Rpop’, ‘SGD’,‘Adadelta’, ‘Adagrad’

以及学习率调整机制：

‘LambdaLR’, ‘StepLR’, ‘MultiStepLR’, ‘ExponentialLR’, ‘CosineAnnealingLR’, ‘ReduceLROnPlateau’, ‘CyclicLR’, ‘CosineAnnealingWarmRestarts’

示例二

#STDP learner
self._learner = spaic.Learner(trainable=self.connection1, algorithm='full_online_STDP', w_norm=3276.8)

在示例二中，采用了STDP算法，用户在 trainable 参数中传入需要训练的对象，一般为神经网络的指定层。最后的 w_norm 是传入 STDP 的参数，具体传入的参数名根据特定算法而定。

示例三

# global reward
self.reward = spaic.Reward(num=label_num, dec_target=self.layer3, coding_time=run_time,
                             coding_method='global_reward', pop_size=pop_size, dec_sample_step=time_step)
# RSTDP
self._learner = spaic.Learner(trainable=[self.connection3], algorithm='RSTDP',
                         lr=1, A_plus=1e-1, A_minus=-1e-2)

在示例三中，采用了RSTDP算法，该算法为有监督算法，用户需要用 Reward 传入reward作为惩罚、奖励信号。 Reward 和 RSTDP 需要传入的参数分别在 spaic.Neuron.Rewards 和 spaic.Learning.STDP_Learner 中查看。 在训练过程中，区别于其他算法，需要用 Reward 传递监督信号并使用 optim_step 函数将后端的 parameters_dict 和 variables 的参数同步，如下所示。

Net.input(data)
Net.reward(label)
Net.run(run_time)
output = Net.output.predict
Net._learner.optim_step()

Note

对于 RSTDPET 学习算法， batch_size 应设为1

编码解码

本章节主要关注 SPAIC 平台中的编码、解码、信号生成、奖励以及动作生成。该章节主要分为五大块，编码器、生成器、解码器、奖励器以及动作器。

编码器(Encoder)

Encoder 类是 Node 类的子类，编码器主要用于在脉冲神经网络中，将输入的数据转化为脉冲神经网络可用的时序脉冲数据。因为对于脉冲神经网络而言，以往人工神经网络中的数值输入不符合生理特征，通常使用二值的脉冲数据数据输入。并且静态的数据输入无法获取数据的时间特征，转化为具有时序的脉冲数据能够更好地表现数据的时序特征。在 SPAIC 中，我们内置了一系列较为常见的编码方式：

• SingleSpikeToBinary (‘sstb’)

• MultipleSpikeToBinary (‘mstb’)

• PoissonEncoding (‘poisson’)

• Latency (‘latency’)

• NullEncoder (‘null’)

实例化编码类时，用户需要指定 shape 或 num , coding_method 以及其他相关参数.

SingleSpikeToBinary (‘sstb’)

如果输入数据是记录了神经元发放时间的向量，并且一个神经元对应一个神经发放，我们可以使用 SingleSpikeToBinary 将发放时间转换为二进制矩阵，如图所示，其中发放时间转化为时间窗口的索引。

例如，将该输入样本的发放时间[0.9,0.5,0.2,0.7,0.1]转换为二进制矩阵。

self.input = spaic.Encoder(num=node_num, coding_method='sstb')
# or
self.input = spaic.Encoder(shape=[node_num], coding_method='sstb')

Note

Node的 num 等于给定输入样本的大小，即5，网络运行时间等于给定输入样本的最大发放时间加上dt，即1.0。

MultipleSpikeToBinary (‘mstb’)

如果输入数据包含发放神经元的索引及其发放时间，我们可以使用 MultipleSpikeToBinary 将发放索引和发放时间转换为二进制矩阵。例如，将[[0.9,0.5,0.2,0.7,0.1]、[1,1,3,2,6]]转换为二进制矩阵。

self.input = spaic.Encoder(num=node_num, shape=[2, node_num], coding_method='mstb')

Note

• 由于一个神经元对应于零或多个脉冲时间，每个样本的大小可能不同。

• Node的 num 等于给定输入样本的神经元索引的最大id加1，即7。

• 一个样本的 shape 应该是[2, node_num]，其中第一行是发放时间向量，第二行是神经元ID向量。

• 对于 MultipleSpikeToBinary 编码方法，需要同时指定初始化参数 num 和 shape 。

PoissonEncoding (‘poisson’)

泊松编码方法编码方法属于速率编码。刺激越强，放电频率越高。以图像输入为例，首先将像素强度映射到输入神经元的瞬时放电速率。然后在每个时间步，生成0到1之间的均匀随机数，并与瞬时放电率进行比较。如果随机数小于瞬时放电率，则生成脉冲。

下面的代码定义了一个 PoissonEncoding 对象，它将输入转换为泊松脉冲序列。

self.input = spaic.Encoder(num=node_num, coding_method='poisson')

Note

• 对于全连接，可以不指定初始化参数 shape 。

• 对于卷积连接，初始化参数 shape 应指定为[channel, width, height]，在这种情况下，可以不指定初始化参数 num 。

• 对于泊松编码，有时我们需要缩放输入强度，这可以通过指定 unit_conversion 参数来实现：

  unit_conversion - 一个缩放输入速率的常量参数，默认为1.0.

Latency (‘latency’)

外部刺激越强，神经元放电越早。以图像输入为例，图像中的灰度值越大，信息越重要，对应的神经元的放电时间越早。

下面的代码定义了一个 Latency 对象，它将输入转换为脉冲序列。

self.input = spaic.Encoder(num=node_num, coding_method='latency')

Note

• 对于全连接，可以不指定初始化参数 shape 。

• 对于卷积连接，初始化参数 shape 应指定为[channel, width, height]，在这种情况下，可以不指定初始化参数 num 。

NullEncoder (‘null’)

如果不需要编码方法，我们可以使用 NullEncoder 。以下代码定义了 NullEncoder 对象。

self.input = spaic.Encoder(num=node_num, coding_method='null')

Note

• 对于全连接，可以不指定初始化参数 shape 。

• 对于卷积连接，初始化参数 shape 应指定为[channel, width, height]，在这种情况下，可以不指定初始化参数 num 。

• 对于全连接，外部输入的形状应为[batch_size, time_step, node_num]。

• 对于卷积连接，外部输入的形状应为[batch_size, time_step, channel, width, height]。

生成器(Generator)

Generator 类是 Node 类的子类。它是一种特殊的编码器，可以在没有数据集的情况下生成脉冲序列或输入电流。生成器主要的作用在于，有时在进行神经元动力学仿真时，我们需要特殊的输入模式，因此我们需要有一些特殊的脉冲或者是电流模式的生成器。在 SPAIC 中，我们内置了一些模式生成器：

• poisson_generator - 根据输入速率生成泊松脉冲序列

• cc_generator - 生成恒定电流输入

实例化编码类时，用户需要指定 shape 或 num 、 coding_method 和其他相关参数。

Poisson_Generator (‘poisson_generator’)

泊松生成器方法根据输入速率生成脉冲序列。在每个时间步，生成0到1之间的均匀随机数，并与输入速率进行比较。如果随机数小于输入速率，则生成脉冲。

下面的代码定义了一个 Poisson_Generator 对象，该对象将输入速率转换为泊松脉冲序列。

self.input = spaic.Generator(num=node_num, coding_method='poisson_generator')

Note

• 对于全连接，可以不指定初始化参数 shape 。

• 对于卷积连接，初始化参数 shape 应指定为[channel, width, height]，在这种情况下，可以不指定初始化参数 num 。

• 如果外部输入为常数值，则默认情况下，所有节点的输入速率相同。

• 如果每个节点需要不同的输入速率，则应传入对应于节点形状的输入矩阵。

• 有时我们需要调整输入速率，这可以通过指定 unit_conversion 参数来实现:

  unit_conversion - 一个缩放输入速率的常量参数，默认为1.0。

CC_Generator (‘cc_generator’)

CC_Generator 可以产生恒定电流输入，这有助于用户观察和模拟各种神经元动力学。

下面的代码定义了一个 CC_Generator 对象，它将输入速率转换为脉冲序列。

self.input = spaic.Generator(num=node_num, coding_method='cc_generator')

Note

CC_Generator 的注意事项和 Poisson_Generator 的类似

解码器

Decoder 类是 Node 类的子类，其主用于在脉冲神经网络中，将输出的脉冲信号或电压转换为数字信号，例如根据 spike_counts 的规则选取发放脉冲数量最多的神经元作为预测结果，亦或是根据 first_spike 的规则选取第一个发放脉冲的神经元作为预测结果。

在 SPAIC 中，我们也内置了大多数较为常见的解码方式：

• Spike_Counts (‘spike_counts’) – 获得目标层中每个神经元的平均脉冲计数。

• First_Spike (‘first_spike’) – 获取目标层中每个神经元的第一次发放时间。

• Final_Step_Voltage (‘final_step_voltage’) – 获得目标层中每个神经元最后一步的电压。

• Voltage_Sum (‘voltage_sum’) – 获得目标层中每个神经元在时间窗口内的电压和。

解码器主要在脉冲输出阶段使用，在实例化解码类时，用户需要指定 num 、 dec_target 、 coding_method 和相关参数

例如，当解码具有10个LIF神经元的 NeuronGroup 对象的脉冲活动时，我们可以创建 Spike_Counts 类的实例：

self.target = spaic.NeuronGroup(num=10, model='lif')
self.output = spaic.Decoder(num=10, dec_target=self.target, coding_method='spike_counts')

Note

• 参数 dec_target 的值是要解码的层对象

• Decoder 类中参数 num 的值应与目标层中 num 的值相同

• 若要实例化其他解码类，只需将相应类的str名称赋值给 coding_method 参数即可

• 参数 coding_var_name 的值是要解码的变量，例如’O’或’V’，’O’表示脉冲，’V’表示电压。

• 对于 Spike_Counts 和 First_Spike ，参数 coding_var_name 的默认值为’O’。

• 对于 Final_Step_Voltage 和 Voltage_Sum ，参数 coding_var_name 的默认值为’V’。

奖励器

Reward 类是 Node 类的子类，它可以被看作是一种不同类型的解码器。主要作用是在执行强化任务的时候，有时需要根据任务目的解码指定对象的活动并设定奖励规则来获取奖励。例如分类任务下的 global_reward 的规则，根据脉冲发放数量或者最大膜电位确定预测结果，若预测结果是期望的结果，则返回正奖励；若不等，则返回负奖励。样本的 batch_size>1 时，返回取均值后的奖励作为全局奖励。在 SPAIC 中，我们内置了一些奖励类：

• Global_Reward (‘global_reward’) – 获得全局奖励。对于分类任务，根据脉冲数或最大膜电位确定预测标签。如果预测标签与实际标签相同，则将返回正奖励。相反，将返回负奖励。

• XOR_Reward (‘xor_reward’) – XOR任务的奖励机制。当输入模式的期望结果为1时，如果输出脉冲数大于0，将获得正奖励。当期望结果为0时，如果输出脉冲数大于0，则获得惩罚。

• DA_Reward (‘da_reward’) – 获得与 dec_target 中神经元相同维度的奖励。

• Environment_Reward (‘environment_reward’) – 从强化学习环境中获得奖励。

奖励器主要在脉冲输出阶段使用，在实例化奖励类时，用户需要指定 num 、 dec_target 、 coding_method 和相关参数例如当解码含有10个LIF神经元的 NeuronGroup 对象的脉冲活动以获得全局奖励时，我们可以这样建立 Global_Reward 类实例:

self.target = spaic.NeuronGroup(num=10, model='lif')
self.reward = spaic.Reward(num=10, dec_target=self.target, coding_method='global_reward')

Note

• 参数 dec_target 的值是要解码的层对象

• Reward 类中参数 num 的值应与目标层中 num 的值相同

• 若要实例化其他奖励类，只需将相应类的str名称赋值给 coding_method 参数即可

• 参数 coding_var_name 的值是要解码的变量，例如’O’或’V’，’O’表示脉冲，’V’表示电压。

• 参数 coding_var_name 的默认值为’O’。

对于 Global_Reward 、 XOR_rewage 和 DA_revage ，我们可以指定一些参数：

• pop_size - 解码神经元的总体大小，默认为1（每个类别由一个神经元表示）

• dec_sample_step - 解码采样时间步长，默认为1（每个时间步长获得奖励）

• reward_signal - 奖励，默认为1.0

• punish_signal - 惩罚，默认为-1.0

动作器

Action 类是 Node 类的子类，它也是一个特殊的解码器，将输出转换为动作。主要作用是在执行GYM强化环境中的强化任务时，需要根据指定对象的活动设定动作选择机制选择接下来要执行的动作。例如 PopulationRate_Action 规则，解码对象的神经元的群体数与动作数目个数一致，以每个群体的发放速率为权重来选择下一步动作，群体的发放速率越大，选中的可能性越大。在 SPAIC 中，我们内置了一些动作类：

• Softmax_Action (‘softmax_action’) – 基于目标层的脉冲，使用softmax函数选择动作。

• PopulationRate_Action (‘pop_rate_action’) – 将具有最大脉冲频率的神经元群体的标签作为动作。

• Highest_Spikes_Action (‘highest_spikes_action’) – 将目标层中发放脉冲最多的神经元的标签作为动作。

• Highest_Voltage_Action (‘highest_voltage_action’) – 将目标层中具有最大电压的神经元的标签作为动作。

• First_Spike_Action (‘first_spike_action’) – 将目标层中第一个发放脉冲的神经元的标签作为动作。

• Random_Action (‘random_action’) – 从动作空间随机采样获得动作。

动作器主要在脉冲输出阶段使用，在实例化动作类时，用户需要指定 num 、 dec_target 、 coding_method 和相关参数例如当解码含有10个LIF神经元的 NeuronGroup 对象的脉冲活动以获得下一步活动时，我们可以这样建立 Softmax_Action 类实例:

self.target = spaic.NeuronGroup(num=10, model='lif')
self.reward = spaic.Action(num=10, dec_target=self.target, coding_method='softmax_action')

Note

• 参数 dec_target 的值是要解码的层对象

• Action 类中参数 num 的值应与目标层中 num 的值相同。

• 若要实例化其他动作类，只需将相应类的str名称赋值给 coding_method 参数即可。

• 参数 coding_var_name 的值是要解码的变量，例如’O’或’V’，’O’表示脉冲，’V’表示电压。

对于 PopulationRate_Action ，我们可以指定 pop_size 参数:

• pop_size - 解码神经元的总体大小，默认为1（每个类别由一个神经元表示）

网络

本章节主要介绍 SPAIC 平台中基于 Network 类构建网络和运行网络的方法。

网络构建

模型构建可以采用三种构建方法：第一种类似Pytorch的module类继承，在_init_函数中构建的形式，第二种是类似Nengo的通过with语句的构建方式。第三种，也可以在建模过程中在现有网络中通过添加函数接口添加新的网络模块。

模型构建方法1： 类继承形式

class SampleNet(spaic.Network):
   def __init__(self):
      super(SampleNet, self).__init__()

      self.layer1 = spaic.NeuronGroup(100, neuron_model='clif')
      ......

Net = SampleNet()

模型构建方法2： with形式

Net = SampleNet()
with Net:

   layer1 = spaic.NeuronGroup(100, neuron_model='clif')
   ......

模型构建方法3： 通过函数接口构建或更改网络

Net = SampleNet()
layer1 = spaic.NeuronGroup(100, neuron_model='clif')
....

Net.add_assembly('layer1', layer1)

当前 Network 提供的构建或修改网络的函数接口包括：

• add_assembly(name, assembly) - 添加神经集群类（包括 Node， NeuronGroup 等），其中参数name代表网络中变量名，assembly代表被添加的神经集群对象

• copy_assembly(name, assembly) - 复制神经集群类（包括 Node， NeuronGroup 等），与add_assembly不同，此接口将assembly对象进行克隆后添加到网络中

• add_connection(name, connection) - 添加连接对象，其中参数name代表网络中变量名，connection代表被添加的连接对象

• add_projection(name, projection) - 添加拓扑映射对象，其中参数name代表网络中变量名，projection代表被添加的拓扑映射对象

• add_learner(name, learner) - 添加学习算法，其中参数name代表网络中变量名，learner代表被添加的学习算法对象

• add_moitor(name, moitor) - 添加监视器，其中参数name代表网络中变量名，moitor代表被添加的监视器对象

网络运行及运行参数设置

Network 对象提供运行和设置运行参数的函数接口，包括：

• run(backend_time) - 运行网络的函数接口，其中backend_time参数是网络运行时间

• run_continue(backend_time) - 继续运行网络的函数接口，与run不同，run_continue不会重置各变量初始值而是根据原初始值继续运行

• set_backend(backend, device, partition) - 设置网络运行的后端， 其中参数backend是运行后端对象或后端名称，device是后端计算的硬件， partition代表是否将模型分布不同device上

• set_backend_dt(backend, dt) - 设置网络运行时间步长， dt为步长

• set_random_seed(seed) - 设置网络运行随机种子

输入输出

数据加载(Dataloader)

Dataloader 是数据集读取的接口，该接口的目的是将自定义的Dataset根据 batch_size 大小、是否shuffle等封装成一个 batch_size 大小的数组，用于网络的训练。

Dataloader 由数据集和采样器组成，初始化参数如下：

• dataset(Dataset) – 传入的数据集

• batch_size(int, optional) – 每个batch的样本数, 默认为1

• shuffle(bool, optional) – 在每个epoch开始的时候，对数据进行重新排序，默认为False

• sampler(Sampler, optional) – 自定义从数据集中取样本的策略

• batch_sampler(Sampler, optional) – 与sampler类似，但是一次只返回一个batch的索引

• collate_fn(callable, optional) – 将一个list的sample组成一个mini-batch的函数

• drop_last(bool, optional) – 如果设置为True，对于最后一个batch，如果样本数小于batch_size就会被扔掉，比如batch_size设置为64，而数据集只有100个样本，那么训练的时候后面的36个就会被扔掉。如果为False（默认），那么会继续正常执行，只是最后的batch_size会小一点。

以导入MNIST数据集为例：

root = './Datasets/MNIST' # 数据集的地址
train_set = dataset(root, is_train=True)   # 训练集
test_set = dataset(root, is_train=False)   # 测试集
bat_size = 20
# 创建DataLoader
train_loader = spaic.Dataloader(train_set, batch_size=bat_size, shuffle=True)
test_loader = spaic.Dataloader(test_set, batch_size=bat_size, shuffle=False)

Note

需要注意的是：

1、创建 Dataloader 时如果指定了 sampler 这个参数，那么 shuffle 必须为False

2、如果指定 batch_sampler 这个参数，那么 batch_size ，shuffle ，sampler， drop_last 就不能再指定了

模拟后端 Backend

Backend 是 SPAIC 平台后端的核心部件，负责网络整体的运行模拟。 dt 与 runtime 是 backend 中最为重要的两个参数，分别代表了模拟的时间步与时间窗的长度。而 time 代表了当前模拟到的时刻，n_time_step 代表了当前模拟到的时间步。

Backend 中可供用户使用的函数有：

• set_runtime – 设置时间窗的长度，或者称之为模拟的时间长度

• add_variable – 向后端添加变量，用于自定义算法时，需要添加新的变量至backend中

• add_operation – 向后端添加新的计算式，用于自定义算法、神经元等操作中，需要依照一定的格式

• register_standalone – 注册独立操作，主要用于向后端添加一些平台不支持的操作

• register_initial – 注册初始化操作，初始化操作中的计算将会在每个时间窗的最开始运算一次，而不会随着时间步的运行每个dt都进行运算

add_variable

使用 add_variable 时，必须添加的参数有 name 与 shape ，可以选择性输入的参数有 value、 is_parameter 、 is_sparse、 init、 min 与 max 。

name 决定了该变量在后端存储时的 key，而 shape 决定了维度, value 代表了这个变量的值，init 决定了该变量在每一次初始化之后的值， is_parameter 这个参数决定了该变量是否可训练。

add_operation

使用 add_operation 时，必须添加的参数有 op , op 这个参数传入的是具体需要后端进行运算的计算式。使用该函数之后，将会把用户所提供的计算式添加至整体的网络计算图中，再经由后端的构建函数，具体安排计算流程。本函数主要用于自定义计算， 用于将用户自定义的函数计算式添加至计算流程中。

保存与读取模型

该部分将详细描述两种存取网络信息的方式。

Network中预先定义的函数

采用 Network 中预定义的 save_state 与 state_from_dict 函数将权重直接进行存取。

save_state 函数可选的参数有 filename 、direct 以及 save 。用户如果直接调用 save_state 函数时，将会以默认的随机名称 autoname 将后端中的权重变量直接存储于当前目录下的'./autoname/parameters' 文件夹下的 '_parameters_dict.pt' 文件中。启用 filename 时，将会以用户给予的 filename 替换 autoname 。 启用 direct 参数则用于指定存储的目录。 save 参数默认为 True ，即启用保存，若为 False ，则该函数会直接返回后端中存储的权重信息。

state_from_dict 函数的参数与 save_state 类似，不同点在于多了 state 与 device 参数而少了 save 参数。 state 参数如果传入参数，则该函数会直接使用传入的参数来替代后端的权重参数，在该参数为None的情况下，则会根据 filename 与 direct 来决定文件的读取路径。 使用此函数时选用的 device 则会将读取出来的权重参数存储于对应的设备上。

Net.save_state('Test1', True)
...
Net.state_from_dict(filename='Test1', device=device)

network_save 与 network_load

Library 中的网络存储模块 spaic.Network_saver.network_save 函数与 spaic.Network_loader.network_load 函数将会将完整的网络结构以及权重信息分别存储下来，该方式在使用时需要一个文件名 filename ，然后平台会在用户提供的目录或是默认的当前目录下新建 'filename/filename.json' 用于保存网络结构，权重的存储路径与 net.save_state 相同，都会在目标目录下进行存储。 其次，用户在使用 network_save 时，还可以选择存储的文件格式， json 或是 yaml 。

network_dir = network_save(Net=Net, filename='TestNet',
                                        trans_format='json', combine=False, save=True)

# network_dir = 'TestNet'
Net2 = network_load(network_dir, device=device)

在 network_save 中，

• Net – 具体 SPAIC 网络中的网络对象

• filename – 文件名称， network_save 将会将 Net 以该名称进行存储，若不提供则会根据网络名存储

• path – 文件的存储路径，将会在目标路径下根据文件名新建文件夹

• trans_format – 存储格式，此处可以选择的是‘json’或是’yaml‘，默认为‘json’结构。

• combine – 该参数制定了权重是否与网络结构存储在一起，默认为 False ，分开存储网络结构与权重信息。

• save – 该参数决定了平台是否会将网络结构存储下来，若为 True ，则最后会返回存储的名称以及网络信息，若为 False ，则不会存储网络，仅仅只会将网络结构以字典的形式返回

• save_weight – 该参数决定了平台是否会存储权重部分（后端部分），若为 True 则会存储权重。

在存储网络各部分参数过程中，如果神经元的参数采用Tensor的形式传入，则在存储文件中将会存储这些参数的名称，并将实际参数存储于与权重同一目录下的diff_para_dict.pt文件中。

下面，举例说明保存下来的网络结构中各个参数所代表的意义：

# 输入节点的存储信息
-   input:
        _class_label: <nod> # 表示该对象为节点类型
        _dt: 0.1 # 每个时间步的长度
        _time: null #
        coding_method: poisson # 编码方式
        coding_var_name: O # 该节点输出的对象
        dec_target: null # 解码对象，由于是input节点，没有解码对象
        name: input # 节点名称
        num: 784 # 节点中的元素个数
        shape: # 维度
        - 784

# 神经元层的存储信息
-   layer1:
        _class_label: <neg> # 表示该对象为NeuronGroup类型
        id: autoname1<net>_layer1<neg> # 表示该NeuronGroup的id，具体含义为，该对象是在名为autoname1的网络下的名为layer1的神经元组
        model_name: clif # 采用的神经元模型的类型
        name: layer1 # 该NeuronGroup的姓名
        num: 10 # 该NeuronGroup中Neuron的数量
        parameters: {} # 额外输入的kwargs中的parameters，在神经元中为各类神经元模型的参数
        shape: # 维度
        - 10
        type: null # 该type表示的是神经元是兴奋还是抑制，用于Projection中policy功能

-   layer3:
    -   layer1:
            _class_label: <neg> # 表示该对象为NeuronGroup类型
            id: autoname1<net>_layer3<asb>_layer1<neg>  # 表示该NeuronGroup的id，具体含义为，该对象是在名为autoname1的网络下的名为layer3的组合中的名为layer1的神经元组
            model_name: clif # 采用的神经元模型的类型
            name: layer1 # 该NeuronGroup的姓名，由于是在layer3内部，所以不会出现与上述layer1重名的现象
            num: 10 # 该NeuronGroup中Neuron的数量
            parameters: {} # 额外输入的kwargs中的parameters，在神经元中为各类神经元模型的参数
            shape: # 维度
            - 10
            type: null # 该type表示的是神经元是兴奋还是抑制，暂未启用该参数

    -   connection0:
            _class_label: <con> # 表示该对象为Connection类型
            link_type: full # 连接形式为全链接
            max_delay: 0 # 连接的最大延迟
            name: connection0 # 连接的姓名
            parameters: {}
            post: layer3   # 突触后神经元为layer3层, 此处为特殊情况，layer3其实为一个assembly
            post_var_name: WgtSum   # 该连接对突触后神经元的输出为WgtSum
            pre: layer2    # 突触前神经元为layer2层
            pre_var_name: O         # 该连接接受突触前神经元的输入为‘O’
            sparse_with_mask: false # 是否启用mask，该设定为平台对于系数矩阵所设置，具体可移步connection中查看具体说明
            weight: # 权重矩阵
                autoname1<net>_layer3<asb>_connection0<con>:autoname1<net>_layer3<asb>_layer3<neg><-autoname1<net>_layer3<asb>_layer2<neg>:{weight}: # 此处为该权重的id，在平台后端变量库中可以获取
                -   - 0.05063159018754959
                # 该权重的id的格式解读为：这是一个属于网络autoname1的组合layer3中的名为connection0的连接，该链接由'<-'标识后方的autoname1中的layer3下的layer2层连接向autoname1中的layer3中的layer3
                # 即， layer3为autoname1中的一个组合层，该连接为组合层layer3中的layer2连向了layer3

# 连接的存储信息
-   connection1:
        _class_label: <con> # 表示该对象为Connection类型
        link_type: full # 连接形式为全链接
        max_delay: 0 # 连接的最大延迟
        name: connection1 # 连接的姓名
        parameters: # 连接的参数，此处为连接初始化时所用的参数，有给定权值时将会采用给定的权值
            w_mean: 0.02
            w_std: 0.05
        post: layer1   # 突触后神经元为layer1层
        post_var_name: WgtSum   # 该连接对突触后神经元的输出为WgtSum
        pre: input     # 突触前神经元为input层
        pre_var_name: O         # 该连接接受突触前神经元的输入为‘O’
        sparse_with_mask: false # 是否启用mask，该设定为平台对于系数矩阵所设置，具体可移步connection中查看具体说明
        weight: # 权重矩阵
            autoname1<net>_connection1<con>:autoname1<net>_layer1<neg><-autoname1<net>_input<nod>:{weight}:
            -   - 0.05063159018754959
                ......

# 学习算法的存储信息
-   learner2:
        _class_label: <learner> # 表示该对象为Learner类型，为学习算法
        algorithm: full_online_STDP # 表示Learner对象采用的学习算法是 full_online_STDP
        lr_schedule_name: null # 表示该Learner对象采用的 lr_schedule优化算法，null为未采用
        name: _learner2 # 该Learner对象的名称
        optim_name: null # 表示该Learner对象采用的optimizer优化算法，null为未采用
        parameters: {} # 表示该Learner对象的额外参数，例如在STCA中需要设定一个alpha值
        trainable: # 表示该Learner对象作用的范围，此处即学习算法针对connection1与connection2起作用
        - connection1
        - connection2

自定义

尽管平台已经包含了各类丰富的模型，目前常用的编解码、神经元、连接形式都已实现，但是在研究的过程中我们时常有更多的不同的需求，在这个章节中我们将会介绍如何在 SPAIC 平台上自定义添加各类模块。

编解码方法自定义

本章节主要介绍编码器、生成器、解码器、奖励器以及动作器的自定义，以便当本平台提供的内置方法无法满足用户需求时，用户可以方便的添加符合自己需求的编解码方案。

编码器自定义

编码是将输入的数据转化为脉冲神经网络可用的时序脉冲数据，是搭建神经网络要考虑的重要一步，不同的编码方法会生成不同的时序脉冲数据，为了满足用户的大多数应用需求，在本平台中内置了6种最常用的编码方法，内置的编码方法可能无法满足用户的任意需求，这时候就需要用户自己添加一些更符合其实验目的的编码方案。定义编码方案的这一步可以依照 spaic.Neuron.Encoders 文件中的格式进行添加。

编码方法初始化

自定义的编码方法需继承 Encoder 类，其初始化方法中的参数名需与 Encoder 类的一致，若需要传入初始化参数以外的参数，可以通过 kwargs 传入，以 PoissonEncoding 类初始化函数为例：

def __init__(self, shape=None, num=None, dec_target=None, dt=None, coding_method='poisson',
         coding_var_name='O', node_type=('excitatory', 'inhibitory', 'pyramidal', '...'), **kwargs):
    super(PoissonEncoding, self).__init__(shape, num, dec_target, dt, coding_method, coding_var_name, node_type,
                                      **kwargs)
    self.unit_conversion = kwargs.get('unit_conversion', 1.0)

在这个初始化方法中，unit_conversion 是 PoissonEncoding 类所需要的参数，我们通过从 kwargs 中获取的方式来设定。

定义编码函数

编码函数是编码方法的实现部分，因为平台计划支持多后端（ pytorch 、 TensorFlow 等），不同的后端支持的数据类型不同，相关的数据操作也不同，所以针对不同的计算后端需要在前端编码方法中实现对应的编码函数。 我们以 PoissonEncoding 编码方法的 torch_coding 实现过程作为示例进行展示：

def torch_coding(self, source, device):
    # Source is raw real value data.
    # For full connection, the shape of source is [batch_size, num]
    # For convolution connection, the shape of source is [batch_size] + shape
    if source.__class__.__name__ == 'ndarray':
        source = torch.tensor(source, device=device, dtype=self._backend.data_type)
    # The shape of the encoded spike trains.
    spk_shape = [self.time_step] + list(self.shape)
    spikes = torch.rand(spk_shape, device=device).le(source * self.unit_conversion*self.dt).float()
    return spikes

在最后，需要添加 Encoder.register("poisson", PoissonEncoding) 用于将该编码方法添加至编码方法的库中。

生成器自定义

生成器可用于生成服从特定分布的时空脉冲数据或者一些特殊的电流模式，在平台中内置了2种最常用的生成器方法，内置的生成器方法可能无法满足用户的任意需求，这时候就需要用户自己添加一些更符合其实验目的的生成器方案。定义生成器方案的这一步可以依照 spaic.Neuron.Generators 文件中的格式进行添加。

生成器方法初始化

自定义的生成器方法需继承 Generator 类，其初始化方法中的参数名需与 Generator 类的一致，若需要传入初始化参数以外的参数，可以通过 kwargs 传入，以恒定电流生成器 CC_Generator 类的初始化函数为例：

def __init__(self, shape=None, num=None, dec_target=None,  dt=None,
             coding_method='cc_generator', coding_var_name='O', node_type=('excitatory', 'inhibitory', 'pyramidal', '...'), **kwargs):
    super(CC_Generator, self).__init__(shape, num, dec_target, dt, coding_method, coding_var_name, node_type,
                                   **kwargs)

定义生成器函数

生成函数是生成方法的实现部分，因为平台计划支持多后端（ pytorch 、 TensorFlow 等），不同的后端支持的数据类型不同，相关的数据操作也不同，所以针对不同的计算后端需要在前端生成方法中实现对应的生成函数。 我们以 CC_Generator 生成方法的 torch_coding 实现过程作为示例进行展示：

def torch_coding(self, source, device):

    if not (source >= 0).all():
        import warnings
        warnings.warn('Input current shall be non-negative')
    if source.__class__.__name__ == 'ndarray':
        source = torch.tensor(source, dtype=self._backend.data_type, device=device)

    spk_shape = [self.time_step] + list(self.shape)
    spikes = source * torch.ones(spk_shape, device=device)
    return spikes

在最后，需要添加 Generator.register('cc_generator', CC_Generator) 用于将该生成器方法添加至生成器方法的库中。

解码器自定义

解码是将输出的脉冲信号进行一定程度的取舍和转换，为了满足用户的大多数应用需求，平台中内置了5种常用的解码方法，内置的解码方法可能无法满足用户的任意需求，这时候就需要用户自己添加一些更符合其实验目的的解码方案。定义解码方案的这一步可以依照 spaic.Neuron.Decoders 文件中的格式进行添加。

解码方法初始化

自定义的解码方法需继承 Decoder 类，其初始化方法中的参数名需与 Decoder 类的一致，若需要传入初始化参数以外的参数，可以通过 kwargs 传入，以 Spike_Counts 类的初始化函数为例：

def __init__(self, shape=None, num=None, dec_target=None, dt=None, coding_method='spike_counts',
        coding_var_name='O', node_type=('excitatory', 'inhibitory', 'pyramidal', '...'), **kwargs):
    super(Spike_Counts, self).__init__(shape, num, dec_target, dt, coding_method, coding_var_name, node_type,
                                  **kwargs)
    self.pop_size = kwargs.get('pop_size', 1)

在这个初始化方法中，pop_size 是 Spike_Counts 类实现群体脉冲数解码所需要的参数，我们通过从 kwargs 中获取的方式来设定。

定义解码函数

解码函数是解码方法的实现部分，因为平台计划支持多后端（ pytorch 、 TensorFlow 等），不同的后端支持的数据类型不同，相关的数据操作也不同，所以针对不同的计算后端需要在前端解码方法中实现对应的解码函数。 我们以 Spike_Counts 解码方法的 torch_coding 实现过程作为示例进行展示：

def torch_coding(self, record, target, device):
    # record is the activity of the NeuronGroup to be decoded
    # the shape of record is (time_step, batch_size, n_neurons)
    # target is the label of the sample
    spike_rate = record.sum(0).to(device=device)
    pop_num = int(self.num / self.pop_size)
    pop_spikes_temp = (
        [
            spike_rate[:, (i * self.pop_size): (i * self.pop_size) + self.pop_size].sum(dim=1)
            for i in range(pop_num)
        ]
    )
    pop_spikes = torch.stack(pop_spikes_temp, dim=-1)
    return pop_spikes

在最后，需要添加 Decoder.register('spike_counts', Spike_Counts) 用于将该解码方法添加至解码方法的库中。

奖励器自定义

奖励用于将目标对象的活动转化为奖励信号。为了满足用户的大多数应用需求，平台中内置了4种常用的奖励方法，内置的奖励方法可能无法满足用户的任意需求，这时候就需要用户自己添加一些更符合其实验目的的奖励方案。定义奖励方案的这一步可以依照 spaic.Neuron.Rewards 文件中的格式进行添加。

奖励方法初始化

自定义的奖励方法需继承 Reward 类，其初始化方法中的参数名需与 Reward 类的一致，若需要传入初始化参数以外的参数，可以通过 kwargs 传入，以 Global_Reward 类的初始化函数为例：

def __init__(self,shape=None, num=None, dec_target=None, dt=None, coding_method='global_reward', coding_var_name='O', node_type=('excitatory', 'inhibitory', 'pyramidal', '...'), **kwargs):
    super(Global_Reward, self).__init__(shape, num, dec_target, dt, coding_method, coding_var_name, node_type, **kwargs)
    self.pop_size = kwargs.get('pop_size', 1)
    self.reward_signal = kwargs.get('reward_signal', 1)
    self.punish_signal = kwargs.get('punish_signal', -1)

在这个初始化方法中，pop_size, reward_signal, punish_signal 是 Global_Reward 类需要的参数，我们通过从 kwargs 中获取的方式来设定。

定义奖励函数

奖励函数是奖励方法的实现部分，因为平台计划支持多后端（ pytorch 、 TensorFlow 等），不同的后端支持的数据类型不同，相关的数据操作也不同，所以针对不同的计算后端需要在前端奖励方法中实现对应的奖励函数。 我们以 Global_Reward 奖励方法的 torch_coding 实现过程作为示例进行展示：

def torch_coding(self, record, target, device):
    # the shape of record is (time_step, batch_size, n_neurons)
    spike_rate = record.sum(0)
    pop_num = int(self.num / self.pop_size)
    pop_spikes_temp = (
        [
            spike_rate[:, (i * self.pop_size): (i * self.pop_size) + self.pop_size].sum(dim=1)
            for i in range(pop_num)
        ]
    )
    pop_spikes = torch.stack(pop_spikes_temp, dim=-1)
    predict = torch.argmax(pop_spikes, dim=1)  # return the indices of the maximum values of a tensor across columns.
    reward = self.punish_signal * torch.ones(predict.shape, device=device)
    flag = torch.tensor([predict[i] == target[i] for i in range(predict.size(0))])
    reward[flag] = self.reward_signal
    if len(reward) > 1:
        reward = reward.mean()
    return reward

在最后，需要添加 Reward.register('global_reward', Global_Reward) 用于将该奖励方法添加至奖励方法的库中.

动作器自定义

动作用于将目标对象的活动转化为下一步的动作。为了满足用户的大多数应用需求，平台中内置了6种常用的动作方法，内置的动作方法可能无法满足用户的任意需求，这时候就需要用户自己添加一些更符合其实验目的的动作方案。定义动作方案的这一步可以依照 spaic.Neuron.Actions 文件中的格式进行添加。

动作方法初始化

自定义的动作方法需继承 Action 类，其初始化方法中的参数名需与 Action 类的一致，若需要传入初始化参数以外的参数，可以通过 kwargs 传入，以 Softmax_Action 类的初始化函数为例：

def __init__(self, shape=None, num=None, dec_target=None, dt=None, coding_method='softmax_action', coding_var_name='O', node_type=('excitatory', 'inhibitory', 'pyramidal', '...'), **kwargs):
    super(Softmax_Action, self).__init__(shape, num, dec_target, dt, coding_method, coding_var_name, node_type, **kwargs)

定义动作函数

动作函数是动作方法的实现部分，因为平台计划支持多后端（ pytorch 、 TensorFlow 等），不同的后端支持的数据类型不同，相关的数据操作也不同，所以针对不同的计算后端需要在前端动作方法中实现对应的动作函数。 我们以 Softmax_Action 奖励方法的 torch_coding 实现过程作为示例进行展示：

def torch_coding(self, record, target, device):
    # the shape of record is (time_step, batch_size, n_neurons)
    assert (
        record.shape[2] == self.num
    ), "Output layer size is not equal to the size of the action space."
    spikes = torch.sum(record, dim=0)
    probabilities = torch.softmax(spikes, dim=0)
    return torch.multinomial(probabilities, num_samples=1).item()

在最后，需要添加 Action.register('softmax_action', Softmax_Action) 用于将该动作方法添加至动作方法的库中。

神经元模型自定义

神经元模型是进行神经动力学仿真环节中最为重要的一步，不同的模型与不同的参数都会产生不同的现象。为了应对用户不同的应用需求， SPAIC 内置了许多最为常用的神经元模型，但是偶尔还是会有力所不能及，这时候就需要用户自己添加一些更符合其实验的个性化神经元。定义神经元的这一步可以依照 spaic.Neuron.Neuron 文件中的格式进行添加。

定义变量以及外部参数

在定义变量的阶段，我们需要先了解平台中设定的几个变量的形式：

• _tau_variables – 指数衰减常数

• _membrane_variables – 衰减常数

• _variables – 普通变量

• _parameter_variables – 参数变量

• _constant_variables – 固定变量

对于 _tau_variables 会进行变换 tau_var = np.exp(-dt/tau_var), 对于 _membrane_variables 会进行变换 membrane_tau_var = dt/membrane_tau_var,

在定义变量时，同时需要设定初始值，在网络的每一次运行后，神经元的参数都会被重置为此处设定的初始值。在定义神经元模型的最初部分，我们需要先定义该神经元模型可以变更的一些参数，这些参数可由传参来改变。例如在 lif 神经元中，我们将其原本的公式经过变换后可得：

"""
LIF model:
# V(t) = tuaM * V^n[t-1] + Isyn[t]   # tauM: constant membrane time (tauM=RmCm)
O^n[t] = spike_func(V^n[t-1])
"""

在这个公式中，tauM 以及阈值 v_th 都是可变的参数，所以我们通过从 kwargs 中获取的方式来改变，完整的变量定义如下：

self._variables['V'] = 0.0
self._variables['O'] = 0.0
self._variables['Isyn'] = 0.0

self._parameter_variables['Vth'] = kwargs.get('v_th', 1)
self._constant_variables['Vreset'] = kwargs.get('v_reset', 0.0)

self._tau_variables['tauM'] = kwargs.get('tau_m', 20.0)

定义计算式

计算式是神经元模型最为重要的部分，一行一行的计算式决定了神经元的各个参数在模拟过程中将会经过一些什么样的变化。

在添加计算式时，有一些需要遵守的规则。首先，每一行只能计算一个特定的计算符，所以需要将原公式进行分解，分解为独立的计算符。目前在平台中内置的计算符可以参考 spaic.backend.backend 中对各个计算符具体的介绍:

• add, minus, div – 简单的加减除的操作

• var_mult, mat_mult, mat_mult_pre, sparse_mat_mult, reshape_mat_mult – 变量乘法，矩阵乘法，对第一个因子进行维度转换的矩阵乘法，稀疏矩阵乘法，对第二个因子进行维度转换的矩阵乘法

• var_linear, mat_linear – result=ax+b 变量的一阶线性乘法加和与矩阵的一阶线性乘法加和

• threshold – 阈值函数

• cat

• exp

• stack

• conv_2d, conv_max_pool2d

在使用这些计算符时的格式，我们以 LIF 模型中计算化学电流的过程作为示例：

# [updated]符号目前代表该数值取的是本轮计算中计算出的新值，临时变量无需添加，
# Vtemp = V * tauM + I, 此处的tauM需要注意，因为tauM为 _tau_variables
self._operations.append(('Vtemp', 'var_linear', 'tauM', 'V', 'Isyn[updated]'))

# O = 1 if Vtemp >= Vth else 0， threshold起的作用为判断Vtemp是否达到阈值Vth
self._operations.append(('O', 'threshold', 'Vtemp', 'Vth'))

# 此处作用为在脉冲发放之后重置电压V
self._operations.append(('V', 'reset', 'Vtemp',  'O[updated]'))

在最后，需要添加 NeuronModel.register("lif", LIFModel) 用于将该神经元模型添加至神经元模型的库中。

突触、连接模型自定义

本章节主要介绍连接模型的自定义，以便当本平台提供的内置方法无法满足用户需求时，用户可以方便的添加符合自己需求的连接方案。

连接模型自定义

连接作为脉冲神经网络最为基本的组成结构之一，包含了网络最为重要的权重信息。不同的连接方法会生成不同的空间连接结构，为了满足用户的大多数应用需求，在本平台中内置了10种最常用的连接模型，包含了全连接，卷积连接，一对一连接，稀疏连接等。与此同时，作为类脑计算平台，本平台中的连接支持仿生连接的形式，即支持反馈连接与连接延迟以及突触连接等具有一定生理特征的连接方式。内置的连接方法可能无法满足用户的任意需求，这时候就需要用户自己添加一些更符合其实验目的的连接模型。定义连接模型的这一步可以依照 spaic.Network.Connection 文件中的格式进行添加。

连接方法初始化

自定义的连接方法需继承 Connection 类，其初始化方法中的参数名需与 Connection 类的一致，若需要传入初始化参数以外的参数，可以通过 kwargs 传入，以 FullConnection 类初始化函数为例：

def __init__(self, pre, post, name=None, link_type=('full', 'sparse_connect', 'conv','...'),
             syn_type=['basic_synapse'], max_delay=0, sparse_with_mask=False, pre_var_name='O', post_var_name='Isyn',
             syn_kwargs=None, **kwargs):
    super(FullConnection, self).__init__(pre=pre, post=post, name=name,
                                         link_type=link_type, syn_type=syn_type, max_delay=max_delay,
                                         sparse_with_mask=sparse_with_mask,
                                         pre_var_name=pre_var_name, post_var_name=post_var_name, syn_kwargs=syn_kwargs, **kwargs)
    self.weight = kwargs.get('weight', None)
    self.w_std = kwargs.get('w_std', 0.05)
    self.w_mean = kwargs.get('w_mean', 0.005)
    self.w_max = kwargs.get('w_max', None)
    self.w_min = kwargs.get('w_min', None)

    self.is_parameter = kwargs.get('is_parameter', True) # is_parameter以及is_sparse为后端使用的参数，用于确认该连接是否为可训练的以及是否为稀疏化存储的
    self.is_sparse = kwargs.get('is_sparse', False)

在这个初始化方法中， FullConnection 类所额外需要的参数，通过从 kwargs 中获取的方式来设定。

突触模型自定义部分

突触模型是进行神经动力学仿真环节中非常重要的一步，不同的模型与不同的参数都会产生不同的现象。为了应对用户不同的应用需求， SPAIC 内置了两种最常用的突触模型（化学突触和电突触），但是偶尔还是会有力所不能及，这时候就需要用户自己添加一些更符合其实验的个性化突触模型。定义突触的这一步可以参考 Network.Synapse 文件依照格式进行添加。

定义可从外部获取的参数

在定义神经元模型的最初部分，我们需要先定义该神经元模型可以变更的一些参数，这些参数可由传参来改变。例如在化学突触的一阶衰减模型中，我们将其原本的公式经过变换后可得：

class First_order_chemical_synapse(SynapseModel):
    """
    .. math:: Isyn(t) = weight * e^{-t/tau}
    """

在这个公式中，self.tau 是可变参数，所以我们通过 kwargs 中获取的方式来改变：

self._syn_tau_variables['tau[link]'] = kwargs.get('tau', 5.0)

定义变量

在定义变量阶段，我们要先了解突触的几个变量形式：

• _syn_tau_constant_variables – 指数衰减常数

• _syn_variables – 普通变量

对于 _syn_tau_constant_variables 我们会进行一个变换 value = np.exp(-self.dt / var) ,

在定义变量时，同时需要设定初始值，在网络的每一次运行后，神经元的参数都会被重置为此处设定的初始值。

self._syn_variables[I] = 0
self._syn_variables[WgtSum] = 0
self._syn_tau_constant_variables[tauP] = self.tau_p

定义计算式

计算式是突触模型最为重要的部分，一行一行的计算式决定了各个参数在模拟过程中将会经过一些什么样的变化。

在添加计算式时，有一些需要遵守的规则。首先，每一行只能计算一个特定的计算符，所以需要将原公式进行分解，分解为独立的计算符。目前在平台中内置的计算符可以参考 backend.basic_operation :

• add, minus, div

• var_mult, mat_mult, mat_mult_pre, sparse_mat_mult, reshape_mat_mult

• var_linear, mat_linear

• reduce_sum, mult_sum

• threshold

• cat

• exp

• stack

• conv_2d, conv_max_pool2d

在使用这些计算符时的格式，我们以化学突触模型中计算化学电流的过程作为示例：

# Isyn = O * weight 的公式转化为以下计算式并添加至self._syn_operations中，
# conn.post_var_name作为计算结果放置在第一位，
# 计算符mat_mult_weight放置在第二位，
# input_name以及weight[link]代表着计算的因子，放置于第三位及以后，
# [updated]符号目前代表该数值取的是本轮计算中计算出的新值，临时变量无需添加，
self._syn_operations.append(
    [conn.post_var_name + '[post]', 'mat_mult_weight', self.input_name,
     'weight[link]'])

算法自定义

替代梯度算法

在脉冲神经网络反向传播过程中，脉冲激活函数 \(output\_spike = sign(V>V_th)\) 的导数为：

\[\begin{split}\begin{array}{c} \delta (v) = \left\{\begin{matrix} +\infty, & v = v_{th} \\ 0, & v\ne v_{th} \end{matrix}\right. \end{array}\end{split}\]

显然，直接使用脉冲激活函数进行梯度下降会使得网络的训练及其不稳定，因此我们使用梯度替代算法近似脉冲激活函数。替代梯度算法最重要的过程即为使用用户自己的梯度函数替代计算图中原有的梯度函数，此处我们以 STCA 算法与 STBP 算法的差 别举例。两个不同算法，在平台上以 PyTorch 后端进行实现时，仅有的差别在于，

STCA 算法 [1] 中，梯度函数为:

\(h(V)=\frac{1}{\alpha}sign(|V-V_th|<\alpha)\)

STBP 算法 [2] 中我们选取的梯度函数为:

\(h_4(V)=\frac{1}{\sqrt{2\pi a_4}} e^{-\frac{(V-V_th)^2)}{2a_4}}\)

两中梯度替代算法的梯度替代函数和原始脉冲激活函数的对比如图：

以下代码块为替代梯度方程的前向和反向传播过程。

@staticmethod
def forward(
        ctx,
        input,
        thresh,
        alpha
):
    ctx.thresh = thresh
    ctx.alpha = alpha
    ctx.save_for_backward(input)
    output = input.gt(thresh).float()
    return output


@staticmethod
def backward(
        ctx,
        grad_output
    ):
    input, = ctx.saved_tensors
    grad_input = grad_output.clone()
    temp = abs(input - ctx.thresh) < ctx.alpha  # 根据STCA，采用了sign函数
    # temp = torch.exp(-(input - ctx.thresh) ** 2 / (2 * ctx.alpha)) \  # 根据STBP所用反传函数
    #                  / (2 * math.pi * ctx.alpha)
    result = grad_input * temp.float()
    return result, None, None

突触可塑性算法

赫布法则(Hebbian Rule)在关于神经元间突触形成的理论中表明，突触前和突触后的一对神经元的放电活动会影响二者间突触的强度，突触前后神经元放电的时间差决定了突触权值变化的方向和大小。这种基于突触前后神经元脉冲发放的时间差的权值调整方法被称为脉冲时间依赖可塑性(STDP)，属于无监督学习方法。在我们平台上实现了两种 STDP 学习算法，一种是基于全局突触可塑性的 STDP 学习算法 [3] ，一种是基于最邻近突触可塑性的 STDP 学习算法 [4] 。两种算法的区别在于突触前后脉冲迹的更新机制。此处以全局突触可塑性STDP算法为例。

全局突触可塑性STDP学习算法

该算法的权重更新公式 [2] 以及权重归一化公式：

\[\begin{split}dw &= Apost * (output\_spike * input\_trace) – Apre * (output\_trace * input\_spike) \\ weight &= weight + dw \\ weight &= self.w\_norm * weight/sum(torch.abs(weight))\end{split}\]

其中，全局突触可塑性STDP学习算法的突触前与突触后脉冲迹为：

\[\begin{split}input\_trace &= input\_spike + input\_trace * trace\_decay \\ output\_trace &= output\_spike + output\_trace * trace\_decay\end{split}\]

不同于此，基于最邻近突触可塑性的STDP学习算法在有脉冲发放时将相应的迹复位为1，其余时刻均衰减。其突触前与突触后脉冲迹为：

\[\begin{split}input\_trace &= input\_spike + (1 - input\_spike) * input\_trace * trace\_decay \\ output\_trace &= output\_spike + (1 - output\_spike) * output\_trace * trace\_decay\end{split}\]

首先从 trainable_connection 中获取该学习算法训练的突触前神经元组以及突触后神经元组

preg = conn.pre
postg = conn.post

之后获取学习算法需要的参数在后端的名称，例如：输入脉冲，输出脉冲，连接权重引用了 Connection 中的获取名字的函数

pre_name = conn.get_input_name(preg, postg)
post_name = conn.get_group_name(postg, 'O')
weight_name = conn.get_link_name(preg, postg, 'weight')

再将算法需要用到的参数添加到后端

self.variable_to_backend(input_trace_name, backend._variables[pre_name].shape, value=0.0)
self.variable_to_backend(output_trace_name, backend._variables[post_name].shape, value=0.0)
self.variable_to_backend(dw_name, backend._variables[weight_name].shape, value=0.0)

之后将运算公式添加进后端

self.op_to_backend('input_trace_temp', 'var_mult', [input_trace_name, 'trace_decay'])
self.op_to_backend(input_trace_name, 'add', [pre_name, 'input_trace_temp'])

self.op_to_backend('output_trace_temp', 'var_mult', [output_trace_name, 'trace_decay'])
self.op_to_backend(output_trace_name, 'add', [post_name, 'output_trace_temp'])

self.op_to_backend('pre_post_temp', 'mat_mult_pre', [post_name, input_trace_name+'[updated]'])
self.op_to_backend('pre_post', 'var_mult', ['Apost', 'pre_post_temp'])
self.op_to_backend('post_pre_temp', 'mat_mult_pre', [output_trace_name+'[updated]', pre_name])
self.op_to_backend('post_pre', 'var_mult', ['Apre', 'post_pre_temp'])
self.op_to_backend(dw_name, 'minus', ['pre_post', 'post_pre'])
self.op_to_backend(weight_name, self.full_online_stdp_weightupdate,[dw_name, weight_name])

权重更新代码：

with torch.no_grad():
    weight.add_(dw)

权重归一化代码：

weight[...] = (self.w_norm * torch.div(weight, torch.sum(torch.abs(weight), 1, keepdim=True)))
weight.clamp_(0.0, 1.0)

[1]

Pengjie Gu et al. “STCA: Spatio-Temporal Credit Assignment with Delayed Feedback in Deep SpikingNeural Networks.” In:Proceedings of the Twenty-Eighth International Joint Conference on Artificial Intelligence, IJCAI-19. International Joint Conferences on Artificial Intelligence Organization, July 2019,pp. 1366–1372. doi:10.24963/ijcai.2019/189.

[2] (1,2)

Yujie Wu et al. “Spatio-Temporal Backpropagation for Training High-Performance Spiking Neural Networks” Front. Neurosci., 23 May 2018 | doi:10.3389/fnins.2018.00331.

[3]

Sjöström J, Gerstner W. Spike-timing dependent plasticity[J]. Spike-timing dependent plasticity, 2010, 35(0): 0-0._

[4]

Gerstner W, Kempter R, van Hemmen JL, Wagner H. A neuronal learning rule for sub-millisecond temporal coding. Nature. 1996 Sep 5;383(6595):76-81. doi: 10.1038/383076a0. PMID: 8779718.

奖励调节的突触可塑性算法

奖励调节的突触可塑性算法可以看作为对正确或错误决策分别采取STDP/Anti-STDP学习机制 ，即用由神经网络的行为结果而产生的奖励或惩罚信号来对神经元的权重更新施加影响。在我们平台上实现了两种 RSTDP 学习算法，一种是基于资格迹的 RSTDP 学习算法 [5] ，一种是基于替代梯度的 RSTDP 学习算法 [6] 。下面以第一种算法为例。

基于资格迹的RSTDP学习算法

该算法的权重更新公式：

\[\begin{split}\begin{gather*} dw = lr * reward * eligibility\_trace \\ weight = weight + dw \end{gather*}\end{split}\]

其中，资格迹更新公式为：

\[\begin{split}input\_trace &= input\_trace * e^{-\frac{dt}{\tau_{+} } } \\ input\_trace &= input\_trace + input\_spike \\ output\_trace &= output\_trace * e^{-\frac{dt}{\tau_{-} } } \\ output\_trace &= output\_trace + output\_spike \\ eligibility\_trace &= Apre * input\_trace * output\_spike + Apost * output\_trace * input\_spike\end{split}\]

首先从 trainable_connection 中获取该学习算法训练的突触前神经元组以及突触后神经元组

preg = conn.pre
postg = conn.post

之后获取学习算法需要的参数在后端的名称，例如：输入脉冲，输出脉冲，连接权重，引用了 Connection 中的获取名字的函数，同时定义中间变量名，如突触前后脉冲迹和资格迹。

pre_name = conn.get_input_name(preg, postg)
post_name = conn.get_group_name(postg, 'O')
weight_name = conn.get_link_name(preg, postg, 'weight')
p_plus_name = pre_name + '_{p_plus}'
p_minus_name = post_name + '_{p_minus}'
eligibility_name = weight_name + '_{eligibility}'

再将算法需要用到的参数添加到后端

self.variable_to_backend(p_plus_name, pre_shape, value=0.0)
self.variable_to_backend(p_minus_name, backend._variables[post_name].shape, value=0.0)
self.variable_to_backend(eligibility_name, backend._variables[weight_name].shape, value=0.0)

之后将运算公式添加进后端

self.op_to_backend('p_plus_temp', 'var_mult', ['tau_plus', p_plus_name])
if len(pre_shape_temp) > 2 and len(pre_shape_temp) == 4:
    self.op_to_backend('pre_name_temp', 'feature_map_flatten', pre_name)
    self.op_to_backend(p_plus_name, 'var_linear', ['A_plus', 'pre_name_temp', 'p_plus_temp'])
else:
    self.op_to_backend(p_plus_name, 'var_linear', ['A_plus', pre_name, 'p_plus_temp'])

self.op_to_backend('p_minus_temp', 'var_mult', ['tau_minus', p_minus_name])
self.op_to_backend(p_minus_name, 'var_linear', ['A_minus', post_name, 'p_minus_temp'])

self.op_to_backend('post_permute', 'permute', [post_name, permute_name])
self.op_to_backend('pre_post', 'mat_mult', ['post_permute', p_plus_name + '[updated]'])

self.op_to_backend('p_minus_permute', 'permute', [p_minus_name + '[updated]', permute_name])
if len(pre_shape_temp) > 2 and len(pre_shape_temp) == 4:
    self.op_to_backend('post_pre', 'mat_mult', ['p_minus_permute', 'pre_name_temp'])
else:
    self.op_to_backend('post_pre', 'mat_mult', ['p_minus_permute', pre_name])
self.op_to_backend(eligibility_name, 'add', ['pre_post', 'post_pre'])
self.op_to_backend(weight_name, self.weight_update, [weight_name, eligibility_name, reward_name])

权重更新代码：

with torch.no_grad():
    weight.add_(dw)

[5]

Răzvan V. Florian; Reinforcement Learning Through Modulation of Spike-Timing-Dependent Synaptic Plasticity. Neural Comput 2007; 19 (6): 1468–1502. doi: https://doi.org/10.1162/neco.2007.19.6.1468

[6]

11. Stewart, G. Orchard, S. B. Shrestha and E. Neftci, “On-chip Few-shot Learning with Surrogate Gradient Descent on a Neuromorphic Processor,” 2020 2nd IEEE International Conference on Artificial Intelligence Circuits and Systems (AICAS), Genova, Italy, 2020, pp. 223-227, doi: 10.1109/AICAS48895.2020.9073948.

监视器

监视器主要的作用是监控网络运行过程中各类变量的变化过程，在SPAIC中，我们内置了两种形式的监视器，分别是 StateMonitor 与 SpikeMonitor 。

StateMonitor 与 SpikeMonitor 的建立方式相同， StateMonitor 是神经元及网络连接等的一般状态量的监视，而 SpikeMonitor 是针对脉冲发放频率的监视：

self.mon_V = spaic.StateMonitor(self.layer1, 'V')
self.mon_O = spaic.StateMonitor(self.input, 'O')
self.spk_O = spaic.SpikeMonitor(self.layer1, 'O')

在监视器初始化中，我们可以指定如下参数：

• target – 需要监视的对象，对于StateMonitor可以是NeuronGroup、Connection等任何包含变量的网络模块，对于SpikeMonitor一般是NeuronGroup、Encoder等具有脉冲发放的模块

• var_name – 需要监视的变量名，需要是监视对象具有的变量，比如神经元的膜电压 ‘V’

• index – 检测变量的索引值，例如一层神经集群中选择某几个神经元进行记录，可以使用 index=[1,3,4,…]，默认为整个变量全部记录

• dt – 监视器的采样时间间隔，默认与仿真步长相同

• get_grad – 是否需要记录梯度，True为需要梯度，False为不需要，默认为False

• nbatch – 是否需要记录多个Batch的数据，True则会保存多次run的数据，False则每次run覆盖数据，默认为False

在:code:`StateMonitor`和:code:`SpikeMonitor`两种监视器中具有如下通用的函数接口： - monitor_on – 将监视器设置为开启记录状态。监视器创建后默认是开启状态。 - monitor_off – 将监视器设置为关闭记录状态。 - clear – 清除监视器当前记录的所有数据。

这两个监视器的区别在于，StateMonitor 中存储了五个数据：

• nbatch_times – 将会存储所有批次的时间步信息，数据的shape结构为(第几批次，第几个时间步)

• nbatch_values – 将会存储所有批次的目标层的监视参数的情况，数据的shape结构为(第几批次，第几个神经元，第几个时间步，batch中的第几个样本)

• times – 将会存储当前批次的时间步信息，数据的shape结构为(第几个时间步)

• values – 将会存储当前批次的目标层的监视参数的变量，数据的shape结构为(本batch中第几个样本，第几个神经元，第几个时间步)

• tensor_values – 将会存储当前批次的目标层的监视的原Tensor变量，数据的shape结构为(本batch中第几个样本，第几个神经元，第几个时间步)

• grad – 将会存储当前批次的目标变量的梯度情况，数据的shape与values的shape结构相同

而 SpikeMonitor 中存储着另外四个数据：

• spk_index – 存储着当前批次脉冲发放的神经元的编号

• spk_times – 存储着当前批次脉冲发放的时刻信息

• time – 存储着当前批次的时间步的信息

• time_spk_rate – 存储着当前批次的目标层的瞬时发放频率

• spk_rate – 储存当前批次的目标层的平均发放率

• spk_count – 储存当前批次的目标层每个神经元的脉冲个数，数据结构为(本batch中第几个样本，第几个神经元）

使用示例:

time_line = Net.mon_V.times  # 取layer1层时间窗的坐标序号
value_line = Net.mon_V.values[0][0]  # 取本batch中layer1层第一个样本的第一个神经元的整个时间窗内的电压变化数据
input_line = Net.mon_O.values[0][0]  # 取本batch中input层第一个样本的第一个神经元的整个时间窗内的脉冲发放情况

# 由于初始化时nbatch为False，默认只有单个批次
output_line_index = Net.spk_O.spk_index[0] + 1.2  # 取本batch中第一个样本的脉冲发放的index信息，由于只有单个神经元，增加数值1.2调整脉冲点的位置
output_line_time = Net.spk_O.spk_times[0]  # 取本batch中第一个样本的脉冲发放的时刻信息

plt.subplot(2, 1, 1)
plt.title('Monitor Example Appearance')
plt.plot(time_line, value_line, label='V')
plt.scatter(output_line_time, output_line_index, s=40, c='r', label='Spike')

plt.ylabel("Membrane potential")
plt.ylim((-0.1, 1.5))
plt.legend()
plt.subplot(2, 1, 2)
plt.plot(time_line, input_line, label='input spike')
plt.xlabel("time")
plt.ylabel("Current")
plt.legend()

最后的结果如图所示:

联系我们：

洪朝飞：hongchf@zhejianglab.com

袁孟雯：yuanmw@zhejianglab.com

张梦骁：mxzhang@zhejianglab.com

文档索引

• Index

• Module Index

[1] (1,2,3,4)

Pengjie Gu et al. “STCA: Spatio-Temporal Credit Assignment with Delayed Feedback in Deep SpikingNeural Networks.” In:Proceedings of the Twenty-Eighth International Joint Conference on Artificial Intelligence, IJCAI-19. International Joint Conferences on Artificial Intelligence Organization, July 2019,pp. 1366–1372. doi:10.24963/ijcai.2019/189.