Spiking Neural Networks
Issam Alameddine
1. Introduction
Spiking Neural Networks (SNN) is the key to connecting neuroscience and machine learning in a way that is biologically realistic. Unlike previous generations of Artificial Neural Networks (ANN), SNN do not fire at continuous values, but rather fire only when the post synaptic potential reaches a certain value. The generalized Integrate and Fire model and the Hodgkin-Huxley model, the neuron was regarded as a point and did not have any space. Actual biological neurons, however, do have dendrites that possess complicated spatial structures. Moreover, the input is injected explicitly in the form of a simulated current (usually through an electrode). This is not the case with biological neurons since they are embedded in a network where a neuron receives inputs from another neuron. These inputs will be received as a spike in the synapse and consequently induce an output at the post-synaptic side.
Fig 1.1 A: When a spike arrives, a neurotransmitter is released through the cleft and is received by the postsynaptic receptor. B: An AMPA receptor. Sodium and potassium ions can propagate to the membrane when glutamate is attached to the membrane. (Neural Dynamics, Cambridge University Press, 2014)
2. Functions and Types of Biological Synapses
At the top part of Fig. 1.1A is the pre-synaptic terminal. It contains vesicles that are loaded with neurotransmitters. Some sit inside and some on the surface. When an action potential travels along an axon and arrives in the form of a spike at the pre-synaptic neuron, some of the vesicles integrate with the membrane and eject neurotransmitters into the cleft (the space between the pre and post synaptic neurons). When the gate receptor (Figure 1.1B) captures a neurotransmitter, it opens, and allows the sodium ions to flow in, and the potassium ions to flow out of the cell. There are two types of synapses:
Excitatory: An important neurotransmitter at excitatory synapses is glutamate. When it attaches to an AMPA gate receptor, it causes it to open just enough to allow the sodium and potassium ions to flow, but small enough to block calcium for passing through. On the other hand, when it attaches to a NMDA (N-methyl-D-aspartate) channel, calcium can pass through. The dynamics of the NMDA is slower than the AMPA.
Inhibitory: An important neurotransmitter at inhibitory synapses is the GABA (gamma-aminobutyric acid). It comes in two different channel subtypes: GABA-A and GABA-B.
Fig 2.1 Post-synaptic current curves of GABA-A, GABA-B, AMPA and NMDA current with a spike at t=0 for a post synaptic potential of -65. (Neural Dynamics, Cambridge University Press, 2014)
If the spike arrives at an excitatory synapse with many AMPA channels, the current experiences a very rapid rise and decays within a few milliseconds. If the channel contains NMDA, the synaptic current the rise is a bit slower and decays within a few 100ms. For inhibitory synapses, one is fast an the other one is slower. The challenge is to build a model that describes this effective current that flows into the cell when it arrives at the presynaptic terminal.
Silicon Synapses : Analogue VLSI is predominantly used to design artificial neural networks. This is because transistors possess properties like those in nerve membrane channels. It is vital that the design consume very low power since a large network of thousands of neurons will increase power consumption very quickly. A common solution to this problem is to operate the transistors in the weak inversion region, also known as the subthreshold region, where the transistor leaks a very small current in the order of pico-amps to femto-amps. CMOS transistors are the most convenient to use in VLSI design. This is because the CMOS transistors consume low power during idle state where it only drains a very small leakage current, and only consumes current when it is switching state. Other transistors such as BJTs, continuously draw current. Although BJTs operate faster, low power consumption is usually desired. Moreover, CMOS fabrication technology allows the variation of channel dimensions, where length could be reduced to fit more transistors per unit area. A disadvantage that comes with reduced length is that the static power consumption will increase.
3. Leaky Fire and Integrate Model (LIF)
A very basic neuron circuit that fires and resets when the output reaches the voltage potential θ. Although very basic, this model is the basis of many neuron circuits and is fundamental in getting familiar with SNN circuits and their behaviour.
Fig. 3.1 LIF Model
\[\begin{equation} \tag{3.1} \tau\frac{d}{dt}u = -(u-u_{rest})+RI(t) \end{equation}\]
For the linear region, where the potential has not reached the value \(\theta\).
When the potential reaches \(\theta\), the output fires and resets as shown below:
Fig. 3.2 Spike fires and resets when threshold value is reached
4. Synapse Mathematical Models
Mathematical synaptic conductance models are to be developed to give an output synaptic current as shown previously in fig 2.1
\[\begin{equation} \tag{4.1} -I_{syn}(t) = -g_{syn}(t)(u-E_{syn})\tau⠀ \end{equation}\]
The total synaptic current has a drive which is the difference between the momentary voltage u and the reversal potential of the synapse \(E_syn\). The conductance \(g_{syn} (t)\) is time dependent and it can be thought of as an exponential pulse for convenience, which matches with the synaptic current output AMPA (fig 2.1):
\[\begin{equation} \tag{4.2a} g_{syn}(t) = \bar g_{syn}e^{\frac{t-t_{k}}{\tau} }\Theta(t-t_k) \end{equation}\]
It is this synaptic conductance that is inserted into 2.1, multiplied by the drive which gives the synaptic output. The model can be more realistic when the rise time can be modelled, because biological neurons have two different time constants, where the rising phase is usually faster than its decaying counterpart. A small modification of 2.3 to include the rise time gives:
\[\begin{equation} \tag{4.2b} g_{syn}(t) = \sum_{k} \bar ge^{\frac{t-t_{k}}{\tau}}(1-e^{\frac{t-t_{k}}{\tau_{rise}}})\Theta(t-t_k) \end{equation}\]
⠀ where the total synaptic conductance can be multiplied with the drive potential which gives the synaptic current. The synaptic current could be inserted in the Hodgkin-Huxley model for example:
\[\begin{equation} \tag{4.3} C\frac{du}{dt}= - g_{Na}m^3h(u_{E_{Na}})-g_l(u-E_1)+I_stim(t) \end{equation}\]
There are two different interpretations to this. One could say, that the total stimulating current \(I_stim (t)\) is the current given by 2.2 which means that the output from synaptic current such as the AMPA or NMDA is effectively a positive stimulating current. However, in a real neuron, there is no stimulating current, but channels. Where \(g_Na\),\(g_K\),\(g_l\) are the Sodium, Potassium and leak channel respectively, and \(I_stim (t)\) is simply an additional channel which naturally has a minus sign (eq. 2.2).
As discussed previously, glutamate produces an excitatory synapse, and GABA an inhibitory. Both are described by the same equation given in 2.2. However, the difference comes from the reversal potential where for excitatory synapses, it is high \(E_syn≈0mV\) and for inhibitory synapses, it is low \(E_syn≈-75mV\).
5. Fundamental Synapse Circuit Design
A basic, very simple synapse circuit is shown in the figure below. It converts a voltage pulse into a current whose magnitude is controlled by the weight voltage \(V_w\). Naturally, the current’s pulse width is same as the voltage input’s width.
Fig. 5.1 A simple synapse circuit design
However, input voltages can be very small in width and uncontrolled in which case, \(V_τ\) sets the time constant or pulse width of the current \(I_o\) as shown below:
Fig. 5.2 A simple synapse circuit with magnitude and width control
Fig. 5.3a Scaling the current output using two voltage controls
Typically, the transistors work in the subthreshold region as the exponential property is desired, as well as the very low power requirement. At the subthreshold region, (2.5) describes the operation of the synapse circuit,
\[\begin{equation} \tag{5.1} I_0 = I_i e^{\frac{k(V_1-V_2)}{2V_T} } \end{equation}\]
where \(V_T\) is the thermal voltage.
Scaling of the current using current controlsis done by using the following circuit:
Fig 5.3b Scaling the curent output using I1 and I2
In this case the scaling is described by:
\[\begin{equation} \tag{5.2} I_0 = I_i \sqrt(\frac{I_1}{I_2}) \end{equation}\]
The scaling property of the circuits in Fig. 5.3a,b can be implemented in synapse circuits where adaptation is required. An example of an adaptive circuit that uses the scaling circuit is shown in figure 5.4. The input of the circuit is the current \(I_i\) and the output is \(I_o\) which is a scaled version of \(I_i\). The supplementary input of the circuit (indicated with a pulse input to the gate of the transistor) controls the scaling of the output current to be larger than the input current. The If the supplementary input is not active, there is no scaling performed. The voltage \(V_l\) sets a limit to the scaling of the current. The supplementary input is used to produce learning algorithms, by substituting it with some logical combination of various inputs.
Fig. 5.4 An adaptation synapse circuit
6. Selected Model
6.1. Overview
There are many synapse circuits available and many of them use the current mirror synapse that is simple but not very suitable since it suffers from problems regarding gain transfer, tau and various other parameters. The model presented by Horiuchi, uses exponential decay using log domain filtering which causes the circuit to behave as an LTI system.
The CMOS synapse presented by Horiuchi and Shi, is a circuit where the leak current, synaptic gain and the decaying time constant can be independently controlled. This article describes the circuit’s function, along with analysis and simulation results.
The model provides separate control of the leakage current, synaptic weight and the decaying time constant. It uses the current feedback to obtain a first order behavior. The circuit consists of five PMOS transistors and three NMOS transistors and a capacitor as shown in figure 6.1.1.
Fig. 6.1.1 CMOS synapse circuit model
6.2. Circuit Description
The action potential is modeled as an input to the synapse circuit as an inverted logic narrow width pulse into the gate of transistor \(M_1\). determines the current that goes through transistor \(M_7\). This current,\(I_\tau\) , is what determines the decaying time constant of the output. \(V_w\) determines the magnitude of the synapse (the synaptic weight). \(M_6\) converts the voltage across the capacitor into a current. The current mirror formed by \(M_4\) and \(M_5\) receive the current that is sent through from \(M_6\) and pass it through the source follower that is formed by \(M_3\) and \(M_4\). The drain current of \(M_8\) is the inhibitory synaptic current output. Bulk terminals of all NMOS transistors are connected to ground and bulk terminals of all PMOS transistors are connected to \(V_{dd}\) except for \(M_3\).
6.3. Circuit Analysis
In figure 6.1.1, the transistors are operating at the subthreshold level (weak inversion region). At this mode of operation, the current-voltage relationship is exponential and is best described by the drain current equation:
\[\begin{equation} \tag{6.3a} i_{ds} = S_nI_{0n}e^{\frac{1-k_nv_{gs}}{v_T}}e^{ \frac{(1-k_n)v_{bs}}{v_T} }(1-e^{\frac{-v_{ds}}{v_T}}) \end{equation}\]
but
\[\begin{equation} \tag{6.3b} i_4 = \frac{S_4i}{S_5} \end{equation}\]
\[\begin{equation} \tag{6.3c} i_4 = s_3I_{op}e^{\frac{kp(V_w-V)}{v_T}}(1-e^{-\frac{(-V_3)}{V_T})} \end{equation}\]
But \(V_{bs}=0\) and \(V_0 > 4V_T\) (saturation). For \(M_3\):
\[\begin{equation} \tag{6.3d} i_4 = S_3I_{op}[e^(V_w-V)\frac{k_p}{V_T}] \end{equation}\]
\[\begin{equation} \tag{6.3e} \frac{iS_4}{S_3S_5I_{op}}=e^{V_w-V}\frac{k_p}{V_T} \end{equation}\]
\[\begin{equation} \tag{6.3f} \ln(\frac{iS_4}{S_3S_5I_{op}}) = (V_w-V)\frac{k_p}{V_T} \end{equation}\]
\[\begin{equation} \tag{6.3g} V = V_w + \frac{V_T}{k_p}\ln{(\frac{iS_4}{S_3S_5I_op})} \end{equation}\]
\[\begin{equation} \tag{6.3h} e^{V^{\frac{V_T}{k_p}}} = \frac{iS_4}{S_3S_5I_{op}}e^{V_w\frac{k_p}{V_T}} \end{equation}\]
\[\begin{equation} \tag{6.3i} i_2 = S_2I_{op}e^{\frac{k_p}{V_T}(V_{dd}-V)} \end{equation}\]
but
\[\begin{equation} \tag{6.3j} i = S_6I_{on}e^{\frac{k_nV_c(t)}{V_T}} \end{equation}\]
\[\begin{equation} \tag{6.3k} e^{-V\frac{V_T}{k_p}}= \frac{S_3S_5I_{op}}{S_4S_6I_{on}}e^{-V_w\frac{k_p}{V_T}}e^{\frac{k_n}{V_T}V_c(t)} \end{equation}\]
\[\begin{equation} \tag{6.3l} i_2 = \frac{S_2S_3S_5I_{op}^2}{S_4S_6I_{on}}e^{\frac{k_p}{V_T}(V_{dd}-V_w)} \end{equation}\]
Using Kirchoff’s current law where:
\[\begin{equation} \tag{6.3l} i_c = i_2 - i_\tau \end{equation}\]
where
\[\begin{equation} \tag{6.3m} i_c = C\frac{dV_c(t)}{dt} \end{equation}\]
gives
\[\begin{equation} \tag{6.3m} C\frac{dV_c(t)}{dt} = \frac{S_2S_3S_5I_{op}^2}{S_4S_6I_on}e^{\frac{k_p}{V_T}(V_{dd}-V_w)}-I_\tau \end{equation}\]
\[\begin{equation} \tag{6.3n} I_\tau = S_7I_{on}e^{\frac{V_\tau}{V_T}k_n}, since, V_{b3}=0, and, V_{d3}>4V_T \end{equation}\]
\[\begin{equation} \tag{6.3o} i_{syn}(t) = S_8I_{on}e^{\frac{k_n}{V_t}v_c(t)} \end{equation}\]
\[\begin{equation} \tag{6.3p} \frac{d_{i_{syn}}}{dt} = S_8I_{on}\frac{k_n}{V_T}[\frac{S_2S_3S_5I_{op}^2}{S_4S_6I_{on}C}e^{\frac{k_p(V_{dd}-V_w)}{V_T}}e^{\frac{-k_nV_c(t)}{V_T}}\frac{I_\tau}{C}]e^{\frac{k_n}{V_T}V_c(t)} \end{equation}\]
\[\begin{equation} \tag{6.3q} \frac{d_{i_{syn}}(t)}{dt}+\frac{k_nI_\tau}{CV_T}i_{syn(t)} = \frac{S_8S_2S_3S_5k_nI_{op}^2}{S_4S_6CV_T}e^{\frac{k_p}{V_T}(V_{dd}-V_w)} \end{equation}\]
\[\begin{equation} \tag{6.3r} \tau = \frac{CV_T}{K_nI_\tau} \end{equation}\]
but
\[\begin{equation} \tag{6.3s} i_{syn}(0) = S_8I_{on}e^{\frac{k_n}{V_T}V_c(0)} \end{equation}\]
\[\begin{equation} \tag{6.3t} d_{i_{syn}}(\infty)+\frac{k_nI_\tau}{CV_T}i_{syn}(\infty) = \frac{S_2S_3S_5S_8k_nI_{op}^2}{S_4S_6CV_T}e^{\frac{k_p}{V_T}(V_{dd}-V_c)} \end{equation}\]
but
\[\begin{equation} \tag{6.3u} \lim_{t \to\infty}\frac{d_{i_{syn}}(t)}{d_t}=0 \end{equation}\]
\[\begin{equation} \tag{6.3v} i_{syn}(\infty) = \frac{S_2S_3S_5S_8k_nI_{op}^2}{S_4S_6I_2}e^{\frac{k_p}{V_T}(V_{dd}-V_w)} \end{equation}\]
\[\begin{equation} \tag{6.3w} i_{syn}(t) = i_{syn}(t_{pw})e^{\frac{-(t-t_{pw})}{\tau}} \end{equation}\]
where the leakage current is ignored when \(t>t_{pw}\)
7. Testing and Simulation
7.1. Requirements
This project designs a synapse circuit and therefore completely relies on software simulation. The LTspice electronic simulator software was used. It’s a free software that uses the spice simulator. It provides various analysis options such as DC and transient that were used in designing the circuit. Many different transistors are available in the LTspice library where the parameters can be changed. The transistors’ parameters used are listed in the table below. A pulse type input voltage source is used as an input to the circuit. LTspice allows the modification of the period, duty cycle, rise time, fall time, delay and number of cycles all of which are set accordingly to the circuit’s specific requirements.
NMOS parameter values
## Parameter PMOS NMOS
## 1 Rg 3 3
## 2 Rd 14m 4.8m
## 3 Rs 10m 3.6m
## 4 Vto -0.8 0.8
## 5 Kp 32 0.7
## 6 Cgd (max) 0.5n 0.7n
## 7 Cgd (min) 0.07n 0.25n
## 8 Cgs 0.9n 1n
## 9 Cjo 0.26n 0.36n
## 10 Is 26p 0.1u
## 11 Rb 17m 6m
## 12 Vds -20 20
## 13 Ron 34m 12m
## 14 Qg 13n 18nDimension requirements
## Part Number Length Width
## 1 M1 1.6u 2.4u
## 2 M2 4u 8u
## 3 M3 4u 10u
## 4 M4 4u 4u
## 5 M5 4u 4u
## 6 M6 4u 4u
## 7 M7 4u 4u7.2.Subthreshold Characteristics
The transistors are operating at the subthreshold level (weak inversion region). At this mode of operation, the current-voltage relationship is exponential and is best described by (6.3a) where \(V_T=\frac{kT}{q}\) is the thermal voltage and \(S=\frac{W_n}{L_n}\). In order to find the technology current \(I_{on}\), the gate and source terminals are grounded and the bulk is shorted to the source so that \(v_{bs} = v_{qs}=0\), and the imensions are set to \(W=L=10\mu\) so that \(S=1\). This simplifies (6.3a) to:
\[\begin{equation} \tag{7.2.1} i_{ds} = I_{on}(1-e^{\frac{-V_{ds}}{V_\tau}}) \end{equation}\]
The temperature was set to \(300K\) which makes \(V_T \approx 0.103V\) the exponential approaches zero and therefore
\[\begin{equation} \tag{7.2.2} i_{ds} \approx I_{on}\space when\space V_{ds} > 0.103V\space at\space 300K \end{equation}\]
Fig. 7.2.1 Finding the technology current for the NMOS. \(I_{on} \approx 1nA\) when $ V_T > 0.1V$
<>
Simulations were carried based on the discussion above to find the technology current of the PMOS and NMOS transistors. As shown in the figures below \(I_{on}\approx1nA\) and \(I_{op} \approx 18nA\).
Fig. 7.2.2 Finding the technology current for the PMOS
<>
7.3. Synaptic Strength and Time Constant
The leaking current that is going through \(M_7\) determines the decaying time constant of the synaptic output. Tuning the voltage \(V_\tau\) achieves this. The circuit simulated on LTspice is shown below:
Fig. 7.3.1 The circuit simulated on LTspice
<>
The synaptic output taken from the drain of \(M_6\) along with the input potential is shown in the following figure.
Fig. 7.3.2a Simulation showing the synapse current output and pulse input for \(V_\tau =0.2\)
<>
The figure below shows the voltage waveform across the capacitor \(V_c(t)\)
Fig. 7.3.2b Output waveform \(V_c(t)\)
<>
In order to test the results above for correctness, a sample calculation is performed using (6.3o) to find the value of the current output \(i_{syn}(t)\), where \(S_8=1\) and \(k_n=0.7\), \(I_{on}=1nA\) and \(v_c(t)|_{t=3.117s}=635.mV\) (fig. 7.3.2b). Substituting these values give:
\[ \tag{7.3.1} i_{syn}(t) = 10^{-9}e^{\frac{0.7(0.63555862)}{0.02586492} }\approx 0.295\mu A \]
Which matches closely with \(0.299\mu A\) from figure 7.3.2a
The following two figures show how the decaying time constant changes proportionally with \(V_\tau\) while keeping \(V_w\) constant at 3.75 volts (pulse width set at 10ms)
Fig. 7.3.3a Synaptic current output for \(V_\tau = 0.23\)
Fig. 7.3.3b Synaptic current output for \(V_\tau = 0.25\)
As it can be seen from figures 7.3.2a, 7.3.3a, 7.3.3b and 6.7 \(V_\tau\) directly effects the time constant. It is noticed that the magnitude slightly increased as well.
The voltage \(V_w\) controls the synapse gain. In the figures below, \(V_\tau\) is held constant at 0.25V and \(V_w\) is changed. The magnitude (gain) decreases as the weight voltage increases. Observe that the time constant did not change throughout.
Fig. 7.3.4a \(V_w = 3.6V\)
Fig. 7.3.4b \(V_w = 3.7V\)
Fig. 7.3.4c \(V_w = 3.8V\)
7.4. Spike Train Response
The figure below illustrates the summing characteristic and saturation of the incoming spikes. Pulse widths are set at 2ms.
Fig. 7.4.1 Input (blue), synapse output (green)
8. Nonidealities and Design Optimization
8.1 Towards a True First Order System
There were two assumptions that simplified the circuit analysis of the proposed design. First, all transistors were assumed to be operating at the subthreshold region. This is not the case since transistor \(M_7\) operates momentarily in the triode region when \(v_c(t) < 4V_T \approx 0.103V\) which makes the synapse current response faster than anticipated.
Second, all parasitic capacitances were ignored which can greatly impact the circuit’s output at low currents because of a slow feedback response. If \(v_c(t)\approx 0\) and an input potential arrives, the transistors’ transconductances of the feedback loop are typically in the order of \(10^{-12}\) to \(10^{−15}\) which causes the loop to respond with delay. This causes the voltage across the capacitor and consequently the synaptic output to no longer be a true first order system. In order to solve this problem, a small bias voltage at the source of \(M_7\) is added so that the minimum value of \(v_c(t)\) is the bias voltage. This means that, when there is no input potential, the synaptic output current will have a small leaking current value. This can be fixed if this characteristic is not desired: the source of M6 could be directly connected to the bias voltage. The following schematic shows the modified circuit design.
Fig. 8.1.1 Modified circuit after the addition of a small bias voltage \(V_s\)
\[\begin{equation} \tag{8.1.1} i_{syn}(t) = S_8 I_{on}e^{\frac{k_nv_c(0)}{V_T}} + \frac{S_2S_3S_5S_8I_{op}^2}{S_4S_6I_\tau}e^{\frac{k_p(v_{dd}-v_w)}{V_T}}(1-e^{-\frac{t-t_j}{\tau}}) \end{equation}\]
However, the inclusion of the bias voltage controlled the quiescent current value as shown in the figure below.
Fig. 8.1.2 Inclusion of the bias current. When the capacitor fully discharges, it discharges down to 100mV in agreement with the bias voltage value. Period is exaggerated to show a fully discharged capacitor.
Fig. 8.1.3 Synapse output with true exponential rise and decay
When the bias voltage \(V_s\) is set to 100mV the synaptic current has risen and decayed exponentially which ensures no delay in the feedback loop which leads to a more accurate description of the first order dynamic characteristic desired.
8.2 Biologically Accurate: Two Different Time Constants
As discussed earlier, the current through M7 is the only parameter that determines the time constant for the synapse output. However, biological synapses have two different time constants where phase is usually much faster than its decaying counterpart. A single time constant is especially a problem in SNN where the pulse width of the spike is very narrow in comparison to the spike period. The following circuit design enables independent control of the rising and falling time constants.
Fig. 8.2.1 Modified circuit for independent rising and falling time constant control
A parallel current branch to \(M_7\) is formed by \(M_8\) and \(M_9\). This branch provides a current \(I_{Tr}+I_\tau\). When there is no input, \(M_8\) and \(M_9\) becomes an open circuit and the operation is like that of figure 8.1.1. If the current through M7 increases, \(i_{syn}(\infty)\) decreases because the added current branch decreases the current that charges the capacitor. The figures below illustrate demonstrate the independent control of the rising and falling time constants.
The decaying time constant is given by: \[
\tag{8.2.1}
\tau_d = \frac{CV_T}{k_nI_\tau}
\]
The rising time constant is given by:
\[ \tag{8.2.2} \tau_r = \frac{CV_T}{k_n(I_{\tau r}+I_\tau)} \]
Fig. 8.2.2 Effect of changing \(V_{t2}\) on the decaying time constant where (from top to bottom): \(V_{t2} = 0.1\), \(V_{t2}=0.25\), \(V_{t2}=0.4\). Input to \(M_1\) (blue), and input to \(M_9\) (red)
Fig. 8.2.3 Keeping \(V_{t2}\) constant and varying \(V_{t1}\) (from top to bottom respectively): 0.3, 0.33, 0.36
9. Conclusion and Future work
This article presented a CMOS synapse circuit model for SNNs. The design ensured that all transistors are working at the subthreshold level. In the model provided by Horiuchi and Shi, 2004, transistor \(M_7\) operates in the triode region when the voltage \(v_c(t)\) falls below \(4V_T\) which is approximately 100mV. The addition of the small bias voltage ensured that the voltage across the capacitor when it is fully discharged did not fall below 100mV. In the original circuit, the current through \(M_7\) was the only determinant factor to control the time constant. An additional branch parallel to \(M_7\) with a switching circuitry provided additional control where the rise and fall of the synaptic output are easily controlled. The circuit simulations were successful and matches with the circuit analysis and numerical calculations.
The synapse circuit is useful in integration into large neural networks. It can be implemented in STDP circuits in which the author is currently working on. The design could be further optimized because it best works for pulse widths that are in milliseconds but spikes could be much smaller. In principle, this could be overcomed by using a different way than current loop feedback, or by simply using a pulse extender circuit that is attached to it. Moreover, the switching transient effects must be optimized because they can hugely alter the circuit’s intended behavior.
10. References
Afifi, A., Ayatollahi, A. and Raissi, F. (2009). Implementation of biologically plausible spiking neural network models on the memristor crossbar-based CMOS/nano circuits. 2009 European Conference on Circuit Theory and Design.
Gerstner, W., Kistler, W., Naud, R. and Paninski, L. (n.d.). Neuronal dynamics.
Hong, Q., Zhao, L. and Wang, X. (2019). Novel circuit designs of memristor synapse and neuron. Neurocomputing, 330, pp. 11-16.
Indiveri, G., Chicca, E. and Douglas, R. (2006). A VLSI. A VLSI Array of Low-Power Spiking Neurons and Bistable Synapses With Spike-Timing Dependent Plasticity. IEEE Transactions on Neural Networks, 17(1), pp. 211-221.
Izhikevich, E. (2003). Simple model of spiking neurons. IEEE Transactions on Neural Networks, 14(6), pp. 1569-1572.
Kazuki Nakada, Tetsuya Asai and Yoshihito Amemiya (n.d.). Analog CMOS Implementation of a Bursting Oscillator with Depressing Synapse. Proceedings of the 2004 Intelligent Sensors, Sensor Networks and Information Processing Conference, 2004.
Kim, T., Oh, M., Kwon, M. and Park, B. (2018). Integrate-and-fire spiking neuron circuit exhibiting spike-triggered adaptation through input current modulation with back gate effect. Electronics Letters, 54(17), pp. 1022-1024.
Koickal, T., Hamilton, A., Tan, S., Covington, J., Gardner, J. and Pearce, T. (2007). Analog VLSI Circuit Implementation of an Adaptive Neuromorphic Olfocation Chip. IEEE Transactions on Circuits and Systems I: Regular Papers, 54(1), pp. 60-73.
Lazzaro, J. and Wawrzynek, J. (1994). Low-Power Silicon Neurons, Axons and Synapses. Silicon Implementation of Pulse Coded Neural Networks, pp. 153-164.
Li, M., Xiong, L., Li, Y., Li, K. and Wu, L. (2018). A switched-current synapse circuit for spiking neural networks based on compact structure and TLBO Algorithm. 2018 International Conference on Electronics Technology (ICET).
Liu, S. (2003). Analog VLSI Circuits for Short-Term Dynamic Synapses. EURASIP Journal on Advances in Signal Processing, 2003(7).
Park, Y., Liaw, J., Sheu, B. and Berger, T. (2000). Compact VLSI neural network circuit with high-capacity dynamic synapses. Proceedings of the IEEE-INNS-ENNS International Joint Conference on Neural Networks. IJCNN 2000. Neural Computing: New Challenges and Perspectives for the New Millennium.
Rahimi Azghadi, M., Iannella, N., Al-Sarawi, S., Indiveri, G. and Abbott, D. (2014). Spike-Based Synaptic Plasticity in Silicon: Design, Implementation, Application, and Challenges. Proceedings of the IEEE, 102(5), pp. 717-737.
Shi, R.Z., & Horiuchi, T.K. (2004). A Summating, Exponentially-Decaying CMOS Synapse for Spiking Neural Systems. NIPS.
Tete, A., Deshmukh, A., Bajaj, P. and Keskar, A. (2010). Design of Dynamic Synapse Circuits with VLSI Design Approach. 2010 3rd International Conference on Emerging Trends in Engineering and Technology.
Tsividis, Y. (1997). Externally linear, time-invariant systems and their application to companding signal processors. IEEE Transactions on Circuits and Systems II: Analog and Digital Signal Processing, 44(2), pp. 65-85.
Watts, L. (1993). Designing Networks of Spiking Silicon Neurons and Synapses. Computation and Neural Systems, pp. 127-131.
Wijekoon, J. and Dudek, P. (2007). Spiking and Bursting Firing Patterns of a Compact VLRSI Cortical Neuron Circuit. 2007 International Joint Conference on Neural Networks.
Yajie Chen, McDaid, L., Hall, S. and Kelly, P. (2008). A programmable facilitating synapse device. 2008 IEEE International Joint Conference on Neural Networks (IEEE World Congress on Computational Intelligence).
Yakopcic, C., McLean, M., Taha, T., Hasan, R. and Palmer, D. (2014). Memristor-based neuron circuit and method for applying learning algorithm in SPICE. Electronics Letters, 50(7), pp. 492-494.
Zohora, F., Debnath, S. and Rashid, A. (2019). Memristor-CMOS Hybrid Implementation of Leaky Integrate and Fire Neuron Model. 2019 International Conference on Electrical, Computer and Communication Engineering (EECE).