What is an Echo State Network?
An Eco State Network is an architecture for supervised learning via a recurrent neural network (RNN). If you don't know what a neuro network is then the video below can provide a brief introduction to the concept:
A recurrent neural network is a neural network which has "neurons" which seed feedback signals to each other. The different types of RNNs include the binary, linear, continuous nonlinear, additive STM equation, shunting STM equation and finally generalized STM equation. Because the purpose of this blog post is to communicate on the utility of Echo State Networks, which is based on RNN, we will focus on that and not go into the details of RNNs.
The current review divides bRNNS into those in which feedback signals occur in neurons within a single processing layer, which occurs in networks for such diverse functional roles as storing spatial patterns in short-term memory, winner-take-all decision making, contrast enhancement and normalization, hill climbing, oscillations of multiple types (synchronous, traveling waves, chaotic), storing temporal sequences of events in working memory, and serial learning of lists; and those in which feedback signals occur between multiple processing layers, such as occurs when bottom-up adaptive filters activate learned recognition categories and top-down learned expectations focus attention on expected patterns of critical features and thereby modulate both types of learning.
A liquid state machine is described by Wikipedia as being a particular kind of spiking neural network:
In addition to neuronal and synaptic state, SNNs also incorporate the concept of time into their operating model. The idea is that neurons in the SNN do not fire at each propagation cycle (as it happens with typical multi-layer perceptron networks), but rather fire only when a membrane potential – an intrinsic quality of the neuron related to its membrane electrical charge – reaches a specific value. When a neuron fires, it generates a signal which travels to other neurons which, in turn, increase or decrease their potentials in accordance with this signal.
Echo State Networks can be useful for cryptography
In a recent paper titled: "Using Echo State Networks for Cryptography" it was discovered that you can do a novel cryptography scheme which works very simply:
- Alice and Bob share an Echo State Network.
- Alice trains her copy of the Echo State Network to memorize a message which she can then communicate the trained part of the network to Bob who can plug it into his copy to regenerate the message.
As we can see, an Echo State Network can have intriguing uses for cryptography.