Why don't everyone poke through the essence of deep learning?
Human beings are slowly approaching the essence of the world - matter is only the carrier of the information model. Organs outside the human brain are just the support fleet that guarantees this mission.
Since AlphaGo finished the abuse of Li Shizhen last year, the deep learning has gone hot. But no one seems to be able to explain its principle, just use it as a black box. Some people say that deep learning is a nonlinear classifier? Some people say that deep learning is a simulation of the human brain... but I don't think it's poke through the essence.
After reading Jeff Hawkins's "On Intelligence," "It's it!". I was surprised to find that the original book was published in 2004! I am afraid that I have read this fake book, or the work of a certain American amateur scientist. I specifically went to google it and found that almost everyone who had read this book praised its theory. But the strange thing is that no one seems to be willing to stand for it. The influence of this theory stops here. It seems that everyone is deliberately concealing this secret. It clearly explains the operation mechanism of human brain intelligence completely! Please note is the Real Intelligence, not just Artificial Intelligence!!!
Three insights
The author's insights are much simpler and much more profound than most brain science papers:
For a long time, because we have no ability to observe thinking from the inside, people always equate "intelligence" with "expressive behavior." But when we read the book, there is no change in the eyes of outsiders, and we know that there have been countless associations, epiphanies, and memories during this period. Therefore, "understanding" cannot be measured by external behavior, it is an indicator of intrinsic metrics.
From paramecium to humans, nature will design a set of intelligent mechanisms for each kind of organism, or follow a set of mechanisms, or start a new intelligent mechanism from a certain generation, and continue to use it today (then, what species is the first to generate this intelligent mechanism?)? What kind of intelligence we are talking about is unique to human beings, or is it a universal feature of living things (just how different)? The author believes that intelligence cannot be designed by God for human beings. It must come from some customary tricks of nature.
The cerebral cortex, both structurally and functionally, has the same structure/mechanism (strictly speaking, this is not the author's insight, but was discovered by Vernon Mountcastle as early as 1978).
Starting from these three insights, it naturally leads to the following questions:
If intelligence is not defined by behavior, how do you define it?
Looking forward, how does intelligence evolve?
Looking inward, how does the structure of the cerebral cortex capture the structure of the world?
Simply put, the author's conclusion is:
Intelligence is not as fascinating as people think, it is just a "predictive ability of the future."
The essence of these predictions is nothing more than a by-product of "biological stress" under the "biological self-balancing mechanism" & "environmental pressure".
The core of intelligence is something that is "stable." This is due to the homogenous hierarchical structure of the cerebral cortex.
Below, let's take a look at how the author guessed the essence of intelligence step by step from the three simple insights.
Why is "prediction" the basis of intelligence?
Normally, people understand that the "predictive" step is too big. It is like calculating the placement point accurately from a serve. The "predict" of the human brain is more like "stress". A little bit of fine-tuning. The modern society has developed too fast, so that we can not see the historical appearance of the concept, and thus it is more likely to be confused by the fog of appearance. When we walked back to the beginning of history, the fog naturally dispersed. What is the biggest benefit of intelligence? It is not creating anything, but living. Human beings are entangled in "survival" or "development" all the time. But few people have seen that development is just to deal with unknown survival challenges.
How should we define intelligence? Perhaps the history of evolution can tell us more. Intelligence is an ability to help humans survive: it is the ability to let us fish in the stream to swim, so that we can judge whether it is a friend or a beast with only a vague image... It is necessary to study problems such as "how to maintain balance", not what is the problem of ballistic solution. It is not the evolutionary goal of nature, and naturally there is no brain mechanism.
All survival problems can be attributed to a meta-question: how to identify those constant things in the problem. For example: the fish in the stream, the direction of going home... If there are other elements in the intelligence, such as: imagination, creating tools, solving problems, you can stipulate some abstract means. In the final analysis, there is only one way for humans to solve all problems—using abstraction to reconcile contradictions in a higher dimension.
Everything cannot be separated from "invariant representations."
Abstract essence
Just as people recognize the concept of "negative number", they can finally unify the two operations of "addition" and "subtraction" completely differently (one increase, one decrease) into "addition on the integer field". . Reconciling contradictions from a higher dimension is exactly how the cerebral cortex is constructed and how it works. Constantly find common ground in the phenomenon, extract it, and take a name; these names become the cornerstone of the upper layer of abstraction (or "vocabulary"). This layer by layer until you get the smart holy grail - a constant representation.
For example, how do we identify the edge?
Let's first examine a small 3×3 retina, labeled #1～#9 (as shown below). When a vertical line appears (#1, #4, #7 are activated), the electrical signal is passed to the second layer. Each neuron in the second layer responds to the activation of a group of cells on the retina, respectively. For example, the leftmost neuron in the second layer responds to the activation of a single retinal cell. Another example: the second layer of the second layer of neurons, in response to the activation of any two retinal cells. And so on...
Edge recognition: the lowermost layer is the retinal cells; when a retinal cell combination is activated, it activates the corresponding neurons in the upper layer; and when a certain combination of the upper neurons is activated, it is activated chainwise. Higher-level neurons If we take the time factor into account, we assume that the signal does not disappear immediately, but decays over time, so long as the time is short enough, input (#1, #4, #7), ( #2, #5, #8), (#3, #6, #9) These three groups of stimuli will activate a neuron on the third level, which means "discover a vertical line."
Look, in fact, every neuron is a "word" (or "concept" / "abstract" / "feature"). However, the "words" described by the lower-level neurons are less abstract. For example, the second layer of the #(1, 4, 7) neuron represents "a vertical line appears on the far left of the retina", while the upper layer does not have the constraint "on the far left of the retina".
Memory role
Neurons can complete the collection-integration-output of information in 5 milliseconds, which is equivalent to an operation speed of 200 times per second. Humans can recognize images and make choices in half a second (equivalent to 100 steps)... 100 steps, the machine can't do it. In the algorithm known to man, perhaps only "table" (the answer is stored in memory in advance, not used for calculation, but only extraction) can be done. Therefore, the entire cerebral cortex is a memory system, not a computer.
What is the opposite of deep learning?
A multi-layer network that provides a layer-by-layer abstraction of channels. Nowadays, image recognition systems do exactly this: the bottom layer recognizes the edges, then recognizes the specific shape, and then recognizes certain features at a high level...
Convolution provides a means of obtaining a "constant representation."
What else do we not know?
When we want to extract a certain piece of memory, we usually only need to speak a few words. In other words, memory seems to be stored in a holographic form. Any clip contains all of it.
Also, we still don't know how the brain completes the decision in 100 steps. We don't know why there are so many feedback connections? Axon v.s. What is the difference in function of dendrites? ......
Now let's go back and look at the author's three insights, and repeat it in slang:
Understanding is an internal measure of how the brain forms memories and uses them to make predictions.
Forecasting is a by-product of some sort of self-regulating mechanism.
There is a striking homogeneity in the appearance and structure of the cerebral cortex. In other words, the cerebral cortex uses the same calculations to perform all its functions. All the intelligence that humans display (visual, auditory, physical movements...) is based on a unified set of algorithms.