Deep networks use hidden layers, very loosely analogous to how our brains use abstraction in thought. The deep learning model was thought of by Rina Decter, and improved upon by people such as Yann LeCun and Geoffrey Hinton, both are co writers to the article I wrote an assessment about. Here is the link to my assessment over "Deep Learning": https://docs.google.com/document/d/19aeBsKurwbBlkMwz13E3bcqi2zmjTblboY_3aLUlksk/edit . You can think of deep learning in general as a loose model of the frontal cortex, what we are trying to extract from the idea of the frontal cortex, is for one, the ability to remember information and use that information usefully to solve complicated problems through multiple levels of analysis and abstraction of some input vector. Deep learning is to the frontal cortex, as a plane is to a bird. One is human made with problems, and one is the perfection of evolution, infeasible to copy and put in a machine, but we get closer every day to the right answer.
My only hope in the Machine Learning and deep learning field is to get us closer to that perfection of nature, isn't that the goal for everyone whether they know it or not? Tesla is trying to make a system that recognizes objects and is able to analyze and predict and be autonomous; who does this better than humans? What does this better than nature itself; through hundreds of millions of years of evolution be that? What about natural language recognition, or systems trying to use human languages to communicate with people? Think about Siri, the virtual assistant anyone who owns an iPhone is familiar with, ask yourself what are we trying to mimic in this system; language? hearing? speaking? These are all specifically, explicitly human. Maybe to the person reading this, this all seems kind of sidetracked and unnecessary to say all of this, but I implore you to think about this perspective when thinking about how to solve our problems in machine learning.
I believe that solving machine learning is going in the wrong direction, I think we need to think more about how our brain and psychology works when thinking about problems, because the problems we are trying to solve, such as natural language come from our evolutionary psychology and neurology, because we are human, evolutionary perfection. I had an idea about how to solve a problem when reading an article about a ML software that tries to recognize printed text, and the problem was that the pixel arrangements for the letters are different for different fonts, and it takes a lot of time for the machine to learn a whole different font every time they want to read. This reminded me of a time I was listening to Jordan Peterson, a clinical psychologist, talk about autism in kids and how it affected their perception of how the world works. A home with an autistic child must not change very often as to not upset the child, if the child comes home to the same living room with the same furniture every day she is ok and happy, but if one day you move all of the furniture and TVs and couches around, she will be very confused and be upset because she cannot understand that this is still her living room. She is unable to generalize across the set of all living rooms, and is unable to understand what the idea and concept of what a living room is and specifically apply it to her situation. I see the same problem with the font text recognition problem. One way or another I believe that we must engineer a machine that has the understanding of an abstract concept such as a number, and have letters be just a way to express that abstraction and for the machine to see the letters as a representation of that abstract concept. Right now, if you looked at the letter 'f', no matter which font that letter f is in, you would be able to know it is an f because you have the abstract concept of 'f' as a priority over simply analyzing the pixels.
We know how to create some type of abstraction in machines, but it is not the abstraction that I am going for. Picture a simple neural network with two hidden layers, the job of this network is to classify letters of the English alphabet from a 50 by 50 pixel screen and lots of training examples. In this type of classical network there would be 2,500 input neurons, 2 hidden layers with some set amount of neurons per layer and a final layer of 26 neurons, one for each letter of the alphabet. To start training this program you would give the network an input that is some handwritten letter of the alphabet, it will translate it into pixel values and that will be the input layer of neurons. We often call a "layer" in the network a vector, because all it really is, is a list of numbers. The first time the input vector is shown to the network it will make a completely random output, the network is trained over time to give correct answers by a learning technique called backpropagation or stochastic gradient decent (SGD), it basically minimizes the cost function of the network to produce the wanted answer. The internal weights and biases change based on the negative gradient of the derivatives of all of the internal parameters inside of all of the layers. One problem about this, is you cannot know if this is the best possible solution to the cost function, it is just one valley of the function that SGD finds the bottom of. For all neural networks, the physical abstractions of the input neurons are based on the output neurons. A really smart network might sort itself out in a way so the first hidden layer can recognize curves, straight lines, and loops; the second layer might sort itself out to recognize parts of a whole letter, and from the the third to last layer, it could figure out which parts of letters go together to create whole letters.
This does not solve the problem of generalization of an idea, but it does abstract the input neurons in a way that is helpful to it, even if the hidden layers do have no meaning. It is my hope to bring a solution to this problem in perusing my passion of deep learning.