We discretize simply to approximate functions. A multi-layer perceptron is an exquisite example of this. Approximating a phenomenon using a neural network can be viewed as discretizing a partial differentiable equation, an amazingly genius solution. A partial differentiable equation (PDE) describes a 3 or higher dimensional system of variables in terms of their derivatives, or higher order derivative. In the case of a neural network we update the parameters according to the gradient in the parameter space. We discretize with what we call a learning rate, meaning that there are some number of iterations that we preform to approximate the function with a higher accuracy. The smaller this learning rate is, the more accurately we can measure the function because there is a smaller discretion of the fluid unknown function. One way most people could visualize this is through the idea of approximating a value with Euler steps, a college level basic calculus class should cover this. The smaller the value the step is the more accurate it is. This Euler step method works for a single variable, an ordinary differentiable equation that has only the first derivative. For higher dimensions a gradient is known as a direction, which was to move all of the parameters to approximate a function.
Discretization exists both in the approximation over time through backpropagation, meaning the learning rate, or in the literal sense of neurons being in a network instead of some fluid model. It is to say that we are approximating some fluid learning model by discretizing it into neurons and connections between them. We can assume this because as we increase the number of neurons it performs more and more optimally for a given task. I like to image this as if we are sprinkling a discrete number of neurons on top of a higher truth about learning we would like to approximate, and the more neurons we sprinkle, the better we can approximate the surface because it covers more of it. And maybe there is some fluid model, but I believe we are not at a level of technology or science and its connection to philosophy to understand it. This is to say that we use better and better approximations, discretizations of an optimal learning algorithm to learn discretely (by taking a discrete number of steps) how to solve a problem.
If we think about artificial neural networks as discrete approximations created in the image of a more optimal higher truth about learning, what stops us from thinking of ourselves the same way? Our own minds are in fact discretized analogous to an artificial intelligence's. Who knows about a learning rate, that is specifically an AI thing as far as I am concerned, but the discrete nature of the neural structure itself remains similar. It is, I believe, a reasonable assumption to say that the neural structure in our brains structure themselves through the process of evolution to approximate some optimal function of dealing with the world. Understand this is not far away from a typical religious philosophy about our coming to be on this world. "We are created in the image of perfection, the image of God", quite literally our neural structure is discrete, and is trying to approximate a higher level of thinking.