These are all drafts, by the way. They are not meant to be perfect or to convey all the information I wish to convey flawlessly. My blogs are just a way for me to get ideas and my thoughts realized as an initial development.
Meta-heuristic problems come in all shapes and sizes. They are all an attempt to solve some kind of non-convex optimization problem. Where, before the immediate optimization begins, some things about the nature of the problem are already assumed. Figuring out the optimal starting conditions is a meta-heuristic problem, because it exists outside and of relation to the immediate optimization problem. It is akin to developing a political perspective after already having had personal experiences and beliefs embedded in one’s psyche throughout life. That is to say that two people may have similar methods of attaining completely different political ideologies because of their prior understanding of the way things are. I like to think that the constant disagreeable tone in political discourse between people in every part of the world results from the difficulty for the individual to understand the world in a different way. Our personal beliefs are precious to us, but they are built off of an unquestionable basis of morality and beliefs instilled in us. The methods to attain personal beliefs may be universally reliable, but our prior beliefs can never be. In some way, the validity and righteousness of our prior beliefs is derived from how well they execute themselves after being manifested into personalized opinions and actions. The full expression of the efficacy of one’s prior beliefs results from how well an individual can use persuasive and reasoning skills to convince others of the value of their prior beliefs. In one light, the individual can be looked at as a perpetuator of the efficacy fo their prior beliefs. The individual acts on behalf their prior beliefs, to persuade others that their beliefs are valuable.
This is how evolutionary algorithms can be interpreted, although many evolutionary algorithms lack a necessary separation between immediate optimization and the evolutionary optimization. I hope that it is clear that by immediate optimization, I mean ‘doing the best with what one has’, and by evolutionary optimization I mean ‘finding the best to start out with’. These ideas are tangled and not completely separable because of the recursive relationship between ‘finding the best to start out with’ and ‘doing the best with what one start with’. The former requires a perfection of the techniques of the latter. In order to ‘find the best’ initially, there needs to be a reliable technique to determine what is best. That technique comes from the mastery of ‘doing the best with what one has’ - because if this technique is not reliable, we cannot know what is really a ‘better’ starting position than another. Similarly the latter requires a kind of perfection of the former. To determine how to ‘do the best with what one starts with’ one needs a basis for reasonable thought which is provided by a good place to start.
I want to try to formulate this mathematically. Let x represent the ‘way things really are’ - this is meant to be vague. Let f and g represent immediate and evolutionary methods of optimization, respectively. Let f* and g* represent the optimal methods of both. We wish to optimize both of these, to let f approach f* and to let g approach g*. g(x) produces a basis for interpreting the world and f(g(x)) (or f(basis)) produces an actualization and metric to determine how good g is with respect to x. If the last sentence were true, this optimization would be way easier than it actually is. In reality, the metric for g is with respect to x is given by f*(g(x)). In order to determine how good g is, f has to be optimal. In other words, in order to determine how good an initial condition (or prior belief) is, reasoning methods must be optimal. Easy! So all we have to do is optimize f - the method that determines how optimal g(x) is. Unless it’s not so easy, unless the optimality of f is somehow reliant on g’s optimality. Remember that f is the method of reasoning about g(x), a prior interpretation of an observation, and g is the method of developing such a prior interpretation. The question of “is f somehow reliant on g’s optimality” can be translated to: are our modes of reasoning about prior beliefs reliant on the optimality of those prior beliefs?”.Now this is where the complexity really lies, maybe it is that the efficacy of f cannot be determined by any old g, only by an optimal g. Mathematically, this may look like: f*(g(x)) gives the gradient of g, and f(g*(x)) gives the gradient of f. It may be clear now how complex this problem actually is, hopeless even to improve either g or f. It certainly wasn’t completely obvious to me that some of these meta-heuristic problems involve a kind of indescribable recursive complexity. In future blogs, I want to discuss methods of getting around this problem.