108 Chapter 5. Model-based Object Recognition
able to solve the problem when a sufficient amount of time is given. EMOPaT has the ability to find, among all these possible solutions, the ones that are also able to reach a good fitness as fast as possible, because it also optimizes the other objective, that is not taken into consideration in a single-objective optimization or is difficult to consider during a manual tuning. Figure5.17shows the same results considering translation and rotation errors instead of fitness.
Figure 5.17: Results of the four PSOs with different time budgets.
Chapter 6
Conclusions
An expert is a man who has made all the mistakes which can be made in a very narrow field – Niels Bohr
Several topics have been addressed in this thesis. It begins, in Chapter 2, with a description of several methods to automatically perform tuning and selection of the parameters of a metaheuristic of interest. Meta-Optimization has been chosen as a technique that combines easy implementation with high performance. In Chap-ter 3, a meta-optimization implementation called SEPaT (Simple Evolutionary Pa-rameter Tuning) has been proposed. Comparison with a systematic search and with some other well-established methods which perform the same task have proven that SEPaT is an effective method to find good parameter configurations for a metaheuris-tic. Some of the main drawbacks of SEPaT, mostly its inability of finding validity ranges for the parameters and their role in affecting the algorithm’s performance (shared with most of the meta-optimization techniques) induced us to extend it to the multi-objective paradigm. In doing so, our goal was to extract additional infor-mation regarding a metaheuristic applying the concept of innovization, which con-sists of finding a relationship between goals and parameters by analyzing the Pareto front. The multi-objective version presented in Chapter4, called EMOPaT
(Evolu-110 Chapter 6. Conclusions
tionary Multi-Objective Parameter Tuning), is able to optimize the parameter values of a metaheuristic according to many objectives at the same time. The experiments show it is useful in many tasks, among which we can recall:
• finding good parameter values whose performances are indistinguishable from those obtained using its single-objective version;
• finding the range of validity for the values of a parameter;
• understanding the role and the utility (or the lack of it) of a parameter within an algorithm;
• performing an unbiased comparison between different metaheuristics, since they all undergo the same tuning procedure.
After that, in Chapter5, we described the applications on which we have tested our meta-optimization algorithms: the recognition of objects in images and video sequences. Model-based approaches using bio-inspired optimization are a family of methods that were proved to have a wide applicability and achieve good performance in many different problems. Moreover, this thesis showed that automatic parame-ter tuning can improve the performance of object recognition without affecting its complexity, except for a small overhead in the design phase. In this field, meta-optimization can be considered as an operation that moves some of the complexity of the problem from the online phase to the design phase, simplifying the overall task.
The main direction towards which the work presented in this thesis may be ex-panded is the automation of the processes involved. Both in the analysis of EMOPaT’s results and in the development of model-based object recognition methods, most work needs to be manually performed by the developer. In the former case, an au-tomatic analysis of the Pareto front could be a significant improvement, especially if an investigation of the properties of the functions considered in the tuning process is also included. In the latter, at present, the developer has to define a model, a series of possible operations and a similarity function. These aspects are strictly dependent on the problem considered, but a more general framework can be studied, into which they can be included.
6.0 Conclusions 111
A Final Consideration
The work behind this thesis spans over more than three years. It actually started from what concluded this dissertation, the automatic recognition of objects in images and videos. When working on this approach I realized that the metaheuristics involved, although playing a fundamental role, were usually not used at their best. Therefore, an in-depth analysis showed me that this problem was not related only to the par-ticular family of applications on which I was working on, and not only to the appli-cations that rely on metaheuristics in general, but to the development itself of these techniques. As a result, I started to work mainly at understanding these algorithms, how they work and what is a good way to develop them in a more “scientific” way.
Every year, many “novel” versions of bio-inspired algorithms are presented to the scientific community, by authors who claim that they are able to easily overtake the older versions; nonetheless the vast majority of this work is immediately forgotten.
The main reason is that, following the “horse race” metaphor proposed by John-son [68], winning a race is easy if you can accommodate the rules, choose the track and select your opponents (some of these problems have already been discussed at the beginning of Chapter2). This is obviously not the first time this criticism have been made [37,57] and many ideas have been proposed to tackle this problem [11,47]. My idea is that automatic parameter tuning could be used as an effective technique to im-prove the quality of investigations on metaheuristics. In particular, the multi-objective approach proposed here, EMOPaT, has many properties that make it suitable for this task. EMOPaT is able to provide much information that a developer can use to widen its knowledge about the algorithm he/she is working on, such as whether a parameter is useful (or useless), the role of a parameter in an algorithm and so on. Moreover, as demonstrated in Section4.1.1, EMOPaT is able to “understand” how a setting per-forms when compared to the “standard” ones, therefore it can automatically reject one that is not well-performing. Finally, using EMOPaT, it is possible to run an algo-rithm at its best, therefore it allows one to perform an unbiased comparison between different algorithms, as shown in Section4.1.4.
Appendix A
libCudaOptimize
In this Appendix, the main concepts and advantages of GPGPU programming will be presented, followed by a basic introduction to libCudaOptimize, the GPU-based library we developed and which served as the basis for most of the code written for this work. The library is freely available on SourceForge1.