Multi-Objective Machine Learning

Machine learning usually has to achieve multiple targets, which are often conflicting with each other. For example in feature selection, minimizing the number of features and the maximizing feature quality are two conflicting objectives. It is also well realized that model selection has to deal with the trade-off between model complexity and approximation or classification accuracy. Traditional learning algorithms attempt to deal with multiple objectives by combining them into a scalar cost function so that multi-objective machine learning problems are reduced to single-objective problems. Recently, increasing interest has been shown in applying Pareto-based multi-objective optimization to machine learning, particularly inspired by the successful developments in evolutionary multi-objective optimization. It has been shown that the multi-objective approach to machine learning is particularly successful in 1) improving the performance of the traditional single-objective machine learning methods 2) generating highly diverse multiple Pareto-optimal models for constructing ensembles and, 3) in achieving a desired trade-off between accuracy and interpretability of neural networks or fuzzy systems. Multi-objective machine learning covers the following main aspects:


• Multi-objective clustering, feature extraction and feature selection
• Multi-objective model selection to improve the performance of learning models, such as neural networks, support vector machines, decision trees, and fuzzy systems
• Multi-objective model selection to improve the interpretability of learning models, e.g., to extract symbolic rules from neural networks, or to improve the interpretability of fuzzy systems
• Multi-objective generation of ensembles
• Multi-objective learning to deal with tradeoffs between plasticity and stability, long-term and short-term memories, specialization and generalization

References

Copyright©2000-2006 Last update on October, 2008 by Yaochu Jin. All rights reserved. The views expressed on this home page are personal and do NOT necessarily reflect the views of his employer.