MKL-RT: Multiple Kernel Learning for Ratio-trace Problems
via Convex Optimization
Abstract: Recently, automatic selection or combination of kernels based on multiple kernel learning (MKL) approaches has been receiving significant attention from various research communities. Though MKL has been extensively studied for support vector machines, it is relatively less explored in the context of dimensionality reduction. In this paper, we show that MKL can be formulated as a convex optimization problem for a large class of dimensionality reduction algorithms using the ratio-trace objective function. We also provide an optimization procedure that is guaranteed to converge to the global optimum of the proposed optimization problem. We experimentally demonstrate that the proposed approach, which we refer to as MKL-RT, can be successfully used to select features for discriminative dimensionality reduction and cross modal retrieval. We also show that the proposed ratio-trace-based convex MKL-RT approach outperforms the existing trace-ratio-based non-convex MKL-DR approach in terms of both performance (image classification and retrieval) and efficiency (number of kernels selected).
Contributions
- We show that MKL can be formulated as a convex optimization problem for a large class of ratio-trace problems that covers many existing dimensionality reduction algorithms.
- We provide an optimization procedure that is guaranteed to converge to the global optimum of the proposed problem.
- We compare the proposed MKL-RT approach with the existing MKL-DR approach for image classification and cross-modal retrieval tasks, and show that the ratio-trace-based convex MKL-RT performs better than the trace-ratio-based non-convex MKL-DR in terms of both performance and efficiency.
Preprints:
Raviteja Vemulapalli, Vinay P. Boda, and Rama Chellappa, "Multiple Kernel Learning for Ratio-trace Problems via Convex Optimization", CoRR abs/1410.4470, 2014. [PDF]