TRKM: Twin restricted kernel machines for classification and regression

Abstract

Restricted kernel machines (RKMs) have significantly advanced machine learning by integrating kernel functions with least squares support vector machines (LSSVM), adopting an energy function akin to restricted Boltzmann machines (RBM) to enhance generalization performance. Despite their strengths, RKMs face challenges in handling unevenly distributed or complexly clustered data and incur substantial computational costs when scaling to large datasets due to the management of high-dimensional feature spaces. To address these limitations, we propose the twin restricted kernel machine (TRKM), a novel framework that synergizes the robustness of RKM with the efficiency of twin hyperplane methods, inspired by twin support vector machines (TSVM). TRKM leverages conjugate feature duality based on the Fenchel-Young inequality to reformulate classification and regression problems in terms of dual variables, establishing a bound on the objective function and introducing a new methodology within the RKM framework. By incorporating an RBM-inspired energy function with visible and hidden variables corresponding to both classes, TRKM effectively captures complex data patterns. The kernel trick is employed to project data into a high-dimensional feature space, where an optimal separating hyperplane is identified using a regularized least squares approach, enhancing both performance and computational efficiency. Extensive experiments on 36 diverse datasets from UCI and KEEL repositories demonstrate TRKM's superior accuracy and scalability compared to baseline models. Additionally, TRKM's application to the brain age estimation dataset underscores its efficacy in predicting brain age, a critical biomarker for early Alzheimer's disease detection, highlighting its potential for real-world medical applications. To the best of our knowledge, TRKM is the first twin variant of the RKM framework, offering a robust and efficient solution for complex classification and regression tasks. The source code of the proposed TRKM model is available at https://github.com/mtanveer1/TRKM. © 2025 Elsevier Ltd