Development of hyperplane-based models for classification and regression problems

Abstract

Support vector machines (SVMs), renowned for their robust mathematical and statistical underpinnings, offer optimal classification solutions through the construction of a single hyperplane that maximizes the margin between two classes. Despite their elegance, SVMs face a significant limitation: high computational complexity. This challenge led to the development of the twin support vector machine (TWSVM), a model that is approximately four times faster and employs two non-parallel hyperplanes to classify data more efficiently. Subsequently, least squares SVM (LSSVM) and least squares twin SVM (LSTSVM) introduced the use of least squares loss, further simplifying the computational burden by replacing quadratic programming problems with systems of linear equations. Despite these advancements, the sensitivity of hyperplane-based models to noise remains a challenge. To address this, researchers incorporated the pinball loss function into SVMs, enabling noiseinsensitive learning. Expanding upon these advancements, we proposed the flexible pinball loss SVM (FP-SVM), which extends the versatility of the pinball loss function by broadening its parameter range. This enhancement enables FP-SVM to adapt more effectively to diverse data distributions, offering superior generalization and resilience against noise. Furthermore, we introduced the universum twin support vector machine with truncated pinball loss (Tpin-UTWSVM), which combines the sparsity-inducing properties of truncated pinball loss with the universum learning framework. This novel integration not only preserves the core advantages of pinball loss but also enhances classification accuracy by leveraging additional unlabeled universum data.