Robust and efficient machine learning under data uncertainty: from loss functions to data-aware representations

Abstract

Reliable machine learning in real-world environments is often limited not by the expressive power of a model alone, but by how the learning objective responds to corrupted labels, outliers, overlapping classes, class imbalance, ill-conditioned optimization, and arbitrary random representations. This thesis, entitled Robust and Efficient Machine Learning under Data Uncertainty: From Loss Functions to Data-Aware Representations, develops a coherent set of robust and computationally efficient learning strategies by redesigning four fundamental components of supervised learning: the loss function, the sample-weighting mechanism, the optimization procedure, and the initialization of randomized representations. The first contribution is the RoBoSS loss, a robust, bounded, sparse, and smooth classification loss that limits the influence of extreme samples while preserving useful margin behavior and optimization stability. RoBoSS is integrated into support vector machine (SVM) learning and further extended to twin support vector machine (TSVM) learning through the RoBoSS-based Twin Support Vector Machine (RoBoTS) and its inverse-free variant, demonstrating that bounded sparse loss design can improve robustness without sacrificing efficiency. The thesis then develops the Wave loss function, a bounded, smooth, and asymmetric classification loss, and embeds it into SVM and TSVM frameworks to obtain Wave-SVM and Wave-TSVM for learning under noisy and uncertain data.