A Novel Scalable Kernelized Fuzzy Clustering Algorithms Based on In-Memory Computation for Handling Big Data

Abstract

Traditional scalable clustering algorithms mainly deal with the clustering of linearly separable data, but it is challenging to cluster the non-linear separable data efficiently in the feature space. In this article, we propose a novel Kernelized Scalable Random Sampling with Iterative Optimization Fuzzy c-Means (KSRSIO-FCM) clustering algorithm using Big Data framework. To propose the KSRSIO-FCM, we also propose the Kernelized version of Scalable Literal Fuzzy c-Means (KSLFCM) clustering algorithm, which is an integral part of the proposed KSRSIO-FCM algorithm. These kernelized clustering algorithms are evolved to deal with the non-linear separable problems by applying a kernel Radial Basis Functions (RBF) which maps the input data space non-linearly into a high dimensional feature space. We aim to design and implement the kernelized fuzzy clustering algorithms on Apache Spark, which performs the efficient clustering of Big Data due to its in-memory cluster computing technique. Exhaustive experiments are performed on various big datasets to show the effectiveness of proposed KSRSIO-FCM in comparison with other scalable clustering algorithms, i.e., KSLFCM, SRSIO-FCM, and SLFCM. The reported experimental results show that the KSRSIO-FCM algorithm in comparison with KSLFCM, SRSIO-FCM, and SLFCM achieves significant improvement in terms of time and space complexity, Normalized Mutual Information (NMI), Adjusted Rand Index (ARI), and F-score, respectively. Furthermore, we have carried out a performance analysis of KSRSIO-FCM versus KSLFCM. Thus, the reported results show that the KSRSIO-FCM implemented on Apache Spark has better potential for Big Data clustering as compared to traditional scalable fuzzy clustering methods. © 2017 IEEE.