HPC enabled a Novel Deep Fuzzy Scalable Clustering Algorithm and its Application for Protein Data

Abstract

Fuzzy clustering is a common way to divide data into groups. Even though it has been improved a lot, fuzzy clustering still has problems while clustering real high-dimensional Big Data with complicated latent distributions. To solve this problem, this study comes up with a way to represent the data in a feature space that was built from a scalable deep neural network using Apache Spark on HPC. In this paper, we proposed SDnnRSIO-FCM, a Scalable Deep Neural Network Random Sampling Iterative Optimization-FCM clustering algorithm, and the SDnnLFCM, a scalable version of the Deep Neural Network Literal Fuzzy c-Means algorithm. We focus on the design and implementation of the proposed SDnnRSIO-FCM and SDnnLFCM algorithms using the Apache Spark cluster in a High-Performance Computing (HPC) environment by representing the data in a feature space produced by the neural network to handle Big Data. First, data is mapped into new feature space to aid in the reconstruction of the original data by providing a good representation. Second, scalable fuzzy clustering is embedded with neural networks to propose deep fuzzy clustering methods. The experimental results conducted on two huge benchmark datasets show that the SDnnRSIO-FCM algorithm outperforms the SDnnLFCM algorithm in terms of Normalized Mutual Information (NMI), Adjusted Rand Index (ARI), and F-score. Furthermore, the proposed SDnnRSIO-FCM applied to huge soybean protein sequences in comparison with SDnnLFCM shows a significant improvement in terms of Silhouette index (SI), Davies-Bouldin index (DBI), and Calinski-Harabasz index (CHI). © 2022 IEEE.