Distributed Semi-supervised Fuzzy Regression with Interpolation Consistency Regularization

Abstract

Recently, distributed semi-supervised learning (DSSL) algorithms have shown their effectiveness in leveraging unlabeled samples over distributed networks with multiple interconnected agents. However, existing DSSL algorithms cannot cope with data uncertainties and may suffer from high computation and communication overhead problems. Hence, we propose a distributed semi-supervised fuzzy regression model, called DSFR to tackle these issues with a two-pronged strategy - first, a structure learning with a distributed fuzzy C-means method (DFCM) that identifies the parameters in the antecedent component of fuzzy if-then rules; and, second, a parameter learning with distributed interpolation consistency regularization (DICR) to obtain the parameters in the consequent component. Since DFCM is both distributed and unsupervised, it can thus extract feature representation from both labeled and unlabeled samples among multiple agents. Meanwhile, DICR expands sample space with interpolated unlabeled instances in a distributed scheme and forces decision boundaries to lie in sparse data areas, thus increasing the models robustness. Both DFCM and DICR are implemented following the alternating direction method of multipliers method. Notably, none of the procedures involve backpropagation, so the model converges very quickly. Further, with the benefit of DFCM and DICR, DSFR is highly scalable to large datasets. Experiments on both artificial and real-world datasets show that this approach yields much lower loss values than the current state-of-the-art DSSL algorithms at a fraction of the computation cost. Our code is available online\footnote{\url{https://github.com/leijiezhang/DSFR}}. IEEE