Ensembles of decision tree and random vector functional link neural network for classification problems

Abstract

Decision tree, neural network and support vector machine are the powerful machine learning models widely used in classification problems. Decision tree is composed of terminal and non-terminal nodes. Each non-terminal node evaluates a series of deci sions and optimizes the best split. The terminal nodes represent di↵erent class labels or their distributions. Intuitively, decision tree is a sequence of If-Then rules which are mostly understood by humans and are easy to implement. Neural network (NN) is another category of machine learning algorithms. It is composed of several inter connected computational units known as neurons. The neurons are interconnected via weights learned usually via back propagation algorithms. However, in randomized neural networks like random vector functional link network (RVFL) some weights are initialized randomly (fixed while training) and other weights are optimized via closed form solution or iterative algorithm. On the other hand, support vector machine (SVM) is an algorithm based on the concept of margin maximization. SVM imple ments the structural risk minimization and is a stable classifier.