N-gram approach for a URL similarity measure

Abstract

This work addresses the problem of URL topic classification by making use of the text of Uniform Resource Locators (URLs). We have introduced a method for classifying the web pages into topics by extending the Jaccard distance measure and using the n-gram approach. We have also compared our method with the best performing known distance measures for Boolean data in the literature i.e. Jaccard, Dice and Cosine distance measures. The proposed method achieves a significant reduction of 3-7% in the misclassification error rate of the URLs over the Jaccard, Dice and Cosine distance measures. © 2016 IEEE.