Hierarchical Cluster Labeling

Analysis of Structural Relationships for Hierarchical Cluster Labeling

Abstract: Cluster label quality is crucial for browsing topic hierarchies obtained via document clustering. Intuitively, the hierarchical structure should influence the labeling accuracy. However, most labeling algorithms ignore such structural properties and therefore, the impact of hierarchical structures on the labeling accuracy is yet unclear. In our work we integrate hierarchical information, i.e. sibling and parent-child relations, in the cluster labeling process. We adapt standard labeling approaches, namely Maximum Term Frequency, Jensen-Shannon Divergence, Chi Square Test, and Information Gain, to take use of those relationships and evaluate their impact on 4 different datasets, namely the Open Directory Project, Wikipedia, TREC Ohsumed and the CLEF IP European Patent dataset. We show, that hierarchical relationships can be exploited to increase labeling accuracy especially on high-level nodes.

Download source: https://kdev.know-center.tugraz.at/corpi/dmoz-html/crawledpages.tar.bz2.
At the root directory you find a file containing a lookup between a URL in the DMOZ articles and a sub-directory with the crawled HTML content.