Weighting by Tying: A New Approach to Weighted Rank Correlation
Sascha Henzgen Eyke Hüllermeier
Measures of rank correlation are commonly used in statistics to capture the
degree of concordance between two orderings of the same set of items. Standard
measures like Kendall's tau and Spearman's rho coefficient put equal emphasis
on each position of a ranking. Yet, motivated by applications in which some of
the positions (typically those on the top) are more important than others, a
few weighted variants of these measures have been proposed. Most of these
generalizations fail to meet desirable formal properties, however. Besides,
they are often quite inflexible in the sense of committing to a fixed weighing
scheme. In this paper, we propose a weighted rank correlation measure on the
basis of fuzzy order relations. Our measure, called scaled gamma, is related to
Goodman and Kruskal's gamma rank correlation. It is parametrized by a fuzzy
equivalence relation on the rank positions, which in turn is specified
conveniently by a so-called scaling function. This approach combines soundness
with flexibility: it has a sound formal foundation and allows for weighing rank
positions in a flexible way.