Consistency analysis in quality classification problems with multiple rank-ordered agents

A relatively diffused quality decision problem is that of classifying some objects of interest into predetermined nominal categories. This problem is particularly interesting in the case: (i) multiple agents perform local classifications of an object, to be fused into a global classification; (ii) there is more than one object to be classified; and (iii) agents may have different positions of power, expressed in the form of an importance rank-ordering. Due to the specificity of the problem, the scientific literature encompasses a relatively small number of data fusion techniques. For the fusion to be effective, the global classifications of the objects should be consistent with the agents’ local classifications and their importance rank-ordering, which represent the input data. The aim of this article is to propose a set of indicators, which allow to check the degree of consistency between the global classification and the input data, from several perspectives, e.g., that of individual agents, individual objects, agents’ importance rank-ordering, etc. These indicators are independent from the fusion technique in use and applicable to a wide variety of practical contexts, such as problems in which some of the local classifications are uncertain or incomplete. The proposed indicators are simple, intuitive, and practical for comparing the results obtained through different techniques. The description therein is supported by several practical examples.