Find in Library

The automatic detection of <italic>Design Smells</italic> has evolved in parallel to the evolution of automatic <italic>refactoring</italic> tools. There was a huge rise in research activity regarding <italic>Design Smell</italic> detection from 2010 to the present. However, it should be noted that the adoption of <italic>Design Smell</italic> detection in real software development practice is not comparable to the adoption of automatic <italic>refactoring</italic> tools. On the basis of the assumption that it is the objectiveness of a refactoring operation as opposed to the subjectivity in definition and identification of <italic>Design Smells</italic> that makes the difference, in this paper, the lack of agreement between different evaluators when detecting <italic>Design Smells</italic> is empirically studied. To do so, a series of experiments and studies were designed and conducted to analyse the concordance in <italic>Design Smell</italic> detection of different persons and tools, including a comparison between them. This work focuses on two well known <italic>Design Smells</italic>: <italic>God Class</italic> and <italic>Feature Envy</italic>. Concordance analysis is based on the Kappa statistic for inter-rater agreement (particularly <italic>Kappa-Fleiss</italic>). The results obtained show that there is no agreement in detection in general, and, in those cases where a certain agreement appears, it is considered to be a fair or poor degree of agreement, according to a <italic>Kappa-Fleiss</italic> interpretation scale. This seems to confirm that there is a subjective component which makes the raters evaluate the presence of <italic>Design Smells</italic> differently. The study also raises the question of a lack of training and experience regarding <italic>Design Smells</italic>.