The ITEM Ontology: A Tool to Elucidate the Anatomy of Psychometric Indicators

Survey-based research in information systems requires valid scales to advance theory, and the discipline has developed rigorous procedures to assess scale validity. In principle, these procedures ensure that scales consist of clear indicators and faithfully represent the focal construct. However, the focus on the psychometric properties of scales has overshadowed the role of lexical and semantic elements in the validation process, leading to invalid scales. This overemphasis on psychometric properties will persist unless researchers have a systematic approach to analyzing the properties of indicators and share the outcome of such analyses in formats that can be peer-reviewed, critiqued, or corroborated by other researchers. Thus, the psychometric community needs a shared language and method to uncover the properties of indicators and identify validity problems that psychometric analysis fails to detect. Drawing on ontology development methods, we propose the Indicator Terminology for Explanation and Measurement (ITEM) Ontology, consisting of four high-level hierarchies of entities: objects, measurables, qualifiers, and response sets, each almost always found within an individual indicator. We develop an approach, a codebook, and a website for applying ITEM to psychometric indicators. Common approaches to ontology evaluation are then used to evaluate its expressiveness, utility, importance, accessibility, suitability, and external validity. We find that the ITEM Ontology is highly generative in that it can be used to address several previously unsolvable problems in survey science, polling, and theory testing.

History: Youngjin Yoo, Senior Editor; Heng Xu, Associate Editor.

Funding: The authors thank the U.S. National Institutes of Health (NIH) for support under Grant 3U24AG052175-08S1, the Social Sciences and Humanities Research Council of Canada (SSHRC) for support under Grant 435-2020-0402, Joerg Evermann for assistance during the early stages of the project, and dozens of research assistants at the University of Colorado.

Supplemental Material: The online appendix is available at https://doi.org/10.1287/isre.2023.0257.