From the Editors: New Decision Analysis Journal Submission Requirements
Replication is central to the practice of empirical science. Unfortunately, over the last 20 years or so, concerns have grown that replicability is not working as well as it should.1 As a result, INFORMS journals, including Decision Analysis, are improving their policies to support the goal of replication. In the case of Decision Analysis, most submissions do not relate to empirical research. Theoretical, methodological, and discursive submissions are clearly not subject to the same types of replicability requirements. Moreover, many empirical papers reporting case studies and applications are also exempt. It is accepted that prescriptive analyses are designed to evolve the perceptions, understanding, beliefs, values, and judgements of the participants. A decision analysis therefore cannot be repeated completely afresh. However, we do publish empirical studies of decision behavior conducted under “laboratory conditions,” and case studies can be evaluated using extensive questionnaire and structured-interview results. Therefore, a small group of members of the editorial board have been considering our requirements for submission, peer review, and subsequent publication of such papers.
Our discussions recognized that the journal does not have many empirical submissions, so we can begin with relatively “light touch” requirements, note how well they work, and perhaps modify them in a few years in the light of experience. More importantly, much decision analysis research takes place in commercially sensitive areas, so there are issues of intellectual property rights and confidentiality. Accordingly, the journal has recently adopted a new policy aimed at helping to ensure the replicability of research.2
“For papers containing significant empirical, observational, or experimental analysis, we strongly recommend sufficient material to be available (either in the paper or in appendices, data-files and codes) to enable the analysis of the data and/or observations to be replicated. Such material should include experimental designs, elicitation protocols and so forth, as well as the raw data and observations. If requested, and recognizing that release of proprietary materials may require non-disclosure agreements, authors using these materials should work with the editors to provide evidence that the paper is accurate, and the conclusions are replicable. On acceptance, the journal would expect that these materials are archived in such a way that future researchers can access them, again recognizing that proprietary requirements may necessitate non-disclosure agreements.”
We hope that this contributes to the movement for better peer review and improved replicability.
