Instructions for IJDS Reviewers

The following is what IJDS sends to a reviewer once he/she agrees to review a submission.  We publish the set of instructions so that the authors are aware of what we ask the reviewers to focus.  The set of instructions borrow certain languages and statements from those used by peer journals such as IISE Transactions and Operations Research.

Dear Reviewers: please start the review report with a summary of the main message in the paper and then discuss its major strengths (only if applicable) and/or major weaknesses (only if applicable). Please put minor comments into a separate list.

The most important question your review can help us decide is whether the submission is on a path towards acceptance or whether it should be rejected at this moment.

Papers on the path towards acceptance have the following characteristics:

• A good paper advances knowledge in our field by presenting insights that researchers in our field are not yet aware of. The paper does not have to be perfect but the authors should study a problem or present an idea which the reviewer finds new, interesting, and relevant. 
• The authors recognize the state of the art in the literature (i.e., the knowledge frontier), specify research questions which, if addressed, would advance that frontier, and identify, when warranted, a reasonable baseline from which numerical comparisons are conducted so that improvement can be quantified.
• The authors carry out an adequate analysis, either theoretically, computationally, or empirically, to address a problem or to provide new insights to guide practical decision-making.

We hope the following guidelines are helpful to you for crafting a quality review:

• A good summary of the main message helps a lot. The message referred to here is much more than what the authors claimed and what they did in the paper but what you think they are advocating in their paper (an insight, some new findings, or a different way of doing things). Is it possible for you to paraphrase their research question and ask yourself why we should care about the question?  Then what do you think the authors’ answer to that question is?  Is the authors’ answer insightful or useful?
• If a paper’s writing is so poor that it is impossible to extract the message, please recommend rejection for the reason of “lack of clarity.” But this reason should be used only after a careful examination of the paper because when a paper arrives at the reviewer’s desk, it has been screened multiple times by various editors.
• Understandably, the authors should have argued for their idea/method. If you disagree, please provide substantive arguments and specific comments, such as offering references to published papers that presented similar ideas, or constructive criticism of the authors’ assumptions, logic, or arguments. Please avoid generic statements that are dismissive of the submission.
• To assist the authors in preparing their revision, please prioritize your concerns and try to avoid providing an unorganized list of changes. In other words, when making a revision recommendation, please outline what the most important or necessary changes are. If you feel the paper needs a complete rewrite or radical changes to the analysis (i.e., more than a major revision in six months), the proper recommendation is rejection.
• Please do not express your editorial recommendation explicitly in your referee report. This is to say, review reports, which are to be shared with the authors, should not include an explicit statement such as “I recommend this paper to be accepted/revised/rejected.” You can express an editorial recommendation by checking the proper choice in the online paper review system or by providing it in the confidential comment box to the handling editor.
• Unless explicitly asked to by the editor, please otherwise refrain from making judgments with regard to whether the paper fits within the scope of IJDS.  Editors should be making determinations about scope.  We would like to ask the reviewers to focus on evaluating the contribution rather than the scope.
• You are not required to replicate the authors’ results, or comment on the code quality of the authors. But if you find that the reproducibility of the work is essential to determining the suitability of a paper’s publication in IJDS, please reach out to the handling editor and work with him/her to see how the data/code can be requested from the authors for verification.
• Just as with other INFORMS journals, we at IJDS generally discourage reviewers from raising new concerns after the first round. We do acknowledge the chance that one could realize a new issue due to the updated presentation in the second round. But, in the first round, please try to be as detailed as possible.
• Please be kind in your language, even when being critical in your assessment.