Commentary on “Visualization in Operations Management Research”: Incorporating Statistical Thinking into Visualization Practices for Decision Making in Operational Management

1. Introduction

Basole et al. (2021) demonstrate that visualization can aid or mislead depending on its use. We strongly agree that visualization is indeed a double-edged sword that can uncover unexpected information and communicate information effectively, but poor practices can lead to misinterpretation or overemphasis of a feature in the plot. Applying the Gestalt laws, Basole et al. (2021) illustrate the importance of choosing the appropriate chart elements. We concur that appropriate visual encoding is vital for enhancing the visual decoding accuracy of a graph; however, there is more discussion needed on the visual encoding process and the incorporation of statistical thinking when working with visual displays of data. These two points form the focus of this commentary.

It is motivating to see that Basole et al. (2021) provide an instructive framework on the goal, benefits, and challenges of visualization, considering the iterative and evolving nature of graphical process and the varying needs and challenges presented at different stages of empirical operational management research. We find the discussion highlighting the continuing and emerging concerns in terms of data security and data ethics and how they relate to visualization intriguing and timely. The strategies and examples to mitigate these concerns covered in the paper also provided significant insights into how and to what extent institutions should incorporate similar measures.

2. Visualizing Encoding Choices

Encoding refers to the process and choices made for displaying the data in a graph. Decoding describes the process of interpreting or perceiving the information from the graph. Different encoding approaches lead to differences in decoding information. Basole et al. (2021) use the Gestalt principles of perceptual organization to demonstrate misrepresentations by inclusion and omission on a graph. We agree that this is highly pertinent to enhance the visual decoding accuracy of a graph and that the discussion can be extended to include the choices in the visual encoding process. In a visual encoding process, retinal variables, such as position, size, shape, color hue, color intensity, orientation, texture, and motion, are used to represent different information in the data (Bertin 1983, Munzner 2014). Christodoulou (2018) provides a website with convenient explanation with a compact diagram to illustrate such choices. The encoding process was formalised into a grammar by Wilkinson (2005) that specifies the mapping of the data variables to retinal variables. The grammar was extended and implemented in the ggplot2 R-package (Wickham 2016). The connection with statistics is tightened because the mapping requires that data are organised as “tidy data” (Wickham 2014), where variables are in columns and observations are in rows. For example, the data used in figure 3 from Basole et al. (2021) contain the variables state, coverage, readmission rate, and cancer screening percentage and additionally two spatial variables (x, y) specifying state tile location. These variables are mapped as follows to produce the visual display: x to the x-axis (like longitude), y to the y-axis (like latitude), size to coverage, using a circle geometry, and color to readmission rate (left plot) or cancer screening percentage (right plot).

Basole et al. (2021, p. 9) use figure 3 to assert that “In this depiction, one can discern that states with highly integrated care also tend to have (i.e., are predictive of) higher cancer screening rates and lower 30-day hospital readmissions.” we interpret integrated care to mean coverage here. We disagree with this statement and feel that the authors may have misinterpreted the plot. A better visual encoding for examining association between two variables is the humble scatter plot, where coverage is mapped to the x-axis and either the readmission rate or cancer screening percentage is mapped to the y-axis, as shown here in Figure 1.¹ In this example, a line corresponding to a local regression fit is added, along with text annotation displaying the sample correlation coefficient. What can be seen is that there are no strong associations between coverage and either of the other two variables, disagreeing with the conclusion of Basole et al. (2021). This interesting finding illustrates the importance of considering various visual encoding choices because this may offer additional insight or a different perspective.

3. Making Inferences from a Single Plot Is Prone to Over-Reading It

Making a conclusion from a single plot is analogous to making a conclusion from a single numerical summary statistic without carefully calibrating by the standard deviation. Plots are (graphical summary) statistics, and thus statistical inference machinery ought to be used to assess the significance of structure seen in the plot (Wickham et al. 2010). A single plot is commonly used to make inferences or assertions; however, this may result in over-reading a plot. A (data) plot can be treated as a statistic with its variation calibrated so as not to wrongfully overemphasize an observed feature in the plot when the feature is a common artefact of random sampling. Buja et al. (2009) introduced the lineup protocol to calibrate a data plot by embedding it randomly among a set of null plots (a null plot has the same mapping and geometrical rendering as the data plot, using data generated under the null hypothesis of no pattern existing). The lineup protocol also adheres to a hypothesis testing framework and can be used to test for statistically significant patterns in plots. It would be recommended to remove any labels that may bias the reader because of preconceived notions about a problem. Figure 2 provides an example lineup as might be used in place of the right plot in figure 3 from Basole et al. (2021). The null plots in Figure 2 are generated by permuting cancer screening percentage against the other variables, thus breaking any association with coverage. If the data plot exhibits a strong association between cancer screening percentage and coverage, then most observers will detect it from among the null plots. On the converse, if most observers cannot detect the data plot, then it is not significantly different to null plots and one must conclude that there is no association between coverage and cancer screening percentage. A lineup is presented in Figure 2 so you may use this to test if you can find a plot that appears strikingly different to others. (The position of the data plot is only revealed in the supplementary material.)

The lineup protocol is a rigorous method to compare the power of different visual encodings (see examples in Hofmann et al. 2012). The same lineup can be constructed using different visual encodings and shown to multiple observers. The most powerful visual encoding will correspond to the data plot being detected in the lineup more often. In the previous section, we advised that the scatterplot is a more powerful visual encoding, than the tile grid, for assessing the association between two variables. The supplementary material has additional lineups that support this.

4. Final Thoughts

The advancements of computational subsystems, such as ggplot2 R-package (Wickham 2016), matplotlib python library (Hunter 2007), and D3 JavaScript (Bostock et al. 2011), have proliferated the production of graphs in a reproducible manner. In particular, we can now generate motion graphs and, by extension, interactive graphs and dashboards very efficiently. The ease of graph production, however, does not assure the quality of visualization practice.

Substandard visualization practice burgeons as a result of informal treatment of plots, including overemphasis on particular features of plots for decision-making processes. Considering plots as a statistic, treating it with the a similar rigour used to treat numerical statistics in inferences, can help in weeding out poor practices. We refer readers interested in learning more about visual inference to VanderPlas et al. (2020).

In addition, interactive graphs and dashboards have gained significant popularity in numerous business and e-commerce applications. An interactive visualization enables us to explore the “what-if” scenarios under different assumptions and to uncover unexpected patterns under different conditions. With live data updates, dashboards are often used for visual reports of key performance indicators, enabling interactive monitoring and real-time analysis. With the abundant and complex data available, it is ever more important that visual encoding choices are chosen to communicate the intended message to the audience efficiently but also understood without misrepresentation. We believe future research benefits from extending existing visual inference foundations to interactive graphics (Cook et al. 2021) and look forward to cultivation of statistical rigour in data visualization.