Commentary on “Visualization in Operations Management Research”: Incorporating Statistical Thinking into Visualization Practices for Decision Making in Operational Management

Published Online:


Supplemental Material: The software that supports this study and helped produce the figures are available within the paper at or are supplied at, or the same version at Code Ocean available here

Data Ethics Statement: The data used for the plots are synthetic data supplied by Basole et al. (2021).

1. Introduction

Basole et al. (2021) demonstrate that visualization can aid or mislead depending on its use. We strongly agree that visualization is indeed a double-edged sword that can uncover unexpected information and communicate information effectively, but poor practices can lead to misinterpretation or overemphasis of a feature in the plot. Applying the Gestalt laws, Basole et al. (2021) illustrate the importance of choosing the appropriate chart elements. We concur that appropriate visual encoding is vital for enhancing the visual decoding accuracy of a graph; however, there is more discussion needed on the visual encoding process and the incorporation of statistical thinking when working with visual displays of data. These two points form the focus of this commentary.

It is motivating to see that Basole et al. (2021) provide an instructive framework on the goal, benefits, and challenges of visualization, considering the iterative and evolving nature of graphical process and the varying needs and challenges presented at different stages of empirical operational management research. We find the discussion highlighting the continuing and emerging concerns in terms of data security and data ethics and how they relate to visualization intriguing and timely. The strategies and examples to mitigate these concerns covered in the paper also provided significant insights into how and to what extent institutions should incorporate similar measures.

2. Visualizing Encoding Choices

Encoding refers to the process and choices made for displaying the data in a graph. Decoding describes the process of interpreting or perceiving the information from the graph. Different encoding approaches lead to differences in decoding information. Basole et al. (2021) use the Gestalt principles of perceptual organization to demonstrate misrepresentations by inclusion and omission on a graph. We agree that this is highly pertinent to enhance the visual decoding accuracy of a graph and that the discussion can be extended to include the choices in the visual encoding process. In a visual encoding process, retinal variables, such as position, size, shape, color hue, color intensity, orientation, texture, and motion, are used to represent different information in the data (Bertin 1983, Munzner 2014). Christodoulou (2018) provides a website with convenient explanation with a compact diagram to illustrate such choices. The encoding process was formalised into a grammar by Wilkinson (2005) that specifies the mapping of the data variables to retinal variables. The grammar was extended and implemented in the ggplot2 R-package (Wickham 2016). The connection with statistics is tightened because the mapping requires that data are organised as “tidy data” (Wickham 2014), where variables are in columns and observations are in rows. For example, the data used in figure 3 from Basole et al. (2021) contain the variables state, coverage, readmission rate, and cancer screening percentage and additionally two spatial variables (x, y) specifying state tile location. These variables are mapped as follows to produce the visual display: x to the x-axis (like longitude), y to the y-axis (like latitude), size to coverage, using a circle geometry, and color to readmission rate (left plot) or cancer screening percentage (right plot).

Basole et al. (2021, p. 9) use figure 3 to assert that “In this depiction, one can discern that states with highly integrated care also tend to have (i.e., are predictive of) higher cancer screening rates and lower 30-day hospital readmissions.” we interpret integrated care to mean coverage here. We disagree with this statement and feel that the authors may have misinterpreted the plot. A better visual encoding for examining association between two variables is the humble scatter plot, where coverage is mapped to the x-axis and either the readmission rate or cancer screening percentage is mapped to the y-axis, as shown here in Figure 1.1 In this example, a line corresponding to a local regression fit is added, along with text annotation displaying the sample correlation coefficient. What can be seen is that there are no strong associations between coverage and either of the other two variables, disagreeing with the conclusion of Basole et al. (2021). This interesting finding illustrates the importance of considering various visual encoding choices because this may offer additional insight or a different perspective.

Figure 1. (Color online) The Figure Shows an Alternative Plot Design to Display the Information in Figure 3 of Basole et al. (2021)
Notes. The plot shows a scatter plot of percentage of readmission and coverage on the left and a scatter plot of percentage of cancer screening and coverage on the right. Both plots are superimposed by a local polynomial regression (displayed as a solid line) as implemented by the loess function in R with confidence interval for the line (displayed in gray) as implemented by the default geom_smooth function in the ggplot2 package. See Figure S2 in the supplementary material for the code.

3. Making Inferences from a Single Plot Is Prone to Over-Reading It

Making a conclusion from a single plot is analogous to making a conclusion from a single numerical summary statistic without carefully calibrating by the standard deviation. Plots are (graphical summary) statistics, and thus statistical inference machinery ought to be used to assess the significance of structure seen in the plot (Wickham et al. 2010). A single plot is commonly used to make inferences or assertions; however, this may result in over-reading a plot. A (data) plot can be treated as a statistic with its variation calibrated so as not to wrongfully overemphasize an observed feature in the plot when the feature is a common artefact of random sampling. Buja et al. (2009) introduced the lineup protocol to calibrate a data plot by embedding it randomly among a set of null plots (a null plot has the same mapping and geometrical rendering as the data plot, using data generated under the null hypothesis of no pattern existing). The lineup protocol also adheres to a hypothesis testing framework and can be used to test for statistically significant patterns in plots. It would be recommended to remove any labels that may bias the reader because of preconceived notions about a problem. Figure 2 provides an example lineup as might be used in place of the right plot in figure 3 from Basole et al. (2021). The null plots in Figure 2 are generated by permuting cancer screening percentage against the other variables, thus breaking any association with coverage. If the data plot exhibits a strong association between cancer screening percentage and coverage, then most observers will detect it from among the null plots. On the converse, if most observers cannot detect the data plot, then it is not significantly different to null plots and one must conclude that there is no association between coverage and cancer screening percentage. A lineup is presented in Figure 2 so you may use this to test if you can find a plot that appears strikingly different to others. (The position of the data plot is only revealed in the supplementary material.)

Figure 2. (Color online) A Tile Grid Was Used in Figure 3 of Basole et al. (2021) to Display the Cancer Screening (by Color) and the Coverage (by Size) with the Tile Order Matching a Rough Spatial Layout of the United States
Notes. The figure shows a lineup for this tile grid plot where one of the plots is made using the data, and the other 19 plots are constructed after first permuting the percentage of cancer screening across different states. (Missing value structure is preserved.) Text and legends have been removed to minimise the bias in reading plots due to the reader being aware of the context. Your task is to identify any plot that is visually different from the others. See the supplementary material to see if you have selected the data plot as the most different and also for the code to produce the lineup.

The lineup protocol is a rigorous method to compare the power of different visual encodings (see examples in Hofmann et al. 2012). The same lineup can be constructed using different visual encodings and shown to multiple observers. The most powerful visual encoding will correspond to the data plot being detected in the lineup more often. In the previous section, we advised that the scatterplot is a more powerful visual encoding, than the tile grid, for assessing the association between two variables. The supplementary material has additional lineups that support this.

4. Final Thoughts

The advancements of computational subsystems, such as ggplot2 R-package (Wickham 2016), matplotlib python library (Hunter 2007), and D3 JavaScript (Bostock et al. 2011), have proliferated the production of graphs in a reproducible manner. In particular, we can now generate motion graphs and, by extension, interactive graphs and dashboards very efficiently. The ease of graph production, however, does not assure the quality of visualization practice.

Substandard visualization practice burgeons as a result of informal treatment of plots, including overemphasis on particular features of plots for decision-making processes. Considering plots as a statistic, treating it with the a similar rigour used to treat numerical statistics in inferences, can help in weeding out poor practices. We refer readers interested in learning more about visual inference to VanderPlas et al. (2020).

In addition, interactive graphs and dashboards have gained significant popularity in numerous business and e-commerce applications. An interactive visualization enables us to explore the “what-if” scenarios under different assumptions and to uncover unexpected patterns under different conditions. With live data updates, dashboards are often used for visual reports of key performance indicators, enabling interactive monitoring and real-time analysis. With the abundant and complex data available, it is ever more important that visual encoding choices are chosen to communicate the intended message to the audience efficiently but also understood without misrepresentation. We believe future research benefits from extending existing visual inference foundations to interactive graphics (Cook et al. 2021) and look forward to cultivation of statistical rigour in data visualization.


1 All figures in this paper are produced by using the R language (R Core Team 2020), and the ggplot2 package (Wickham 2016) is supplied at or the same version supplied in Code Ocean code capsule, which verifies computational reproducibility, at


  • Basole RC, Bendoly E, Chandrasekaran A, Linderman K (2021) Visualization in operations management research. INFORMS J. Data Sci., ePub ahead of print November 17, Scholar
  • Bertin J (1983) Semiology of Graphics: Diagrams, Networks, Maps (ESRI Press, Redlands, CA).Google Scholar
  • Bostock M, Ogievetsky V, Heer J (2011) D3 data-driven documents. IEEE Trans. Visualization Comput. Graphics 17(12):2301–2309.Google Scholar
  • Buja A, Cook D, Hofmann H, Lawrence M, Lee EK, Swayne DF, Wickham H (2009) Statistical inference for exploratory data analysis and model diagnostics. Philos. Trans. Roy. Soc. A: Math. Physical Engrg. Sci. 367(1906):4361–4383.Google Scholar
  • Christodoulou D (2018) Graph workflow model. Accessed December 9, 2021, Scholar
  • Cook D, Reid N, Tanaka E (2021) The foundation is available for thinking about data visualization inferentially. Accessed December 1, 2021, Scholar
  • Hofmann H, Follett L, Majumder M, Cook D (2012) Graphical tests for power comparison of competing designs. IEEE Trans. Visualization Comput. Graphics 18(12):2441–2448.Google Scholar
  • Hunter JD (2007) Matplotlib: A 2D graphics environment. Comput. Sci. Engrg. 9(3):90–95.Google Scholar
  • Munzner T (2014) Visualization Analysis and Design (CRC Press, Boca Raton, FL).Google Scholar
  • R Core Team (2020) R: A language and environment for statistical computing. R Foundation for Statistical Computing, Vienna, Austria. Accessed October 27, 2021, Scholar
  • VanderPlas S, Cook D, Hofmann H (2020) Testing statistical charts: What makes a good graph? Annual Rev. Statist. Its Appl. 7(1):61–88.Google Scholar
  • Wickham H (2014) Tidy data. J. Statist. Software 59(10):1–23.Google Scholar
  • Wickham H (2016) ggplot2: Elegant Graphics for Data Analysis (Springer-Verlag, New York).Google Scholar
  • Wickham H, Cook D, Hofmann H, Buja A (2010) Graphical inference for infovis. IEEE Trans. Visualization Comput. Graphics 16(6):973–979.Google Scholar
  • Wilkinson L (2005) The Grammar of Graphics (Springer, New York).Google Scholar
INFORMS site uses cookies to store information on your computer. Some are essential to make our site work; Others help us improve the user experience. By using this site, you consent to the placement of these cookies. Please read our Privacy Statement to learn more.