Identifying the “Players” in Sports Analytics Research

Research interest in the application of analytics in sports has increased dramatically. As Wright (2009) discusses, operations research (OR) in sports has a 50+ year history, with rising popularity marked by special issues of European Journal of Operational Research (EJOR), IMA Journal of Management Mathematics (IJMM), and Computers and Operations Research (COR) since 2003. To this list we can add special issues of the Journal of the American Statistical Association(JASA) in 1994, Managerial and Decision Economics (MDE) in 1994 and 2004, the Journal of the Royal Statistical Society–Series D (JRSD) in 2002, INFORMS Transactions on Education (ITE) in 2004, Applied Economics (AE) in 2009, the International Journal of Forecasting (IJF) in 2010, and Interfaces (currently). The introductions of the Journal of Quantitative Analysis in Sports (JQAS) in 2005, the Journal of Sports Economics (JSE) in 2000, and the International Journal of Sports Science and Engineering (IJSSE) in 2007 further highlight the trend of increasing popularity of sports analytics (Wright 2009, Norman and Scarf 2005). The same can be said of the advent of the INFORMS' SpORts section in 2004, the Northern California Symposium on Statistics and OR in Sports in 2008 and 2010, the annual MIT Sloan Sports Analytics Conference since 2007, the biennial IMA International Conference on Mathematics in Sport since 2007, and the Symposium on Statistics and OR in Baseball since 2006.

The US sports industry is estimated to be $414 billion (Plunkett Research, Ltd. 2010). The industry size, rising use of analytics by sports organizations (Cochran 2010), and gains in academic stature illustrated above create a constructive environment for sports analytics (SA) as a discipline. However, the SA field remains fragmented. Much of the research represents rare forays into sports by authors who do not further pursue it (Wright 2009). In addition, Wright remarks that the field is likely still too “incoherent” to support academic programs devoted to it. Moreover, because of the competitive and secretive nature of professional sports, a large number of SA results and methods are internal by nature and not published. This also creates a challenge for academics interested in partnering with teams for projects.

Other than the new journals noted above, SA has no natural target. Thus, authors may be unsure of the most appropriate or receptive place to send a paper and, notwithstanding sports' expansive footprint, whether traditionally high-quality journals are interested in—or yet respect—what is likely considered by many as a somewhat eccentric or less generalizable application area.

Many researchers have tried to apply structure to the literature. These include Mottley (1954), Machol and Ladany (1976), Ladany and Machol (1977), Norman and Scarf (1995), Bennett (1998), Mondello and Pederson (2003), Ribeiro and Urrutia (2004), Albert et al. (2005), Albert and Koning (2007), Cochran (2008), Rasmussen and Trick (2008), Wright (2009), and Kendall et al. (2010). However, no prior work has depicted the field's composite size and growth in terms of the number of published refereed journal articles. In addition, no previous author has identified the degree to which analytics-oriented journals actually publish SA research, or the journals that contain the most (or most-cited) SA research. A substantial number of articles in related disciplines rank institutions that contribute to the field's journals, such as the eight editions of the Rothkopf rankings in applied OR (Fricker 2009). However, no such ranking exists for SA.

The objective of this article is to fill these voids. Its goal is to identify the “players” in SA research: the journals, institutions, and set of articles that constitute SA's current body of peer-reviewed academic literature. Such knowledge will help SA authors identify the best potential publishing options, the extent to which high-quality outlets have been receptive, the journals in which SA research has had the largest impact, and the journals in which they can find prior work. The latter addresses Norman and Scarf's (2005, p. 87) concern that “relevant papers are often missed and sometimes papers are published which do not take sufficient account of previously published work.”

Identifying research-generating institutions provides insight into whether the field is indeed internationally diverse with “ubiquitous” interest (Koning et al. 2003), and whether certain institutions, countries, or regions appear more supportive. Institutional rankings also help distinguish SA as a subdiscipline. Faculty in well-established areas can usually identify their field's leading institutions. No such perception has necessarily existed in SA. Because very small pockets of prolific authors may comprise the entire SA research group at a given school, such rankings by extension might provide internal or external recognition for these individuals.

Finally, measuring the field using the primary currency of academic research—refereed journal articles—lends further credibility to SA as a maturing, important subdiscipline, and not necessarily just a “fun” application area “that touches most people but carries little risk” (Watkins and Wolstenholme 2002, p. 128).

Data

Consistent with Analytics' tagline “math, operations research, statistics driving business,” I searched an amalgamation of 140 journals, largely from OR, statistics, and applied mathematics, focusing on those constructed by or for an English-speaking audience or source. Noting Wright's (2009) comment about economics being related to OR, I also included a select set of four applied economics journals that frequently publish SA research. Table 1 lists the set of 140 journals. To save space, I used Web of Science (2010) title abbreviations; for journals not included in the Web of Science, I derived abbreviations using similar title-word shorthand. Table 1 also shows the ranking of each journal within the set of 140, according to its respective h-index reported by Harzing's Publish or Perish (Harzing 2010). The h-index is “defined as the number of papers [h] with citation number ≥ h” (Hirsch 2005, p. 16,569); for example, an h-index of 25 would indicate that a journal has 25 articles that have at least 25 citations.

Table 1 The 140 analytics journals listed, which include selections from OR, statistics, applied mathematics, and applied economics, were searched; they ranked within the set of 140 as shown, according to their respective overall h-indexes.

In late 2009 and early 2010, I conducted online searches of each journal using electronic databases and (or) Google Scholar. I collected refereed articles and notes (and comments, because they are commonly refereed) featuring a sports focus in the title, keyword list, or abstract. I did not include research in which sports examples were used only as convenient illustrations, research that has potential or even common application to sports but that was not necessarily presented as a sports application, or research focused on gambling, casino (or parlor) games, health, physiology, fitness, or engineering. I included articles posted as forthcoming but already available on the JSE website.

Because of various interpretational issues, combined with expediency needs in reviewing thousands of articles, some articles might (might not) have been inadvertently omitted (included). Use of electronic sources was another study limitation; these cover only the last 20–25 years for some journals, and precluded journals that were not readily available electronically. I also did not capture contributions from less analytics-focused outlets and, because I included only four economics journals, I did not capture all contributions of that discipline. However, for researchers whose expertise is in analytics, the journals examined cover the vast majority of the outlets they would favor.

Methodology

For each article, I collected the publication year, primary institutional affiliation (if reported) of each author, and citations received according to Google Scholar as of late January 2010.

I computed the total number of SA articles published in each year, in each journal, and from each institution. Like similar prior research (e.g., Young et al. 1996), I used two approaches to compute institutional figures. In one, each institution received a credit of one for each unique time it appeared in an article's authorship. Summing these values over all articles resulted in a simple count (a) of the articles in which an institution appeared. This is the way institutions were scored in the first seven editions of the Rothkopf rankings of the INFORMS practice literature (Fricker 2009). However, this approach did not accurately portray contributions on multi-institution work. Thus, I also computed proportionally adjusted values, where an institution's credit on an article was the proportion of the authorship from that institution. Summing these values over all articles yielded a proportionally adjusted article count (a_p) for each institution, equivalent to Fricker's (2009) “yield” metric in the 2009 Rothkopf rankings.

I also counted the total citations generated by each journal and each institution using an approach similar to that described above to count each institution's articles. The first (c) assigned full credit for all citations to an article to each institution in its authorship. The second was a proportional citation count (c_p) in which an institution was assigned the number of citations to each article multiplied by the proportion of the article's authorship attributable to that institution.

The above process resulted in a quantity (article) count and quality (citation) count for each journal, and two quantity and two quality counts for each institution. I ranked the journals and the institutions based on each respective metric, and present a top-40 ranking according to each metric.

To derive a summary (quantity plus quality) ranking for journals and institutions, I computed the corresponding SA h-index for each entity. For institutions, I present an h-index (or h) in which I attributed all citations to an article to each institution in the authorship, and a proportional h-index (or h_p) in which I assigned each institution only its proportionally adjusted citations.

Results: Scale and Growth

The search identified 1,146 SA articles. To give some perspective, the 2009 Rothkopf rankings (Fricker 2009) identified the number of articles published in the INFORMS practice literature from 2002 to 2008 as 287 articles. Thus, the number of SA articles in the data is four times the total number of applied-OR articles published in Interfaces and OperationsResearch over a seven-year time frame. These results emphasize that the field is not necessarily small, because it equates to nearly 28 years of OR practice literature. Even if all articles in JSE, JQAS, and IJSSE were ignored, the remaining list would still equal nearly 18 years worth (of literature). Given that Fricker's (2009) time span (seven years) and number of journals examined (two) are each much smaller than what I cover in this paper, this is not meant as a direct or fair comparison of SA to applied OR. The intent is simply to give some perception of the SA literature size vis-à-vis the number of articles published in those two leading journals.

Figure 1 illustrates the recent growth in the number of SA articles and mirrors the time line of 162 sports-scheduling papers included in Kendall et al. (2010). The earliest two articles captured were published in Journal of the Royal Statistical Society in 1945 (to save space, these two articles do not appear in Figure 1); the next two were in 1956. During much of the ensuing time, the number of articles per year increased only slightly. Over roughly the past decade, annual article counts have skyrocketed, reaching 176 in 2009. This is somewhat attributable to the search methodology; older articles are less likely to appear in electronic databases. However, the past 20 years—a window much less affected by the methodology—still demonstrates rapid growth. Growth in the past decade is partly because of the introduction of JSE, JQAS, and IJSSE. However, even if those journals were omitted, the growth is still positive (see Figure 1).

To determine whether the recent rise in appearances in nonsports journals is simply because more papers are being published in those journals in general, I compared the number of SA articles to the total articles published in all the nonsports journals reported in Table 2 (discussed in the Results: Journals section) since 1990. Figure 1 presents the annual percentage of SA articles in these nonsports journals. One-tailed Cochran-Armitage trend tests of the annual proportions (Liu 2007) have been significant at the 0.0001 level since 1990, and at the 0.05 level over the past decade, indicating a positive (upward-sloping) trend in the annual proportions during each time frame.

Table 2 The list shows the top-40 journals in sports analytics articles (a), citations (c), and h-index.

Results: Journals

Table 2 contains the results for the top-40 journals according to each metric. Although Table 2 is ordered based on article count, any journal ranking in the top 40 on any metric is included. (In the interest of space, I omitted journals with h = 1, which did not rank in the top 40 in a or c.)

The journals listed in Table 2 accounted for 97 percent of all SA articles and 99 percent of all citations received. Reaching the top 20 required only 13 articles; reaching the top 30 required only 6. Despite covering a broad swath of journals in numbers, topics, target audiences, and impact, these results suggest that a very large majority of SA can be found in a relatively short list of journals. This is valuable information for future authors and for those searching and referencing previous research.

Seeing JSE and JQAS, two of the sports-focused journals in the data, lead in number of articles published was not surprising; despite relatively short runs, they contributed approximately 28 percent and 8 percent of the articles, respectively. Four of the top seven journals were the applied economics journals included in the search, which collectively contributed about 42 percent of the articles. This finding suggests that these journals represent strong publishing options for SA authors.

Other than the four applied economics journals and JQAS, American Statistician (at 4) was the leading journal for SA research, and was easily the leading statistics journal, nearly tripling the contributions from the 11th-ranked JASA. The Journal of the Operational Research Society (JORS) at 5 and Interfaces at 8 were the leading OR publishers of SA. EJOR and COR also ranked in the top 10, and Operations Research, IJMM, and Management Science ranked t13th, t13th, and 24th, respectively (note that the “t” in “t13th” represents “tied for”). No other OR-research journal published at least 10 articles. Operations Research Letters, International Transactions in Operational Research, and OR Insight were the only additional OR entries in the top 30.

The teaching-focused journals had good representation, with Teaching Statistics at 11, Journal of Statistics Education at 18, and ITE at 20. That teaching journals finished so high is a compliment to sports as teaching examples, as previously noted by Ribeiro and Urrutia (2004), Kendall et al. (2010), and Cochran (2010).

It is worth reiterating here that several of the journals that appear high in the article-count rankings published special issues on SA during the time frame studied. These include AE, MDE, EJOR, COR, JASA, IJMM, JRSD, and ITE. Therefore, their article counts were influenced accordingly.

Among the leading analytics journals in general, only EJOR ranked in the top 10 both in overall (i.e., nonsports analytics) h-index (at 7) and in SA articles published (at 9). JASA and Operations Research were the only other journals that ranked in the top 20 in both overall h-index and in the number of SA articles. The two most frequent publishers of SA from OR—JORS and Interfaces—ranked 22nd and 44th, respectively, in overall h-index.

Although the applied economics journals ranked at or near the top in terms of their proclivity to publish SA research, only AE (at 41) and MDE (at 68) placed in the upper half in terms of their overall h-index.

Each journal's collective SA impact is best reflected in its SA h-index. JSE leads with 25 articles with at least 25 citations. Although JQAS ranks second in number of articles (93), only 6 have yet garnered at least six citations in the journal's short life, placing JQAS 18th in impact. Omitting JQAS, IJSSE, Teaching Statistics, IJMM, and ITE demonstrated the largest drops in h ranking (19, 16, 9, and 7 places, respectively) from what was a top-20 ranking based on article counts. For example, IJSSE ranked 20th in article count (quantity) but ranked 39th in h-index. Thus, its ranking dropped 19 spots when quality was considered in combination with quantity.

Journals that rose notably in ranking when citation frequency was considered with article count included JASA, Operations Research, IJF, Review of Economics and Statistics, and Journal of the Royal Statistical Society–Series C (JRSC), each of which rose to the top 10 in h-index. Journal of the Royal Statistical Society–Series A (JRSA),Management Science, and American Mathematical Monthly entered the top 20 when impact was considered. These rises were not necessarily surprising; only JRSA, IJF, and JRSC ranked worse than 24th in overall h-index.

For contributors from OR, JORS, EJOR, Operations Research, and Interfaces represent the top-four OR journals in terms of SA impact: only these journals have published at least nine articles with at least nine citations, and only these rank among the top-10 journals in the subdiscipline.

Results: Institutions

Table 3 lists the top-40 institutions according to each metric. Although ordered based on article count, the table includes any institution that ranked in the top 40 in any of the six metrics.

Table 3 Among the top-40 institutions, Lancaster University produced the most sports analytics articles (a), proportionally adjusted articles (ap), citations (c), and proportionally adjusted citations (cp), and had the highest h-index and proportionally adjusted h-index (hp).

Of the 648 institutions that contributed articles, only 183 contributed more than two. Only 22 schools contributed at least 10 articles, and only 78 contributed at least 5. Thirteen countries and five continents are represented, lending support to the conjecture of Koning et al. (2003) about the international diversity and ubiquitous nature of the field's research interest.

The leading institutions were British: Lancaster University and the University of Salford. Washington State (WSU) led US schools, followed by Florida, DePaul, Stanford, and Michigan. Australia's Swinburne University of Technology (SUT), the Royal Military College of Canada (RMCC), and Simon Fraser also were in the top 10. Of the top 22 institutions, 10 were not US based.

The same 12 schools—the above 10 plus Bowling Green and Northeastern—comprised the top 12 in both total and proportional article counts. Lancaster and Salford again led in proportionally adjusted articles, with 22.5 and 14.7, respectively; RMCC was third. As a comparison, in the 2009 Rothkopf rankings (Fricker 2009) the top institution contributed 9.06 proportionally adjusted OR practice articles in just two journals during just seven years. A total of 9.06 SA publications would have ranked 13th over the entire sample of journals and years covered by the current search. This implies that the contributions of even the leading institutions in SA are not that large by comparison to a more well-established related discipline—applied OR.

Lancaster and Salford also ranked first in citations, and WSU was fourth, regardless of whether c or c_p was used. Carnegie Mellon, IBM (the only nonuniversity included, but nevertheless an oft-cited business institution contributor to SA), Georgia Tech, Imperial College London (ICL), Universidade Federal Flumenense, SUT, and the University of Maryland Baltimore County (UMBC) also ranked in the top 10 in citations received. In terms of proportional citations, Lancaster, Salford, IBM, WSU, and UMBC were the top five, with Ecole Polytechnique Fédérale De Lausanne, the University of Victoria, the University of Twente, SUT, and the University of Antwerp rounding out the top 10. Thus, 7 of the top-10 most-cited institutions (proportionally adjusted) were not US based.

In terms of h-index, the top contributors to SA have been Lancaster, WSU, Salford, SUT, ICL, and Victoria, meaning that five of the top six are non-US universities. The composition of the first four is the same when citations are proportionally adjusted, with Victoria, Cal State–Long Beach, and Harvard tied for fifth. Lancaster's leading h and h_p values are only 14 and 9, respectively, which are fairly small values. Only 55 institutions have at least four articles with at least four citations. When compared to the size of the data, these findings emphasize the subdiscipline's fragmented nature, as well as Wright's (2009) observation regarding the infrequency with which many SA authors publish such research.

A closer look at articles from the leading institutions reveals that a school's contributions often come from one (or a very few) authors. For example, Michael B. Wright and Robert Simmons have contributed 18.3 of Lancaster University's institution-leading 22.5 total. Rodney Fort, William J. Hurley, Stephen R. Clarke, and James Albert account for the vast majority of the contributions from WSU, RMCC, SUT, and Bowling Green, respectively, with the remaining proportional authorships often largely (if not fully) accounted for by coauthors of the same paper. Raymond T. Stefani of Cal State–Long Beach accounts for all of his institution's articles, and John S. Croucher from Macquarie University (Australia) nearly does so for his university. These observations emphasize that SA has been and (or) still is the bastion of a small and widely distributed group of individuals.

Summary and Conclusion

An expansive search of 140 analytics journals reveals a sizable and growing set of SA literature. This research appears concentrated in a list of approximately 30 journals, led largely by those from the economics field. The four OR journals with the most substantial SA impact are EJOR, JORS, Operations Research, and Interfaces, with Interfaces being the US-based OR outlet that most frequently publishes such research.

Examination of leading institutional contributors indicates strong international diversity and non-US presence in the subdiscipline, led by the universities of Lancaster and Salford in the UK. The prevalence of non-US schools points to possibly more ready acceptance of SA research outside the United States. Investigating the presence of international differences in how publishing in SA is viewed—by editors, current and prospective authors, promotion and tenure reviewers, institutions, or even the analytics or sports communities in general—is a potential avenue for future research. A look at the contributions from leading institutions suggests a field in which research is being performed by a few contributors at each school, supporting Wright's (2009) comments regarding its fragmented nature.

This work represents an effort to better define a field with growing research interest. Although contributions to research include books, editorships, and conference presentations, this study concentrates on the most recognized medium for academic research: refereed journal articles. The findings provide some guidance for those researching SA or considering doing so. They also hopefully provide a measure of definition and advancement for the field.