The Dutch Approach in Selecting Olympic Speed Skaters

The increased performance density at the top of many traditional elite sports has started to cause major challenges when comparing performances of athletes and deciding on winners. For example, performance differences between athletes are increasingly often within the error margins of the measuring systems, making it impossible to determine the winner.

The challenges lay not only in comparing results during competitions, but also in determining the selection process of who gets to compete in the first place. In the case of Dutch Olympic speed skating, the pool of highly competitive athletes is large. Because only a limited number of athletes is allowed to represent each country, the Olympic selection process is a precarious affair.

Just before the 2010 Winter Olympics in Vancouver, we learned about the selection challenge facing the Royal Dutch Speed Skating Federation (KNSB). We offered our ideas for a decision support system that can aid in finding an “optimal” selection with the highest probability of winning medals to the federation’s sport director Arie Koops. It took more time to obtain his email address than to convince him of our ideas. Indeed, he was immediately excited, and from that moment we have been closely involved in the Olympic selection process of Dutch speed skaters. Bertus Talsma was appointed by ORTEC in 2014 to lead the development of said decision-support system.

The cooperation of the University of Groningen, ORTEC Sports and the Royal Dutch Speed Skating Federation has resulted in a system that is now broadly supported by both the athletes and the federation’s supporting staff. Legal challenges to the selection decisions, previously troublesome, are now a thing of the past. During the last two Olympic Winter Games – Sochi (2014) and PyeongChang (2018) – the Dutch squad reached Dutch Olympic Committee technical director Maurits Hendriks’ objective of achieving the top five on the Winter Olympics medal table (Figure 1), with 23 of the 24 medals in 2014 in speed skating, and 16 of the 20 in 2018.

Selection Precariousness

During the 2018 Winter Olympics, the Netherlands received 38 “speed skating starting positions” – 19 for female and 19 for male skaters – across seven events, namely, for 500 (3), 1,000 (3), 1,500 (3), 3,000 (3) and 5,000 (2) meters for women, and 500 (3), 1,000 (3), 1,500 (3), 5,000 (3) and 10,000 (2) meters for men. (The number of starting positions is listed in parentheses.) The recently introduced mass start competition allows two Dutch skaters for both women and men, while three skaters comprise an entry in the team pursuit event.

Figure 1: Winter Olympic Medal Rankings, 2014 and 2018.

For each event, the Netherlands has a large pool of potential medal winners, but only 10 female and 10 male speed skaters are allowed to represent the country. Because the number of starting positions exceeds the number of skaters, some skaters will need to compete in multiple events, and it is therefore not possible in general to select the best athlete for each starting position. We could restrict ourselves to six events, because the team pursuit members have to qualify themselves for a distance event first. In fact, to reach the top 5 in the Olympic medal table, it may be optimal to select “generalists” rather than “specialists.”

It is worth noting that this contrasts with how the World Single Distances Speed Skating Championships (WSDC) are organized. This event has the same program as the Olympics, but the number of starting positions equals the number of participating skaters. In this case, the selection problem is straightforward: simply select the highest-ranked skaters of the trials prior to the tournament for each separate event, which is in fact how it is done in practice.

Obviously, a solution for this selection problem strongly depends on the choice of an operational translation of “reaching the top 5” objective, while satisfying the above quota restrictions. We have chosen to calculate “Olympic medal winning probabilities” for the potential Dutch skaters. It is precisely at these points where our expertise of statistics and operations research (O.R.) are valuable. However, interestingly, the implementation of our initial calculated solution turned out to be not straightforward, as we will discuss later. First, we provide some analysis of the top performance congestion mentioned above.

Gould’s Analysis of the Performance Congestion

Why are the performance differences at the top so small today? Although we probably all perceive elite athletes running faster and jumping higher (see, e.g., [1]), it may be less well known that the differences between top performances tend to decrease with time. An interesting paper by Haake et al. [2] contains graphs of eight classic field events, showing that after World War II the “performances have grown dramatically, while subsequently plateaued during the late twentieth century.” Figure 2 shows box plots of speed skating finishing times, one box per season. The decreasing trend of these results notwithstanding, it is clear from these graphs that the differences between the best and the worst finishing times in one season gradually decrease.

Figure 2: Box plots of finishing times: men (top) and women (bottom); normalized to 500m times (for example, 1,500m times are divided by 3).

What does the analysis of the American evolutionary biologist Stephen Jay Gould tell us on this topic [3]? Free representation of Gould’s hypothesis: When complex systems improve over time and when their best performers always stay playing by the same rules, then these performances equilibrate, and their variations decrease. There will be a constant improvement in the level of competition due to just practicing, called by Gould the maturation process. Therefore, more and more the limits of what is humanly possible are reached, leading to a certain leveling of performances at the top, and extreme performances (i.e., performances that are much better than those of rival athletes) become rarer.

Gould discovered this phenomenon for his favorite sport – American baseball. He observed the disappearance of .400 hitters in Major League Baseball, i.e., players who were able to get a hit in 40% or more of their official at bats over the course of a season. Such batting averages were fairly commonplace in the 1920s and 1930s but have not occurred since 1941. Commentators used the absence of such events to argue that the overall level of the baseball players had gone down. However, Gould came up with the opposite explanation, namely that the overall level had increased. According to Gould, the performance ceiling results in congestion toward that ceiling and an increasing rarity of extreme events. It is therefore no wonder that the differences between finishing times, for speed skating among others, eventually become so small that they are within the error margins of the measuring systems and become in fact incomparable. This has already happened more than once in practice. For example, the Dutch speed skater Koen Verweij lost the 2014 Olympic gold medal for the 1,500-meter race against Zbigniew Bródka from Poland with a difference of 0.003 seconds, well within the error margin of the official measuring system. In our opinion, both athletes should have been awarded a gold medal; according to the official timekeepers, such differences are indeed within the error margins and therefore insignificant.

An Ambitious Academic Solution

As previously mentioned, the leading objective of the Dutch Olympic Committee is reaching the top 5 in the Winter Olympics medal table. To make this objective computable, we did what O.R. professionals always do: collect data, design models and algorithms, and apply scenario analysis for analyzing the stability of the solution in relation to possible uncertainty of the data parameters (e.g., importance weights). A crucial ingredient for the optimization of the number of medals was to answer the question: What exactly do we mean by “winning probabilities” and how do we estimate them? For this problem we used the finishing times from international tournaments from the last two seasons, giving more weight to the more recent tournaments.

One of our tricks was the introduction and use of so-called AV-5 values. For any distance race, the AV-5 value of a skater is the difference between their actual finish time and the average of the first five finishing times of that distance. (The reason for taking AV-5 values instead of, for example, AV-4 or AV-6 values was rather arbitrary, although scenario analysis with AV-5 showed the most stable outcome; see [4] and [5].) This normalization of race times corrects for circumstantial differences between tournaments, such as different altitudes of competition accommodations; the high-altitude rinks of Salt Lake City and Calgary are the purveyors of personal and world records.

For all international skaters in our dataset, statistical distributions of the AV-values were made for all relevant events/distances. These distributions were modeled as log-normal distributions, with which race simulations were performed: 5,000 per distance. For each run, the simulation results were first ranked, and thereafter the percentage of the 5,000 runs a Dutch skater finishes within the first, second or third position were aggregated to a “winning probability” for that distance, using a prioritization on gold over silver over bronze, matching the overall goal of the Speed Skating Federation. The matrix (one for women and one for men) that contained the Dutch skaters and the “winning probabilities” for the various distances was then called the “performance matrix.” This was the statistical side of medaling, namely the calculation of the values of the objective parameters.

The next step was the design of an optimization model that could output an “optimal” selection. It may be no surprise that we used a simple integer linear optimization model. Its basic input ingredients are schematically depicted in Figure 3, which shows a complete bipartite graph with the names of the skaters and various distances (the numbers in parentheses indicate the number of starting positions at that distance). The constraints of the model correspond to the various quota restrictions on the number of participants at the various distances and on the size of the teams (10 men and 10 women). The model has two logical constraints used to couple the “distance participation” with the “team participation.” Maximizing the objective function leads to a team of skaters with a highest total Olympic medal-winning probability. Simple and clear – case closed. At least that is what we expected.

The Gap Between Algorithms and Refractory Practice

Unfortunately, the Royal Dutch Speed Skaters Federation turned out to have interests in more than just collecting Olympic medals. The acceptance of our approach would mean the end of the usual very attractive trials with full stadiums, and our statistical simulations and optimization algorithms would not put funds into the Royal Federation. Luckily, Arie Koops remained optimistic about our contribution and put a new problem on our plate: How can we include the popular and lucrative trials?

The Practical Solution

Dutch speed skating trials take place in an ultramodern speed skating hall located in the city of Heerenveen during four days in December, prior to the Olympic Games in February. During “historical” winters, the waters around Heerenveen were covered with crystal clear ice, inspiring the famous ancient Dutch painters. Although these romantic winters are only memories, and the last 200k-long epic Eleven City Speed Skating Race was organized more than 24 years ago (one of the participants was Gerard Sierksma), speed skating is still in the collective memory of the Dutch. Indeed, the speed skating trials attract paid ticketholders for four days of full stadiums.

How could we incorporate the trial results in the decision-support system? Koops’ proposal was to make so-called selection rankings, one for the men and one for the women, of the 16 starting positions, starting with the first position of the distance with the greatest chance of a Dutch Olympic (gold) medal, and so on until place 16 with the distance position with the smallest chance. We decided to base the ordering of the selection ranking on the values of the performance matrix. This choice is certainly questionable because the matrix contains person values, whereas the selection ranking is based on country values, i.e., probabilities that the Netherlands wins a medal. The reader is challenged to provide a better choice.

When the trials are finished, the selection rankings can be completed. The first position of the ranking is entered with the name of the skater that finished first on the distance of that first position, and so on. As soon as 10 different names of skaters have been entered, then the other positions on the selection ranking are entered using no other names than the ones already filled out, taking into account the rankings of the trials results. It should be noted that the Federation has the right to deviate in a motivated manner from the outcome of the procedure. This may happen, for example, when a “highly valuable” skater – say, for the team pursuit competition – becomes sick just before the trials. The procedure serves as a decision support system. 

High Expectations and Disappointing Realizations

Selection decisions are based on current expectations. Nobody knows ahead of time how athletes will perform during a race. Just like in all facets of life, in sports, high expectations always lead to high interest of the media, sponsors and of course the fans. But it can also turn out differently: High expectations can lead to major disappointments, and low expectations to surprising realizations. The following two events are examples.

Vancouver 2010 Olympics. The great Dutch male speed skater Sven Kramer – who has nine Olympic medals, four of them gold – was destined throughout Holland to win the 10k, the longest Olympic speed skating distance. Not only did he have the highest winning probability (100%) in the performance matrix, he was also No. 1 on that distance during the December trials. With just a quarter of the race to go, his lead on the next skater in the Olympic race was practically unbridgeable. It took a split second to destroy all sky-rocketing expectations; Kramer took a wrong turn (during each lap skaters change lanes) and was disqualified for riding in the wrong lane during the final laps. Despite two more attempts, Kramer never won the Olympic 10k beyond 2010.

Heerenveen 2017 trials. On the final last day of the Olympic trials, the last distance race was the women’s 5k. Being surrounded by “royal” speed skating directors on the grandstand in Heerenveen was not only a privilege, it was also revealing. Our grandstand neighbors argued that this last race was redundant in regard to the selection ranking: The two women skaters preparing for their race were considered to have no chance to enter the selection ranking. We knew better; our calculations showed something else. One of the skaters, Esmee Visser, had a high value in the performance matrix and in our optimal integer linear model solution. About two months later, she was the glorious winner of the 5k Olympic gold medal.

Cooperation for Contribution

In the hectic world of elite sports with small performance differences and limited number of starting places for competitions, selection procedures must be carefully carried out with broad support from the athletes, their accompanying teams, the media and the fans. Otherwise, legal challenges are to be expected. Indeed, selection procedures must be legally watertight. And, maybe equally important, they should be controllable and repeatable, in the sense that when the procedure is repeated later, the result does not differ. During the more than eight years of cooperation with ORTEC and the University of Groningen, the Dutch Olympic Committee has not faced any major legal challenge on its Olympic selection decisions [7]. The stands of the popular speed skating trials are expected to remain full, because the expectations for Dutch Olympic success on the ice will remain high. Hansje Brinker’s [6] finger in the dike will not protect the Dutch against the consequences of the melting ice on the poles and the softening iceless winters. The artificial ice in the 19 (!) speed skating rinks behind high dikes is the Dutch answer to this aspect of climate change. Our contribution was small in this respect, but certainly more effective than Hansje Brinker’s finger.

References

1. G.H. Kuper and E. Sterken, 2003, “Endurance in Speed Skating; the Development of World Records,” European Journal of Operational Research, Vol. 148, pp. 293-301.
2. S. Haake, D. James and L. Foster, 2015, “An Improvement Index to Quantify the Evolution of Performance in Field Events,” Journal of Sports Sciences, Vol. 33, No. 3, pp. 255-267.
3. S.J. Gould, 1996, “Full House: The Spread of Excellence from Plato to Darwin,” Three Rivers Press: New York.
4. https://en.wikipedia.org/wiki/Hans_Brinker,_or_The_Silver_Skates
5. J. Robinson, 2018, “The Secret of Dutch Speed Skating – It’s Not What You Think,” The Wall Street Journal, Feb. 23.
6. G. Sierksma, 2017, “Introduction: Olympics Track & Field,” J.J. Cochran, J. Bennett and J. Albert, eds., “The Oxford Anthology of Statistics in Sport,” Vol. 1, Oxford University Press.
7. B. Talsma, G. Sierksma and M. Turkensteen, 2017, “The Growing Problem of Comparing Elite Sport Performances: The Olympic Speed Skating Case,” Journal of Human Sport and Exercise, Vol. 13, pp. S892-S907.
8. B.G. Talsma, 2013, “Performance Analysis in Elite Sports,” Doctoral dissertation, University of Groningen, SOM Research School.