Revenue in First- and Second-Price Display Advertising Auctions: Understanding Markets with Learning Agents

1. Introduction

Real-time bidding is a means by which advertising inventory is sold on a per-impression basis (Choi et al. 2020). Display ad auctions are a prime application where advertisers compete for ad impressions. Digital advertising is an industry with $129 billion spent in the United States in 2019, which is more than half of total media advertising spending.¹ In 2022, more than 90% of all digital display ad spend was transacted via real-time bidding (Yuen 2022). Moreover, the real-time bidding market size is expected to grow substantially in the next years (Tunuguntla and Hoban 2021). Google Ad Manager, OpenX, PubMatic, and Xandr are among the largest ad exchanges operating these auctions. Publishers and advertisers do not participate directly in these ad exchanges. The fact that these auctions need to be run in milliseconds requires service providers with specialized information systems. Supply-side platforms (SSPs) help publishers sell ad inventory, demand-side platforms (DSPs) help advertisers bid for that inventory, and ad exchanges are the marketplaces where SSPs and DSPs interact to buy and sell ad impressions in real time. Not surprisingly, there is a growing literature in information systems and management sciences on display ad auctions (Sayedi 2018, Choi et al. 2020, Mehta et al. 2020, Christopher et al. 2022, Balocco et al. 2025).

In the past, most ad exchanges used the second-price auction. Consequently, most of the literature on display ad auctions focuses on repeated second-price auctions under different models (Ciocan and Iyer 2021, Conitzer et al. 2022, Chen et al. 2024). However, since 2019, several of them have begun to experiment with first-price auctions, which are now used by the major advertising exchanges, most notably the Google ad exchange (Despotakis et al. 2021). Google’s ad exchange controls approximately half of the market globally.² Google claimed that this switch was to help advertisers by simplifying how they buy online ads.³ One important development that motivated the move to first-price auctions was header bidding.⁴ Header bidding is a widely employed technology in which a publisher solicits first-price bids from a set of SSPs that run their own isolated auction among their DSPs. These bids are injected into the final competition run by the ad exchange, where they get compared with other bids on the ad exchange or direct deals. The header bidding was organized as a first-price auction, whereas the ad exchanges ran a second-price auction, which led to complexities in the bidding strategy. It was also common to use the result to inform the minimum price (also known as price floor) in a subsequent second-price auction in another exchange. One complaint by advertisers regarding second-price auctions was regarding these price floors that sellers introduced to enhance yield (Geradin and Katsifis 2019). Advertisers considered price floors as opaque, and there were concerns about manipulation on the part of the sellers.⁵ Such concerns do not arise with first-price auctions, where advertisers pay what they bid rather than the second price.

Although first-price auctions were adopted by large exchanges, others still offer second-price auctions, and the choice of the auction format is a matter of ongoing debate. Some studies have observed decreased bid prices after the switch (Alcobendas and Zeithammer 2021), and some ad exchanges reported significant revenue losses after the switch to first-price auctions.⁶ All of them report that the adaptation of bidding strategies was not instant but took time. Obviously, if only one bidder starts to shade his bid below and the others do not, he would lose much more often. So, it takes time for participants to adapt. Overall, the empirical analysis of this policy change is challenging because of many confounding factors (Goke et al. 2021). Changes in the economy lead to changes in advertising spending over time. Ad exchanges switched at different times, which might have shifted some of the demand from one to another exchange. Several factors, such as the type of impression, the day of the week, the season, and the type of publisher, all affect prices. Because of changes in macroeconomic and microeconomic conditions, several confounding factors, and limited availability of data (such as advertiser valuations), it is difficult for ad exchanges to identify the causal impact of a change in their auction format on their revenue.⁷

The value of economic models lies in their ability to perform comparative statics, allowing for the analysis of how isolated changes in exogenous parameters affect economic outcomes while holding other factors constant (ceteris paribus). Even though the details of real-world display advertising markets, such as the exact valuations or the bidding strategies implemented, might differ, we aim to understand how auction format changes impact revenue in a model that captures key characteristics of display ad auctions.

Standard game-theoretical models of first-price and second-price auctions are Bayesian games in which agents are assumed to follow Bayes–Nash equilibrium (BNE) strategies that are determined ex ante. The celebrated revenue equivalence theorem predicts that the first-price and second-price auctions achieve the same revenue in expectation in a model with independent and identically distributed values (Vickrey 1961, Myerson 1981). However, display ad auctions are different in at least two important ways; the bidding strategies are learned, and the objective of the bidding agents is return on investment (ROI) rather than payoff (also known as a quasilinear (QL) utility function). ROI helps compare which campaign gives “more bang for the buck,” and it is a natural performance metric in environments where an advertiser decides on a campaign budget a priori and wants to spend it most effectively without getting money back.⁸

1.1. Learning Agents and Algorithmic Collusion

In display ad auctions, bidding is fully automated, and bids need to be submitted in milliseconds. In a first-price auction, bidders want to shade their bid below their valuation for an impression to make a profit. However, they cannot simply follow a predetermined equilibrium bidding strategy. Without knowing the valuations of others or even only a prior distribution of valuations, deriving an equilibrium bidding strategy is impossible. Automated bidding agents need to learn a good strategy while bidding. Not surprisingly, after the move to first-price auctions, a growing literature using (multiarmed) bandit algorithms in display ad auctions has emerged (Han et al. 2020, Tilli and Espinosa-Leal 2021, Ai et al. 2022, Zhang et al. 2022, Liang et al. 2023, Wang et al. 2023, Zhang and Luo 2023, Kumar et al. 2024). Bandit algorithms are fast and adapt to changes in the environment quickly. So, their use for automated bidding is natural. There is extensive literature on such algorithms, which are widely used for online optimization problems (Lattimore and Szepesvári 2020). Empirical evidence from large-scale auction data further suggests that observed bidding behavior is consistent with no-regret learning dynamics; see, for example, Nekipelov et al. (2015) and Noti and Syrgkanis (2021).

However, bidding in repeated auctions is not a standard online learning problem. Advertisers compete against each other in a game-theoretical environment, and their actions influence each others’ profit. It is not clear that algorithms designed for online problems would perform well in game-theoretical environments. Actually, it is well known that online learning algorithms in games may cycle, end up in off-equilibrium states, or even end up in chaotic dynamics (Sanders et al. 2018). The analysis of learning dynamics in games can be arbitrarily complex in general (Andrade et al. 2021), and not much is known about online learning algorithms in auction games.

Among the off-equilibrium outcomes of learning dynamics, algorithmic collusion captured the most attention in the broader public (Calvano et al. 2020, Hansen et al. 2021, Klein 2021, Abada and Lambin 2023). Algorithmic collusion in pricing games, such as a Bertrand oligopoly, refers to algorithms that converge to a supracompetitive price higher than the Nash equilibrium. The topic got the attention of regulators (OECD 2017, Pošćić and Martinović 2020) as it might jeopardize consumer welfare when algorithms are used for pricing on digital platforms. The jury on algorithmic collusion in pricing is still out, and there are arguments for but also against this being a concern in practice in pricing (den Boer et al. 2026).

Display ad auctions are highly automated markets, and algorithmic collusion might arise here as well. In such markets, we have millions of repeated interactions among bidders for similar types of impressions. For example, the Google Display Network serves 24.17 billion impressions per day.⁹ Indeed, Banchio and Skrzypacz (2022) reported the results of numerical experiments with bidding agents based on specific Q-learning algorithms in a complete-information model, where the value of bidders is public knowledge and the competition does not change. Interestingly, they find that first-price auctions with no additional feedback lead to tacit-collusive outcomes, whereas second-price auctions do not. Low revenue in first-price auctions is an obvious concern to ad exchanges that might lose billions of dollars. However, this is also important to understand for advertisers that might either decide to use such algorithms or exploit the off-equilibrium behavior of others in order to increase their payoff. Therefore, a key question in this paper is as follows. Can we assume that the repeated interaction of bidding agents in display ad auctions leads to an equilibrium? If this is not the case, then this is a concern for ad exchanges, advertisers, and publishers alike.

1.2. Return on Investment

If learning agents that maximize payoff converge to equilibrium, then the celebrated revenue equivalence theorem (Vickrey 1961) holds, and we can assume that the change from second-price to first-price auctions is without loss. Given that bidding is highly automated, advertisers need to rely on specialized demand-side platforms to do the bidding on their behalf. DSPs are independent firms, and they do not just maximize the profit of the advertiser without a limit. Rather, they are given a budget by the advertiser for a specific marketing campaign. The advertiser does not want any money back but wants to have the budget used most effectively. There is a large (academic and practitioner) literature on advertising and in particular, on display ad auctions showing that return on investment is a primary objective (see Section 2.1). Revenue equivalence does not need to hold if bidders do not maximize payoff. It is interesting to understand the revenue ranking of auctions if bidders maximize ROI rather than payoff in such advertising markets. Unfortunately, deriving equilibrium for such nonquasilinear utility functions analytically leads to intractable nonlinear differential equations. For such problems, we do not even have an exact mathematical solution theory, and no equilibrium strategies are known for bidders that maximize ROI. We derive an analytical solution for uniform prior distributions and draw on recent breakthroughs in equilibrium computation to find equilibrium for alternative distributions and asymmetric settings.

1.3. Contributions

We make two main contributions. Our first contribution is to characterize equilibrium behavior in single-item auctions when bidders maximize return on investment rather than quasilinear payoff. Prior work has shown equilibrium existence or derived price-of-anarchy bounds on the total value obtained by advertisers (Aggarwal et al. 2019, Chen et al. 2024). However, computing equilibria in such environments is challenging, and closed-form solutions are typically unavailable when bidders do not maximize quasilinear payoff (Aggarwal et al. 2024). For uniform priors, we derive equilibrium bidding strategies analytically. For more general distributions and asymmetric settings, where analytical solutions are not available, we leverage recent advances in equilibrium computation to compute and evaluate equilibrium strategies. These numerical techniques represent state-of-the-art methods for solving auction games that were previously difficult to analyze (Bichler et al. 2025).

This equilibrium analysis allows us to address a nontrivial benchmark question in auction theory and information systems research: whether lower revenues in first-price auctions necessarily point to off-equilibrium algorithmic collusion or whether they can arise from equilibrium behavior under bidder objectives that differ from payoff maximization. In the benchmark model studied here, we find that first-price auctions generate lower expected revenue than second-price auctions when bidders maximize ROI. The revenue difference is largest under low competition and decreases as the number of bidders increases. Thus, the standard revenue-equivalence intuition does not extend to ROI-maximizing bidders in this setting. The results identify an equilibrium mechanism through which lower first-price revenues can arise even without collusive bid suppression.

Our second contribution is to analyze whether standard learning algorithms converge to these benchmark equilibria or instead, generate systematic off-equilibrium outcomes. Motivated by the use of bandit methods in automated bidding, we study several widely used learning algorithms both in symmetric environments where agents use the same algorithm and in heterogeneous environments where different algorithms compete against one another. The learning agents do not know the prior distribution of valuations; rather, they learn through exploration and exploitation from repeated interaction.¹⁰

Across the benchmark environments studied, the learning dynamics converge toward the computed equilibrium strategies and do not exhibit systematic bid suppression. This finding is nontrivial because learning dynamics in games can cycle, fail to converge, or generate complex off-equilibrium behavior. At the same time, our claim is deliberately limited to the stationary repeated-game benchmark. Real-world display advertising markets are not fully stationary; supply, demand, bidder populations, impression characteristics, value estimation pipelines, budget pacing, and feedback structures may change over time. Our analysis, therefore, does not rule out algorithmic collusion or instability in richer marketplace-level environments. Rather, display advertising markets generate millions of closely related auctions over short time periods, during which the relevant competitive environment may be approximately stable. The stationary repeated-game benchmark captures this short-run interaction and gives learning algorithms a clear opportunity to discover and sustain off-equilibrium bid-suppression patterns. Because we do not observe such patterns for the learning algorithms and benchmark environments studied, the simulations are consistent with the equilibrium mechanism identified above; lower first-price revenues in this setting can arise from ROI-based equilibrium behavior without invoking algorithmic collusion. This contrasts with prior numerical evidence showing that specific Q-learning implementations may generate low-revenue outcomes in first-price auctions (Banchio and Skrzypacz 2022).

2. Related Literature

Auctions and in particular, online auctions have a long history in the information systems literature (Pinker et al. 2003, Bichler et al. 2010, Greenwald et al. 2010, Cason et al. 2011, Choi and Mela 2018) beyond what we can cover in this section. We draw on specific strands in the literature close to our key research question. First, we discuss utility models as they have been reported for display ad auctions. Second, we introduce relevant work on collusion in auctions. Finally, we cover the literature on equilibrium learning relevant to this paper.

2.1. Utility Models of Bidders in Display Ad Auctions

There is a large literature on display ad auctions and real-time bidding (Despotakis et al. 2021). The authors assume different utility models to describe the advertisers’ objectives. Originally, authors used a standard quasilinear utility function as is usual in auction theory (Edelman et al. 2007, Despotakis et al. 2021). However, more recent literature does not assume payoff maximization anymore. The ROI became a popular metric because of marketing budget allocation (Szymanski and Lee 2006, Borgs et al. 2007, Wilkens et al. 2017, Jin et al. 2018). Apart from payoff maximization, we analyze agents that learn to maximize expected ROI as a measure of the ratio of profit to cost:

where v denotes the valuation and p denotes the payment or cost.¹¹ One reason for ROI optimization is that advertising is usually delegated from the advertising firm (the principal) to an agent: the DSP (Yuan et al. 2014). This requires a sophisticated technical infrastructure that advertisers do not have. The agent might have an incentive to spend as much as possible of the principal’s money. To control the actions of the agent, the DSPs are given a budget by this advertiser (Bichler and Paulsen 2018). ROI measures how efficiently the investment or budget for a campaign is used by the DSP and has, therefore, become a central objective for DSPs. The advertiser does not get money back, even if the payoff of an impression is low, and the DSPs are compared by the ROI that they achieve. In some cases, advertisers also talk about a related metric called return on spend (ROS) that we discuss in Online Appendix C. The main results that we show for ROI in the main part of this paper also carry over to ROS with budgets (cf. Online Appendix D.3).

2.2. Collusion in Auctions and Pricing

Collusion is a key concern in the literature on auctions, and there is much literature about which auction rules are more susceptible (Fabra 2003, Skrzypacz and Hopenhayn 2004, Blume and Heidhues 2008). It is often a topic in second-price auctions and less so in first-price auctions (see Krishna 2009). Here, we deal with tacit algorithmic collusion where low off-equilibrium prices arise without explicit communication among bidders. In particular, we want to understand algorithmic collusion (i.e., collusion that arises from the repeated interaction of learning agents without them being programmed for explicit collusion).

Algorithmic collusion was first discussed in the context of oligopoly pricing. Early treatments go back to Greenwald and Kephart (2000). Calvano et al. (2020) analyzes firms that play an infinitely repeated game, pricing simultaneously in each stage and conditioning their prices on history. They find that Q-learning algorithms consistently learn to charge supracompetitive prices. These prices are sustained by collusive strategies with a finite phase of punishments followed by a gradual return to cooperation. Klein (2021) shows how Q-learning is able to learn collusive strategies when competing algorithms update their prices sequentially; as opposed to Calvano et al. (2020), for collusion to occur in their sequential-move setting, they do not require that algorithms can condition on own and competitor past prices. Other related papers on algorithmic collusion in oligopoly pricing were written, for example, by Hettich (2021) and Asker et al. (2022).

Banchio and Skrzypacz (2022) were the first to analyze collusion in the context of real-time bidding in display ad auctions. The environment is different from the stylized oligopoly pricing models. They find that when bidders use specific Q-learning algorithms to determine their bids, the auction format and other design choices can have a first-order effect on revenues and bidder payoffs. The revenues can be significantly lower in first-price auctions than in second-price auctions. In particular, first-price auctions with no additional feedback lead to tacit-collusive outcomes, whereas second-price auctions do not. However, a simple practical auction design choice—revealing to bidders the winning bid after the auction—can make the first-price auctions more competitive again. Yet, the paper provides evidence that with specific implementations of symmetric Q-learning agents, the revenue in display ad auctions can be lower than that of the second-price auction.

2.3. Numerical Methods for Equilibrium Computation

Equilibrium analysis in Bayesian games relied exclusively on analytical derivations so far. Unfortunately, the equilibrium problem in most game-theoretical problems results in nonlinear differential equations for which we do not have an exact mathematical solution theory in general. This is why equilibrium predictions are restricted to rather simple models, such as single-object auctions with quasilinear utility functions. Although numerical methods are widely used in engineering and science to solve systems of differential equations, their success in auction theory was rather limited and plagued by numerical instabilities (Fibich and Gavish 2011). Recent breakthroughs now allow us to compute an equilibrium in models for which there is no analytical solution to the resulting differential equations.

As a numerical technique to compute equilibrium for nonquasilinear utility models, we draw on simultaneous online dual averaging (SODA) (Bichler et al. 2025), a gradient-based algorithm that was shown to be very versatile and allowed for the computation of BNE in a large variety of different auction models. Whenever SODA converges to a strategy profile, it has to be an equilibrium. This allows us to verify equilibrium in situations where no analytical solution is known, in particular in those where the objectives of the bidding agents are different from payoff maximization in a quasilinear utility function.

Our contribution differs from the literature on no-regret learning in auctions, which typically studies regret guarantees, welfare bounds, or convergence properties of specific learning algorithms under a given auction environment. We do not propose a new learning algorithm or prove a general convergence theorem. Instead, we use standard bandit algorithms as a behavioral benchmark to ask whether learning dynamics in the ROI-based auction model move toward the computed equilibrium or toward off-equilibrium bid-suppression outcomes. The main contribution is, therefore, an equilibrium and revenue-comparison result; ROI-based bidder objectives can reverse the standard revenue-equivalence intuition, and the learning simulations show that this equilibrium mechanism is reproduced by several common bandit algorithms in the benchmark environment studied here.

3. Equilibrium Strategies

We now formalize the equilibrium problem for the different bidder utility models introduced in Section 2.1. The agent that we model corresponds to the bidding policy layer of a demand-side platform conditional on impression-level value estimates, which are taken as primitives. We abstract from how these values are produced (e.g., via supervised learning) and from campaign-level budget management or pacing, which operate at different layers and timescales in real advertising systems. This single-impression benchmark follows the standard symmetric independent private values framework used in auction theory and information systems research, and it serves as a ceteris paribus setting to isolate two mechanisms central to our analysis: return on investment-based bidding objectives and learning via bandit algorithms.

Within this framework, we derive equilibrium conditions and show that ROI maximization leads to nonlinear differential equations, which we solve analytically where possible and analyze numerically otherwise. The results should be interpreted as equilibrium and learning benchmarks that clarify relative revenue implications across auction formats rather than as a full representation of an end-to-end DSP implementation. Assumptions and limitations of our model are discussed in Online Appendix A.

3.1. Model

Formally, single-item auctions are modeled as incomplete-information games $\mathcal{G}=(\mathcal{I},\mathcal{V},\mathcal{A},F,u)$ with continuous type and action spaces. Each bidder $i\in \mathcal{I}=\{1,\dots ,N\}$ observes a private value (type) $v_i\in {\mathcal{V}}_i\subset \mathbb{R}$ drawn from some prior distribution with cumulative distribution function (CDF) F over $\mathcal{V}≔{\mathcal{V}}_1\times \cdots \times {\mathcal{V}}_N$. With a slight abuse of notation, we also denote the marginal distribution over ${\mathcal{V}}_i$ with F because we only consider symmetric bidders with independent and identically distributed valuations. The probability density function is denoted by f. After observing their private types, bidders submit bids $b_i\in {\mathcal{A}}_i\subset \mathbb{R}$ and receive their payoffs as given by the (ex post) utility function $u_i:\mathcal{A}\times {\mathcal{V}}_i\to \mathbb{R}$ with $\mathcal{A}={\mathcal{A}}_1\times \cdots \times {\mathcal{A}}_N$.

A pure strategy is a function ${\beta }_i:{\mathcal{V}}_i\to {\mathcal{A}}_i$, mapping a bidder’s value to an action. Given a strategy profile $\mathit{\beta }=({\beta }_1,\dots ,{\beta }_N)$, the expected (ex ante) utility is defined by ${\tilde{u}}_i({\beta }_i,{\mathit{\beta }}_{\mathbf{-}\mathbf{i}})≔{\mathbb{E}}_{v_i\sim F}\left[{\overline{u}}_i\right({\beta }_i\left(v_i\right),{\mathit{\beta }}_{\mathbf{-}\mathbf{i}},v_i\left)\right]$ with the ex interim utility ${\overline{u}}_i(b_i,{\mathit{\beta }}_{\mathbf{-}\mathbf{i}},v_i)≔{\mathbb{E}}_{v_{-i}}\left[u_i\right(b_i,{\mathit{\beta }}_{\mathbf{-}\mathbf{i}}\left({\mathit{v}}_{\mathbf{-}\mathbf{i}}\right),v_i\left)\right]$. We say that a strategy profile $\beta$ is a BNE if no agent can increase their expected utility ${\tilde{u}}_i$ by unilaterally deviating:

In the remaining part of this section, we analyze this model for different utility models expressed by different ex post utility functions $u_i$. Let $\mathit{x}:\mathcal{A}\to {[0,1]}^N$ be the allocation vector, where $x_i\left(\mathit{b}\right)=1$ if bidder i gets the item (i.e., $b_i>{\max }_{j\ne i}{b}_j$ and $x_i\left(\mathit{b}\right)=0$ else) and $\mathit{p}:\mathcal{A}\to {\mathrm{ℝ}}^N$ is the price vector. As described in the previous section, we consider payoff-maximizing (or quasilinear (QL))

and ROI-maximizing agents

Note that depending on the payment rule, the price vector takes the values $p_i\left(\mathit{b}\right)=b_i$ in first-price auctions and $p_i\left(\mathit{b}\right)={\max }_{j\ne i}{b}_j$ in second-price auctions for bidder i receiving the item, and $p_j\left(\mathit{b}\right)=0$ for all other bidders.

The ROI objective in (ROI) is a ratio and therefore, is potentially unstable if prices approach zero. To ensure that utilities are well defined and bounded, we assume a strictly positive reserve price $r>0$ so that whenever an allocation occurs, the payment satisfies $p_i\ge r$. This assumption reflects the ubiquitous use of minimum bid floors in display advertising markets and rules out division by zero or unbounded rewards. Throughout the analysis, bidders with valuations below the reserve do not participate, and the ROI utility is uniformly bounded on the relevant strategy space. This boundedness is important both for equilibrium analysis and for the stability of the learning dynamics studied below.

3.2. Payoff-Maximizing Bidders

With quasilinear bidders (QL) that maximize payoff, the equilibrium bidding strategies in first-price and second-price auctions are well known. The second-price auction has a dominant-strategy BNE of bidding truthfully ($\beta (v)=v$). For the first-price auction, we get a closed-form solution for the (nontruthful) equilibrium bidding strategy via the first-order condition of the expected utility function (Krishna 2009). Given that all opponents play according to a symmetric, increasing, and differentiable strategy $\beta$, the utility for bidder i submitting bid b with valuation v is given by ${\overline{u}}_i(b,{\mathit{\beta }}_{\mathbf{-}\mathbf{i}},v)=G\left({\beta }^{-1}\right(b\left)\right)(v-b)$, where $G({\beta }^{-1}(b))$ denotes the probability of winning with bid b. Using the first-order condition $\frac{d}{db}\overline{u}(b,{\beta }_{-i},v)=0$, one can derive the following ordinary differential equation (ODE):

If we assume independent uniformly distributed valuations for all N agents (i.e., $G(v)=v^{N-1}$), given a reserve price $r>0$ for valuations $v<r$, no bidder can make a positive profit and consequently, would not participate in the auction (${\beta }_i(v)=0$). For higher valuations, we can use the ODE to get a BNE with the initial condition $\beta (r)=r$. This leads to

See Krishna (2009) for the equilibrium strategy with general independent and identically distributed (i.i.d.) valuations. Note that the BNE is unique in this setting (Chawla and Hartline 2013). Revenue equivalence of the first-price and second-price auctions with i.i.d. private values and payoff-maximizing bidders is the central result in auction theory discussed in related textbooks.

3.3. ROI-Maximizing Bidders

As discussed earlier, ROI maximization is widely mentioned as the objective of advertisers. The first observation is that the second-price auction continues to be strategy proof as with payoff-maximizing bidders.

Proposition 1.

The second-price sealed-bid single-object auction is strategy proof for ROI-maximizing bidders.

Proof.

Consider bidder 1, and suppose that $p_1={\max }_{j\ne 1}{b}_j$ is the highest competing bid. By bidding $b_1$, bidder 1 will win if $b_1\ge p_1$ and lose if $b_1<p_1$. Let us assume that $b_1$ equals the value of this bidder. Now, suppose that he bids an amount $b_1^{\prime }<b_1$. If $b_1>b_1^{\prime }\ge p_1$, then he still wins, and his ROI is still $(b_1-p_1)/p_1>0$. If $p_1>b_1>b_1^{\prime }$, he still loses. However, if $b_1>p_1>b_1^{\prime }$, then he loses, whereas if he had bid $b_1$, he would have received a positive ROI. Thus, bidding less than $b_1$ can never increase his utility but in some circumstances, may actually decrease it. Similarly, it is not profitable to bid more than $b_1$. If he bids $b_1^{\prime }>b_1$, then there could be $b_1^{\prime }>p_1>b_1$, making his utility negative. If $b_1^{\prime }\ge b_1>p_1$, then this increase to $b_1^{\prime }$ would not change his ROI. □

Let us next discuss the first-price auction. Analogous to the model with payoff-maximizing agents, we can try to use the first-order condition to derive equilibrium strategies for ROI-maximizing bidders. The ex interim utility for bidder i with valuation v and bid b given the opponent’s strategy $\beta$ is defined by ${\overline{u}}_i(b,{\beta }_{-i},v)=G({\beta }^{-1}(b))\frac{v-b}{b}$. Similar to the first-price auction with quasilinear bidders, we can derive a first-order condition assuming symmetric strategies and obtain

This first-order nonlinear ODE is, in general, hard to solve analytically. But, if we restrict our analysis to independent, uniformly distributed valuations, we can derive a closed form for the equilibrium.

Theorem 1.

In a first-price sealed-bid single-object auction with reserve price $0<r<1$ and N ROI-maximizing bidders with symmetric independent private values (SIPV) uniformly distributed on [0, 1], the symmetric equilibrium strategy $\beta$ is given by

$$\beta (v)=\{\begin{array}{ll}\frac{v}{1-\log (r)+\log (v)}\hfill & \mathit{\text{for}} N=2\hfill \\ \frac{(N-2)v^{N-1}}{(N-1)v^{N-2}-r^{N-2}}\hfill & \mathit{\text{for}} N>2.\hfill \end{array}$$(5)

The proofs for this and the following results can be found in Online Appendix B.

Corollary 1

(Revenue Gap for Uniform SIPV with Reserve). Consider the setting of Theorem 1 with $v_i\sim \text{Unif}[0,1]$ i.i.d. and reserve $r\in (0,1)$.

1. Second-price auction. By Proposition 1, truthful bidding is optimal for both payoff- and ROI-maximizing bidders in the second-price auction. Hence, the expected revenue is

   $${\text{Rev}}^{\text{SP}}(r,N)=\mathbb{E}\left[\max \{r,V_{(N-1)}\}\hspace{0.17em}\mathbf{1}\{V_{(N)}\ge r\}\right]=\frac{N-1}{N+1}+r^N-\frac{2N}{N+1}\hspace{0.17em}{r}^{N+1},$$(6)
   where $V_{(N)}\ge V_{(N-1)}$ denote the top two order statistics of the N values.

2. First-price auction with ROI bidders. In the first-price auction, the symmetric ROI equilibrium bid $\beta$ is given in Theorem 1. Because $\beta$ is increasing on [r, 1], the winner is the highest-value bidder whenever $V_{(N)}\ge r$, and the expected revenue is

   $${\text{Rev}}_{\text{ROI}}^{\text{FP}}(r,N)=\mathbb{E}\left[\beta (V_{(N)})\hspace{0.17em}\mathbf{1}\{V_{(N)}\ge r\}\right]={\displaystyle {\int }_r^1\beta }(v)\hspace{0.17em}N{v}^{N-1}\hspace{0.17em}dv.$$(7)

Consequently, the revenue gap in the uniform benchmark can be written exactly as

$$\mathrm{\Delta }(r,N)≔{\text{Rev}}^{\text{SP}}(r,N)-{\text{Rev}}_{\text{ROI}}^{\text{FP}}(r,N)=(\frac{N-1}{N+1}+r^N-\frac{2N}{N+1}{r}^{N+1})-{\displaystyle {\int }_r^1\beta }(v)\hspace{0.17em}N{v}^{N-1}\hspace{0.17em}dv$$(8)

and $\mathrm{\Delta }(r,N)>0$ for all $r\in (0,1)$ and $N\ge 2$ (cf. Result 1).

Proposition 1 pins down second-price behavior (truthful bidding), yielding the closed-form revenue Expression (6); Theorem 1 pins down first-price behavior via $\beta$, yielding (7). Comparing (6) with (7) establishes the result for the uniform SIPV benchmark (Figure 1).

The result for uniform priors can be generalized as follows.

Theorem 2

(ROI Bids are Point Wise Below QL Bids in First Price). Consider a first-price sealed-bid single-object auction with reserve price $0<r<\overline{v}<\infty$ and symmetric bidders with independent private values on $[0,\overline{v}]$ with CDF F and continuous density f with full support. Assume the bidders play according to the strictly increasing, continuously differentiable Bayes–Nash equilibrium bid functions ${\beta }_{QL}$ (quasilinear/payoff) and ${\beta }_{\mathit{\text{ROI}}}$ (ROI). Then, ${\beta }_{\mathit{\text{ROI}}}(v)\le {\beta }_{QL}(v)$ for all $v\in [r,\overline{v}]$ with strict inequality for all $v>r$. Consequently, expected revenue with ROI bidders is (strictly) lower than with payoff-maximizing bidders.

3.4. Equilibrium Computation for Gaussian Priors and Asymmetric Settings

SODA leverages gradient-based techniques from online convex optimization (Shalev-Shwartz 2011) on distributional strategies (Milgrom and Weber 1985)—an extension of mixed strategies to Bayesian games—within a discretized version of the auction game. This approach has proven to be a versatile and efficient method for computing equilibria across a range of different auction settings (Bichler et al. 2025).

To apply the method, we discretize the underlying auction game by selecting equidistant points from the continuous action and type space (i.e., ${\mathcal{V}}^d=\{v_1,\dots ,v_n\}\subset \mathcal{V}$ and ${\mathcal{A}}^d=\{b_1,\dots ,b_m\}\subset \mathcal{A}$). Within the discretized setting, we have the guarantee that if SODA converges, the limit point has to be an equilibrium (Bichler et al. 2025, corollary 1). Additionally, SODA provides exact bounds on the quality of the approximation within the discrete setting, which we can use to evaluate the strategies employed by algorithmic pricing methods in the following section. For a detailed description of the method, see Online Appendix D.1. Using this numerical method for equilibrium computation, we can analyze the revenue in equilibrium in more complex settings with various prior distributions.¹²

3.4.1. Setting.

In the following experiments, we discretize the type and action space $\mathcal{V}=\mathcal{A}=[0,1]$ using $n=m=64$ equidistant points. We initialize the algorithm with random initial strategies and update the strategies in each iteration using dual averaging with the entropic regularization term and a decreasing step size ${\eta }_t=10·t^{-0.05}$. We stop the learning algorithm after 5,000 iterations or whenever the discrete relative utility loss (cf. Online Appendix D.2) is sufficiently small ($\mathcal{\ell }\le {10}^{-4})$. Each experiment is repeated 10 times, and the average metrics, which are computed for each experiment by sampling $2^{22}$ valuations and the corresponding bids according to the computed strategies, are reported.

3.4.2. Results.

In our initial experiments, we demonstrate that SODA successfully learns the equilibrium strategies for QL and ROI-maximizing agents with a uniform prior, matching the analytical results derived in the previous section. The precise metrics can be found in Table 1 in Online Appendix D.2. Building on this validation, we then apply SODA to more complex scenarios with a truncated Gaussian prior and asymmetric agents—settings that cannot be analyzed analytically.

An important insight from this analysis (see Figure 2) is that revenue equivalence breaks for ROI-maximizing agents with i.i.d. valuations. This observation holds for different priors and even for asymmetric settings, where one agent is ROI maximizing and the other is payoff maximizing, although the effects are less severe in that case. The number of bids in ad auctions can vary significantly, and the effects are less pronounced as the number of participating agents increases; however, understanding this benchmark difference is relevant for interpreting auction format effects in display advertising markets.

Result 1.

For ROI, the revenue in the first-price auction is lower compared with that of the second-price auction in equilibrium. The difference in revenue shrinks with increasing levels of competition with uniform and Gaussian value distributions.

The decrease in revenue in a first-price auction compared with the second-price counterpart is consistent throughout variations of the utility models. We include ROS-maximizing agents with budgets and agents with convex combinations of ROI and ROS in additional numerical analyses and make similar observations. The results are reported in Online Appendix D.3.

Although equilibrium analysis is the standard approach to analyzing auctions, real-world pricing algorithms do not necessarily follow predefined equilibrium strategies. They often lack knowledge of opponents’ value distributions and receive only bandit feedback—observing only their own auction outcomes. To study whether the revenue predictions also arise under adaptive learning rather than predefined equilibrium play, we simulate repeated interactions using multiarmed bandit algorithms, where agents learn solely through interaction, without prior knowledge of opponents or additional feedback.

Consider a symmetric bid profile that yields expected revenue below the Bayes–Nash equilibrium in the benchmark first-price auction. Such an outcome may involve bids below best-response levels for some types, creating incentives for unilateral upward deviations. This intuition suggests why stable bid-suppression outcomes are difficult to sustain in the benchmark environment that we study. The learning simulations below examine whether standard bandit algorithms nevertheless generate such outcomes or instead, move toward the computed equilibrium.

4. Simulation of Autobidding Algorithms

The key question that remains is if autobidding algorithms converge to an equilibrium in repeated play, cycle, or end up in collusive pricing strategies. From a learning agent’s point of view, the repeated interaction in an ad auction is an online optimization problem. If we consider an auction where agents have fixed valuations, the online optimization problem can be modeled as a multiarmed bandit where each arm describes one of the possible discrete bids $b\in {\mathcal{A}}^d=\{b_1,\dots ,b_m\}$. At each stage $t=1,2,\dots$, the agent pulls an arm of a slot machine and aims to maximize the cumulative reward (Lattimore and Szepesvári 2020). Well-known bandit algorithms include $\epsilon$-greedy, Thompson sampling, or exponential-weight algorithm for exploration and exploitation (Exp3), which we use in our simulations. Our use of no-regret bandit algorithms is motivated not only by their prevalence in the engineering literature but also, by empirical findings showing that bids in real-world auctions are consistent with no-regret behavior (Nekipelov et al. 2015, Noti and Syrgkanis 2021).

These algorithms extend naturally to incomplete-information settings through contextual bandits. The independent private value $v\in {\mathcal{V}}^d=\{v_1,\dots ,v_n\}$ assigned to each agent per round according to a prior distribution $F^d$ serves as context. In our experiments, we implement a straightforward version of contextual bandits: maintaining separate bandit instances for each context (valuation) (i.e., one bandit per context (Lattimore and Szepesvári 2020, section 18)). This approach aligns with the population game interpretation in Hartline et al. (2015), where each population contains agents differentiated by their valuations $v\in {\mathcal{V}}^d$. During each round, an agent is sampled from the population according to the prior distribution over valuations and interacts with agents from other populations. This framework enables the algorithms to optimize performance against the “average agent” from opposing populations. Algorithm 1 shows the complete process of this procedure.

Algorithm 1

(Bandit Algorithms in Auctions)

Input: Auction game with types ${\mathcal{V}}^d$, actions ${\mathcal{A}}^d$, and prior $F^d$

Initialize learner for each agent-value pair $(i,v)\in \mathcal{N}\times {\mathcal{V}}^d$;

for $t=1,2,\dots ,T$ do

  $\forall i\in \mathcal{N}$: agent i observes their value $v_{i,t}\in {\mathcal{V}}^d$ with $v_{i,t}\sim F^d$;

  $\forall i\in \mathcal{N}$: agent i chooses action $b_{i,t}\in {\mathcal{A}}^d$ according to the strategy of the learner $(i,v_{t,i})$;

  $\forall i\in \mathcal{N}$: agent i receives reward $r_{i,t}=u_i(b_{1,t},\dots ,b_{N,t})$;

  $\forall i\in \mathcal{N}$: agent i updates learner $(i,v_{i,t})$ using bandit feedback $(b_{i,t},r_{i,t})$;

end

We emphasize that assigning zero reward to losing bids is a modeling convention and does not approximate censored feedback in real advertising systems. It represents a stylized realized-reward benchmark in which learning is driven solely by realized allocations. A losing bidder receives no allocation and therefore, zero realized stage-game utility in the model. This should not be interpreted as saying that the bidder observes the counterfactual value or downstream outcome of the lost impression. In production bidding systems, losses generate censored feedback; the bidder may observe that it lost and possibly, some auction-level feedback, but it typically does not observe what downstream outcome would have occurred had it won the impression, such as a click, conversion, or realized value. Our simulations, therefore, study a stylized realized-reward benchmark driven by realized allocations and model-generated rewards, not an end-to-end learning problem with counterfactual value estimation.

The purpose of this simplified learning environment is to test whether standard bandit algorithms, when applied directly to the benchmark auction game, generate systematic bid suppression or instead, move toward the computed equilibrium. The results should be interpreted as evidence for this stylized stationary setting, not as a general convergence claim for richer production bidding systems with censored feedback, contextual information, value estimation pipelines, off-policy correction, or budget-pacing constraints.

4.1. Setting

In our experiments, we simulate independent agents bidding in repeated first-price and second-price auctions. Each experiment consists of $T={10}^7$ iterations and is repeated 10 times. We discretize the type and action space with ${\mathcal{V}}^d={\mathcal{A}}^d=\{0,0.05,\dots ,0.95,1\}$, which corresponds to $n=m=21$ discretization points. The reserve price is set to $r=0.05$. In this coarse discretization, bidding the reserve price for all valuations is the BNE for ROI-maximizing agents if we only consider $N=2$ agents. Therefore, we choose $N=3$. A short description of the used bandit algorithms ($\epsilon$-greedy, Exp3, and Thompson sampling) together with the chosen hyperparameters can be found in Online Appendix E.1.

Note that the purpose of the learning experiments is not to establish general convergence guarantees for all learning algorithms in auction environments. Instead, we ask a narrower and empirically motivated question: whether widely used stochastic bandit heuristics, when applied to repeated first-price and second-price auctions with bandit feedback, empirically converge to Bayes–Nash equilibrium in this benchmark setting. The learning agents do not observe opponents’ valuations or bids and receive only their own realized auction outcomes. All rewards are bounded because of the reserve price, and the environment is stationary by construction, allowing us to isolate learning behavior from nonstationary demand or supply shocks.

4.2. Results

First, we observe that the revenues generated by the agents using bandit algorithms show a similar pattern to the expected revenues in equilibrium (see the previous section). We see that revenue equivalence does not hold for ROI-maximizing agents and that the revenue for the first-price auction is significantly lower compared with that of the second-price auction. Furthermore, the average revenue matches the values predicted by the BNE strategies in the discretized setting computed using SODA.

Result 2.

If we use bandit algorithms ($\epsilon$-greedy, Exp3, or Thompson sampling), the agents learn to bid according to the BNE, and revenues for the first-price auctions are lower compared with the second-price auctions for ROI-maximizing agents.

Not only is the average revenue close to the predictions from the equilibrium analysis, but we can also observe that the agents actually learn to play according to the Bayes–Nash equilibrium using bandit algorithms (Figure 3). To this end, we look at the frequency of the bids within an interval and compare it with the distributional equilibrium strategy computed using SODA. As we can see in Figure 4, the empirical frequency of actions is close to the equilibrium strategy.

It is important to note that we get similar results if different algorithms are used. In particular, we simulate settings where agents use different bandit algorithms (i.e., $\epsilon$-greedy, Exp3, and Thompson sampling) and compete against each other. More detailed results for all simulations can be found in Online Appendix E.2. The convergence results reported here should be interpreted as empirical evidence consistent with low-regret convergence to the Bayes–Nash equilibrium of a stationary benchmark game rather than as general guarantees for all learning algorithms in nonstationary multiagent environments.

5. Conclusions

Consistent with the view of economic models as analogies rather than literal descriptions of reality (Gilboa et al. 2014), our analysis does not aim to mirror all institutional details of display advertising markets. Instead, it studies a transparent benchmark model that isolates how auction format, bidder objectives, and learning dynamics interact in repeated first-price and second-price auctions. The results should, therefore, be interpreted as theoretical benchmark evidence rather than as a complete account of learning and bidding in real-world display advertising markets.

Display ad auctions are a large and growing part of the advertising market. Most large display ad exchanges moved from second-price to first-price auctions in recent years. However, the impact of this policy change is not well understood. Some exchanges have reported revenue losses after the transition, but econometric identification is difficult because of confounding changes in demand, supply, bidder composition, impression characteristics, and market conditions. A stylized game-theoretic model allows us to isolate one set of mechanisms under ceteris paribus assumptions. In particular, we ask whether lower first-price revenues in such a benchmark should be interpreted as evidence of off-equilibrium algorithmic collusion or whether they can arise from equilibrium behavior when bidding agents optimize objectives that differ from quasilinear payoff.

Two features of display advertising motivate our analysis. First, auctions are conducted in milliseconds, and bidding is fully automated. Multiarmed bandit algorithms are natural candidates for such tasks because they are fast, adaptive, and widely studied for online decision problems. This raises the question of whether such learning algorithms converge to equilibrium strategies in repeated auction environments or whether they generate systematic off-equilibrium outcomes, such as bid suppression. Second, return on investment is widely used as an objective for demand-side platforms, which act as agents for advertisers and are typically tasked with using campaign budgets effectively. In the standard independent private values model with payoff-maximizing bidders, first-price and second-price auctions generate the same expected revenue under the usual assumptions. This revenue equivalence does not need to hold when bidders maximize expected ROI.

Our first set of results characterizes equilibrium behavior under ROI-maximizing objectives. For uniform value distributions, we derive equilibrium bidding strategies analytically. For more general distributions and asymmetric settings where the equilibrium conditions lead to nonlinear differential equations without closed-form solutions, we use recent equilibrium-computation techniques to compute and verify approximate equilibria. Across these settings, we find that ROI-maximizing bidders bid more aggressively downward in first-price auctions and that first-price auctions generate lower expected revenue than second-price auctions in equilibrium. Thus, in the benchmark model, lower first-price revenues can arise without invoking off-equilibrium collusion.

Our second set of results studies whether standard learning algorithms converge toward these benchmark equilibria. In repeated first-price and second-price auctions, agents using common bandit algorithms learn bidding behavior that aligns closely with the computed equilibrium strategies. This finding is nontrivial because learning dynamics in games can cycle, fail to converge, or generate complex off-equilibrium behavior. In our benchmark, however, we do not observe systematic bid suppression or collusive low-revenue outcomes. The simulations, therefore, suggest that in this stylized stationary environment, the revenue gap between first-price and second-price auctions is better explained by ROI-based equilibrium behavior than by algorithmic collusion.

This interpretation has important limits. Real-world display advertising markets are not fully stationary; supply, demand, bidder populations, impression characteristics, value estimates, budget pacing, and feedback structures may change over time. Our model abstracts from these features. The purpose of the benchmark is not to rule out algorithmic collusion or instability in richer marketplace-level environments. Put differently, the paper does not provide evidence that algorithmic collusion is unlikely in real display ad markets. It shows that one revenue pattern sometimes associated with collusion, lower first-price revenue, can also arise from noncollusive ROI-based equilibrium behavior in a canonical benchmark model.

There are several additional limitations. First, the bandit algorithms used in our experiments may differ from proprietary bidding algorithms used in practice. Although the literature and our own experience with demand-side platforms suggest that such algorithms are relevant, real systems may condition on richer impression features, use more complex feedback, incorporate value estimation pipelines, or adapt to changes in demand and supply. Such extensions would define a different, stateful game and could change the equilibrium and learning properties. Second, bidder objectives in display advertising are not directly observable. Although the literature suggests that ROI and related performance metrics are central in campaign management, actual platforms may optimize combinations of ROI, budget delivery, pacing, conversion targets, and other objectives.

A further limitation is that our model abstracts from the upstream value estimation layer of production bidding systems. We treat valuations as primitives of the auction game, whereas real DSPs must estimate impression-level values from censored and policy-dependent data. Correction methods, such as inverse propensity weighting or doubly robust estimation, are central to the upstream counterfactual estimation problem. They are not part of our formal analysis because we do not estimate values from observational auction logs; instead, we study equilibrium and strategic learning conditional on values or value estimates. Incorporating such value estimation pipelines would define a richer end-to-end learning model and may affect the resulting market dynamics.

Another promising direction is to study how artificial intelligence (AI) search and conversational interfaces change placed ad auction markets. AI-mediated search may alter user intent signals, ad relevance, targeting, and the effective set of competing advertisers, thereby changing both ROI-based bidding incentives and the potential for learning-driven coordination. Extending equilibrium-computation and learning-based analyses to such AI-mediated ad environments is an important topic for future research.

Subject to these qualifications, the paper contributes to the analysis of display advertising auctions in two ways. It identifies a theoretical mechanism through which the move from second-price to first-price auctions can reduce revenue even when agents play equilibrium strategies. It also provides benchmark evidence that in the stationary repeated auction environment studied here, widely used learning algorithms do not generate collusive bid suppression. These results do not settle the broader empirical question of how bidding algorithms behave in real markets, but they clarify an important alternative explanation for lower first-price revenues and provide a foundation for richer models of automated bidding, budget pacing, and marketplace-level learning.