Efficient Switchback Experiments with Surrogate Variables: Estimation and Experimental Design

References

• Abadie A, Zhao J (2021) Synthetic controls for experimental design. Preprint, submitted August 4, https://arxiv.org/abs/2108.02196.Google Scholar
• Anderer A, Bastani H, Silberholz J (2022) Adaptive clinical trial designs with surrogates: When should we bother? Management Sci. 68(3):1982–2002.Link, Google Scholar
• Armitage P, Hills M (1982) The two-period crossover trial. J. Roy. Statist. Soc. Ser. D (Statistician) 31(2):119–131.Google Scholar
• Aronow PM, Samii C (2017) Estimating average causal effects under general interference, with application to a social network experiment. Ann. Appl. Statist. 11(4):1912–1947.Google Scholar
• Athey S, Eckles D, Imbens GW (2018) Exact p-values for network interference. J. Amer. Statist. Assoc. 113(521):230–240.Crossref, Google Scholar
• Athey S, Chetty R, Imbens GW, Kang H (2019) The surrogate index: Combining short-term proxies to estimate long-term treatment effects more rapidly and precisely. NBER Working Paper No. 26463, National Bureau of Economic Research, Cambridge, MA.Google Scholar
• Bakshy E, Eckles D, Bernstein MS (2014) Designing and deploying online field experiments. Chung C-W, ed. Proc. 23rd Internat. Conf. World Wide Web (Association for Computing Machinery, New York), 283–292.Google Scholar
• Basse G, Ding Y, Toulis P (2019) Minimax crossover designs. Preprint, submitted August 9, https://arxiv.org/abs/1908.03531.Google Scholar
• Begg CB, Leung DH (2000) On the use of surrogate end points in randomized trials. J. Roy. Statist. Soc. Ser. A (Statist. Soc.) 163(1):15–28.Crossref, Google Scholar
• Bojinov I, Shephard N (2019) Time series experiments and causal estimands: Exact randomization tests and trading. J. Amer. Statist. Assoc. 114(528):1665–1682.Crossref, Google Scholar
• Bojinov I, Simchi-Levi D, Zhao J (2023) Design and analysis of switchback experiments. Management Sci. 69(7):3759–3777.Link, Google Scholar
• Box M, Draper NR (1971) Factorial designs, the—X’x—criterion, and some related matters. Technometrics 13(4):731–742.Crossref, Google Scholar
• Chamandy N (2016) Experimentation in a ridesharing marketplace. Accessed September 4, 2025, https://eng.lyft.com/experimentation-in-a-ridesharing-marketplace-b39db027a66e.Google Scholar
• Farias VF, Li AA, Peng T, Zheng AT (2022) Markovian interference in experiments. Preprint, submitted June 6, https://arxiv.org/abs/2206.02371.Google Scholar
• Ferreira KJ, Lee BHA, Simchi-Levi D (2016) Analytics for an online retailer: Demand forecasting and price optimization. Manufacturing Service Oper. Management 18(1):69–88.Link, Google Scholar
• Fisher RA (1936) Design of experiments. British Medical J. 1(3923):554.Crossref, Google Scholar
• Frangakis CE, Rubin DB (2002) Principal stratification in causal inference. Biometrics 58(1):21–29.Crossref, Google Scholar
• Garg N, Nazerzadeh H (2022) Driver surge pricing. Management Sci. 68(5):3219–3235.Link, Google Scholar
• Glynn PW, Johari R, Rasouli M (2020) Adaptive experimental design with temporal interference: A maximum likelihood approach. Adv. Neural Inform. Processing Systems, vol. 33 (Curran Associates Inc., Red Hook, NY), 15054–15064.Google Scholar
• Hahn J (1998) On the role of the propensity score in efficient semiparametric estimation of average treatment effects. Econometrica 66(2):315–331.Crossref, Google Scholar
• Hedayat A, Afsarinejad K (1978) Repeated measurements designs, II. Ann. Statist. 6(3):619–628.Google Scholar
• Henmi M, Eguchi S (2004) A paradox concerning nuisance parameters and projected estimating functions. Biometrika 91(4):929–941.Crossref, Google Scholar
• Hitomi K, Nishiyama Y, Okui R (2008) A puzzling phenomenon in semiparametric estimation problems with infinite-dimensional nuisance parameters. Econom. Theory 24(6):1717–1728.Crossref, Google Scholar
• Holland PW (1986) Statistics and causal inference. J. Amer. Statist. Assoc. 81(396):945–960.Crossref, Google Scholar
• Horvitz DG, Thompson DJ (1952) A generalization of sampling without replacement from a finite universe. J. Amer. Statist. Assoc. 47(260):663–685.Crossref, Google Scholar
• Hothorn T, Kneib T, Bühlmann P (2014) Conditional transformation models. J. Roy. Statist. Soc. Ser. B Statist. Methodology 76(1):3–27.Crossref, Google Scholar
• Hu Y, Wager S (2022) Switchback experiments under geometric mixing. Preprint, submitted September 1, https://arxiv.org/abs/2209.00197.Google Scholar
• Hyndman RJ, Bashtannyk DM, Grunwald GK (1996) Estimating and visualizing conditional densities. J. Comput. Graphic Statist. 5(4):315–336.Crossref, Google Scholar
• Imbens GW, Rubin DB (2015) Causal Inference in Statistics, Social, and Biomedical Sciences (Cambridge University Press, New York).Crossref, Google Scholar
• Johari R, Li H, Liskovich I, Weintraub GY (2022) Experimental design in two-sided platforms: An analysis of bias. Management Sci. 68(10):7069–7089.Link, Google Scholar
• Kenward MG, Jones B (2014) Chapter 27: Crossover trials. Methods and Applications of Statistics in Clinical Trials: Concepts, Principles, Trials, and Design (John Wiley & Sons Inc., Hobooken, NJ).Google Scholar
• Kohavi R, Thomke S (2017) The surprising power of online experiments. Harvard Bus. Rev. 95(5):74–82.Google Scholar
• Kohavi R, Tang D, Xu Y (2020) Trustworthy Online Controlled Experiments: A Practical Guide to a/b Testing (Cambridge University Press, Cambridge, UK).Crossref, Google Scholar
• Kohavi R, Crook T, Longbotham R, Frasca B, Henne R, Ferres JL, Melamed T (2009) Online experimentation at microsoft. Data Mining Case Stud. 11(2009):39.Google Scholar
• Laird NM, Skinner J, Kenward M (1992) An analysis of two-period crossover designs with carry-over effects. Statist. Medicine 11(14–15):1967–1979.Crossref, Google Scholar
• Li X, Ding P (2017) General forms of finite population central limit theorems with applications to causal inference. J. Amer. Statist. Assoc. 112(520):1759–1769.Crossref, Google Scholar
• Li H, Zhao G, Johari R, Weintraub GY (2022) Interference, bias, and variance in two-sided marketplace experimentation: Guidance for platforms. Proc. ACM Web Conf. (Association for Computing Machinery, New York), 182–192.Google Scholar
• Ni T, Bojinov I, Zhao J (2023) Design of panel experiments with spatial and temporal interference. Preprint, submitted June 4, https://dx.doi.org/10.2139/ssrn.4466598.Google Scholar
• Oman SD, Seiden E (1988) Switch-back designs. Biometrika 75(1):81–89.Crossref, Google Scholar
• Prentice RL (1989) Surrogate endpoints in clinical trials: Definition and operational criteria. Statist. Medicine 8(4):431–440.Crossref, Google Scholar
• Puelz D, Basse G, Feller A, Toulis P (2019) A graph-theoretic approach to randomization tests of causal effects under general interference. Preprint, submitted October 24, https://arxiv.org/abs/1910.10862.Google Scholar
• Ratkowsky D, Alldredge R, Evans MA (1992) Cross-over Experiments: Design, Analysis and Application, vol. 135 (CRC Press, Boca Raton, FL).Google Scholar
• Rigby RA, Stasinopoulos DM (2005) Generalized additive models for location, scale and shape. J. Roy. Statist. Soc. Ser. C (Appl. Statist.) 54(3):507–554.Crossref, Google Scholar
• Rubin DB (1974) Estimating causal effects of treatments in randomized and nonrandomized studies. J. Ed. Psych. 66(5):688.Crossref, Google Scholar
• Rubin DB (1980) Randomization analysis of experimental data: The fisher randomization test comment. J. Amer. Statist. Assoc. 75(371):591–593.Google Scholar
• Senn S, Lambrou D (1998) Robust and realistic approaches to carry-over. Statist. Medicine 17(24):2849–2864.Crossref, Google Scholar
• Sussman DL, Airoldi EM (2017) Elements of estimation theory for causal effects in the presence of network interference. Preprint, submitted February 12, https://arxiv.org/abs/1702.03578.Google Scholar
• Thomke SH (2020) Experimentation Works: The Surprising Power of Business Experiments (Harvard Business Press, Boston).Google Scholar
• Ugander J, Karrer B, Backstrom L, Kleinberg J (2013) Graph cluster randomization: Network exposure to multiple universes. Proc. 19th ACM SIGKDD Internat. Conf. Knowledge Discovery Data Mining (Association for Computing Machinery, New York), 329–337.Google Scholar
• Wager S, Xu K (2021) Experimenting in equilibrium. Management Sci. 67(11):6694–6715.Link, Google Scholar
• Xiong R, Chin A, Taylor S (2023) Bias-variance tradeoffs for designing simultaneous temporal experiments. Proc. KDD Workshop Causal Discovery Prediction Decision (PMLR, New York), 115–131.Google Scholar
• Xiong R, Athey S, Bayati M, Imbens G (2019) Optimal experimental design for staggered rollouts. Preprint, submitted November 9, https://arxiv.org/abs/1911.03764.Google Scholar