Nonstationary Experimental Design Under Structured Trends

Experimentation is increasingly used across domains such as healthcare and online platforms to inform decision making, with a central goal often being the estimation of the average treatment effect (ATE). However, in many real-world settings, treatment effects, or even the treatments themselves, evolve over time, making classical experimental designs, which assume stationarity, less effective or even misleading. This paper studies nonstationary experimental design under structured trends, addressing two key objectives: (i) accurately estimating the dynamic treatment effect and (ii) minimizing regret (e.g., revenue or welfare loss) during the experiment. We propose a flexible design framework that can be tailored to achieve Pareto-optimal tradeoff between the two objectives. We further analyze how time-varying noise levels affect this tradeoff. Additionally, we establish asymptotic normality of the estimators and show that different trend orders yield different convergence rates, enabling efficient statistical tests for the existence of high-order trends.

This paper was accepted by J. George Shanthikumar, data science.

Funding: D. Simchi-Levi and C. Wang were partially supported by the Massachusetts Institute of Technology (MIT) Data Science Lab and the MIT-IBM partnership in artificial intelligence. Z. Zheng was partially supported by the National Science Foundation Division of Mathematical Sciences [AwardDMS-2220537] and the Avenir Public Welfare Foundation.

Supplemental Material: The online appendices and data files are available at https://doi.org/10.1287/mnsc.2023.03329.