The Problem of Ignoring the Ignorability Assumption in Advertising Measurement: A Matrix Completion Solution

References

• Agarwal A, Dahleh M, Shah D, Shen D (2021) Causal matrix completion. Preprint, submitted September 30, https://arxiv.org/abs/2109.15154.Google Scholar
• Ascarza E (2018) Retention futility: Targeting high-risk customers might be ineffective. J. Marketing Res. 55(1):80–98.Crossref, Google Scholar
• Athey S, Imbens GW, Wager S (2018) Approximate residual balancing: Debiased inference of average treatment effects in high dimensions. J. Roy. Statist. Soc. Ser. B Statist. Methodology 80(4):597–623.Crossref, Google Scholar
• Athey S, Tibshirani J, Wager S (2019) Generalized random forests. Ann. Statist. 47(2):1148–1178. Crossref, Google Scholar
• Athey S, Bayati M, Doudchenko N, Imbens G, Khosravi K (2021) Matrix completion methods for causal panel data models. J. Amer. Statist. Assoc. 116(536):1716–1730.Crossref, Google Scholar
• Belloni A, Chernozhukov V, Hansen C (2014) Inference on treatment effects after selection among high-dimensional controls. Rev. Econom. Stud. 81(2):608–650.Crossref, Google Scholar
• Bennett J, Lanning S (2007) The Netflix prize. Proc. KDD Cup Workshop 2007 (Association for Computing Machinery, New York), 35.Google Scholar
• Candès EJ, Recht B (2009) Exact matrix completion via convex optimization. Foundations Comput. Math. 9(6):717–772.Crossref, Google Scholar
• Candès EJ, Tao T (2010) The power of convex relaxation: Near-optimal matrix completion. IEEE Trans. Inform. Theory 56(5):2053–2080.Crossref, Google Scholar
• Chen Y, Fan J, Ma C, Yan Y (2019) Inference and uncertainty quantification for noisy matrix completion. Proc. Natl. Acad. Sci. USA 116(46):22931–22937.Crossref, Google Scholar
• Chernozhukov V, Chetverikov D, Demirer M, Duflo E, Hansen C, Newey W, Robins J (2018) Double/debiased machine learning for treatment and structural parameters. Econometrics J. 21(1):C1–C68.Crossref, Google Scholar
• Choi H, Mela CF, Balseiro SR, Leary A (2020) Online display advertising markets: A literature review and future directions. Inform. Systems Res. 31(2):556–575.Link, Google Scholar
• Crump RK, Hotz VJ, Imbens GW, Mitnik OA (2009) Dealing with limited overlap in estimation of average treatment effects. Biometrika 96(1):187–199.Crossref, Google Scholar
• D’Amour A, Ding P, Feller A, Lei L, Sekhon J (2021) Overlap in observational studies with high-dimensional covariates. J. Econom. 221(2):644–654.Crossref, Google Scholar
• Goli A, Lambrecht A, Yoganarasimhan H (2022) A bias correction approach for interference in ranking experiments. Preprint, submitted January 29, https://dx.doi.org/10.2139/ssrn.4021266.Google Scholar
• Goli A, Reiley DG, Zhang H (2025) Personalizing ad load to optimize subscription and ad revenues: Product strategies constructed from experiments on pandora. Marketing Sci. 44(2):327–352.Google Scholar
• Gordon BR, Moakler R, Zettelmeyer F (2022) Close enough? A large-scale exploration of non-experimental approaches to advertising measurement. Preprint, submitted October 4, https://arxiv.org/abs/2201.07055.Google Scholar
• Gordon BR, Moakler R, Zettelmeyer F (2023) Predictive incrementality by experimentation (PIE) for ad measurement. Preprint, submitted April 13, https://arxiv.org/abs/2304.06828v1.Google Scholar
• Gordon BR, Zettelmeyer F, Bhargava N, Chapsky D (2019) A comparison of approaches to advertising measurement: Evidence from big field experiments at Facebook. Marketing Sci. 38(2):193–225.Link, Google Scholar
• Gross D (2011) Recovering low-rank matrices from few coefficients in any basis. IEEE Trans. Inform. Theory 57(3):1548–1566.Crossref, Google Scholar
• Gui G, Nair H, Niu F (2021) Auction throttling and causal inference of online advertising effects. Preprint, submitted December 30, https://arxiv.org/abs/2112.15155v1.Google Scholar
• Gui Y, Barber R, Ma C (2023) Conformalized matrix completion. Adv. Neural Inform. Processing Systems 36:4820–4844.Crossref, Google Scholar
• Imbens GW, Rubin DB (2015) Causal Inference in Statistics, Social, and Biomedical Sciences (Cambridge University Press, Cambridge, UK).Crossref, Google Scholar
• Jesson A, Mindermann S, Shalit U, Gal Y (2020) Identifying causal-effect inference failure with uncertainty-aware models. Adv. Neural Inform. Processing Systems 33:11637–11649.Google Scholar
• Johnson GA, Lewis RA, Nubbemeyer EI (2017a) Ghost ads: Improving the economics of measuring online ad effectiveness. J. Marketing Res. 54(6):867–884.Crossref, Google Scholar
• Johnson GA, Lewis RA, Reiley DH (2017b) When less is more: Data and power in advertising experiments. Marketing Sci. 36(1):43–53.Link, Google Scholar
• Juan Y, Zhuang Y, Chin W-S, Lin C-J (2016) Field-aware factorization machines for ctr prediction. Proc. 10th ACM Conf. Recommender Systems (Association for Computing Machinery, New York), 43–50.Google Scholar
• Kallus N, Mao X, Udell M (2018) Causal inference with noisy and missing covariates via matrix factorization. Adv. Neural Inform. Processing Systems 31:6921–6932.Google Scholar
• Kim S, Dornfeld M, Zhang T (2024) Introducing a global retrieval ranking model in the ads funnel. Accessed June 12, 2025, https://www.reddit.com/r/RedditEng/comments/1d2wfsd/introducing_a_global_retrieval_ranking_model_in/.Google Scholar
• Koren Y, Rendle S, Bell R (2021) Advances in collaborative filtering. Ricci F, Rokach L, Shapira B, eds. Recommender Systems Handbook, 3rd ed. (Springer, Cham, Switzerland), 91–142.Google Scholar
• Lewis RA, Rao JM (2015) The unfavorable economics of measuring the returns to advertising. Quart. J. Econom. 130(4):1941–1973.Crossref, Google Scholar
• Lewis RA, Rao JM, Reiley DH (2011) Here, there, and everywhere: Correlated online behaviors can lead to overestimates of the effects of advertising. Proc. 20th Internat. Conf. World Wide Web (Association for Computing Machinery, New York), 157–166.Google Scholar
• Lodish LM, Abraham M, Kalmenson S, Livelsberger J, Lubetkin B, Richardson B, Stevens ME (1995) How TV advertising works: A meta-analysis of 389 real world split cable tv advertising experiments. J. Marketing Res. 32(2):125–139.Crossref, Google Scholar
• Ma W, Chen GH (2019) Missing not at random in matrix completion: The effectiveness of estimating missingness probabilities under a low nuclear norm assumption. Adv. Neural Inform. Processing Systems 32:14871–14880.Google Scholar
• Ma X, Wang J (2020) Robust inference using inverse probability weighting. J. Amer. Statist. Assoc. 115(532):1851–1860.Crossref, Google Scholar
• Mazumder R, Hastie T, Tibshirani R (2010) Spectral regularization algorithms for learning large incomplete matrices. J. Machine Learn. Res. 11:2287–2322.Google Scholar
• Menon AK, Chitrapura K-P, Garg S, Agarwal D, Kota N (2011) Response prediction using collaborative filtering with hierarchies and side-information. Proc. 17th ACM SIGKDD Internat. Conf. Knowledge Discovery Data Mining (Association for Computing Machinery, New York), 141–149.Google Scholar
• Myerson RB (1981) Optimal auction design. Math. Oper. Res. 6(1):58–73.Link, Google Scholar
• Negahban S, Wainwright MJ (2011) Estimation of (near) low-rank matrices with noise and high-dimensional scaling. Ann. Statist. 39(2):1069–1097.Crossref, Google Scholar
• Nethery RC, Mealli F, Dominici F (2019) Estimating population average causal effects in the presence of non-overlap: The effect of natural gas compressor station exposure on cancer mortality. Ann. Appl. Statist. 13(2):1242.Crossref, Google Scholar
• Neyman J (1923) Sur les applications de la théorie des probabilités aux experiences agricoles: Essai des principes. Roczniki Nauk Rolniczych 10:1–51.Google Scholar
• Nie X, Wager S (2021) Quasi-oracle estimation of heterogeneous treatment effects. Biometrika 108(2):299–319.Crossref, Google Scholar
• Ostrovsky M, Schwarz M (2023) Reserve prices in internet advertising auctions: A field experiment. J. Polit. Econom. 131(12):3352–3376.Crossref, Google Scholar
• Rafieian O (2023) Optimizing user engagement through adaptive ad sequencing. Marketing Sci. 42(5):910–933.Link, Google Scholar
• Rafieian O, Yoganarasimhan H (2021) Targeting and privacy in mobile advertising. Marketing Sci. 40(2):193–218.Link, Google Scholar
• Rafieian O, Zuo S (2024) Personalization, algorithmic dependence, and learning. Preprint, submitted December 2, http://dx.doi.org/10.2139/ssrn.5037671. Google Scholar
• Rafieian O, Kapoor A, Sharma A (2025) Multi-objective personalization of marketing interventions. Marketing Sci. 44(2):457–477.Google Scholar
• Rashid M, Rafieian O, Ghili S (2025) Auctions meet bandits: An empirical analysis. Preprint, submitted August 28, https://arxiv.org/abs/2508.21162.Google Scholar
• Recht B (2011) A simpler approach to matrix completion. J. Machine Learn. Res. 12(12):3413–3430.Google Scholar
• Rosenbaum PR, Rubin DB (1983) The central role of the propensity score in observational studies for causal effects. Biometrika 70(1):41–55.Crossref, Google Scholar
• Shalit U, Johansson FD, Sontag D (2017) Estimating individual treatment effect: Generalization bounds and algorithms. Precup D, Teh YW, eds. Proc. 34th Internat. Conf. Machine Learn., vol. 70, (PMLR, New York), 3076–3085. Google Scholar
• Shapiro BT, Hitsch GJ, Tuchman AE (2021) TV advertising effectiveness and profitability: Generalizable results from 288 brands. Econometrica 89(4):1855–1879.Crossref, Google Scholar
• Shi A, Chan T, Zhang D (2022) Leveraging algorithmic scores for causal inference in platform operations. Preprint, submitted December 19, http://dx.doi.org/10.2139/ssrn.4262156.Google Scholar
• Simester D, Timoshenko A, Zoumpoulis SI (2020) Efficiently evaluating targeting policies: Improving on champion vs. challenger experiments. Management Sci. 66(8):3412–3424.Link, Google Scholar
• Swaminathan A, Joachims T (2015) Counterfactual risk minimization: Learning from logged bandit feedback. Bach F, Blei D, eds. Proc. 32nd Internat. Conf. Machine Learn., vol. 37 (PMLR, New York), 814–823.Google Scholar
• Waisman C, Nair HS, Carrion C (2025) Online causal inference for advertising in real-time bidding auctions. Marketing Sci. 44(1):176–195.Link, Google Scholar
• Yoganarasimhan H, Barzegary E, Pani A (2023) Design and evaluation of optimal free trials. Management Sci. 69(6):3220–3240.Link, Google Scholar
• Zantedeschi D, Feit EM, Bradlow ET (2017) Measuring multichannel advertising response. Management Sci. 63(8):2706–2728.Link, Google Scholar
• Zhao Y, Udell M (2020) Matrix completion with quantified uncertainty through low rank Gaussian copula. Adv. Neural Inform. Processing Systems 33:20977–20988.Google Scholar