Frontiers: Reducing Disparity Between LLMs and Humans: Optimal LLM Sample Calibration

References

• Argyle LP, Busby EC, Fulda N, Gubler JR, Rytting C, Wingate D (2023) Out of one, many: Using language models to simulate human samples. Political Anal. 31(3):337–351.Crossref, Google Scholar
• Bail CA (2024) Can generative AI improve social science? Proc. Natl. Acad. Sci. USA 121(21):e2314021121.Crossref, Google Scholar
• Brand J, Israeli A, Ngwe D (2023) Using LLMs for market research. Preprint, submitted March 21, https://doi.org/10.2139/ssrn.4395751.Google Scholar
• Dietterich TG (2000) Ensemble methods in machine learning. Kittler J, Roli F, eds. Multiple Classifier Systems, Lecture Notes in Computer Science, vol. 1857 (Springer, Berlin), 1–15.Crossref, Google Scholar
• Dillion D, Tandon N, Gu Y, Gray K (2023) Can AI language models replace human participants? Trends Cognitive Sci. 27(7):597–600.Crossref, Google Scholar
• Gao Y, Lee D, Burtch G, Fazelpour S (2025) Take caution in using LLMs as human surrogates. Proc. Natl. Acad. Sci. USA 122(24):e2501660122.Crossref, Google Scholar
• Goli A, Singh A (2024) Frontiers: Can large language models capture human preferences? Marketing Sci. 43(4):709–722.Link, Google Scholar
• Groves RM, Fowler FJ Jr, Couper MP, Lepkowski JM, Singer E, Tourangeau R (2009) Survey Methodology, 2nd ed. (John Wiley & Sons, Hoboken, NJ).Google Scholar
• Gui G, Toubia O (2023) The challenge of using LLMs to simulate human behavior: A causal inference perspective. Preprint, submitted December 24, https://arxiv.org/abs/2312.15524.Google Scholar
• Horton JJ (2023) Large language models as simulated economic agents: What can we learn from homo silicus? Technical report, National Bureau of Economic Research, Cambridge, MA.Google Scholar
• Leng Y, Yuan Y (2026) SUVA: A probabilistic framework for auditing LLMs with an application to social preferences. Inform. Systems Res. 37(3):1805–1830.Link, Google Scholar
• Li P, Castelo N, Katona Z, Sarvary M (2024) Frontiers: Determining the validity of large language models for automated perceptual analysis. Marketing Sci. 43(2):254–266.Link, Google Scholar
• Lohr SL (2021) Sampling: Design and Analysis, 3rd ed. (Chapman and Hall/CRC, Boca Raton, FL).Crossref, Google Scholar
• Ludwig J, Mullainathan S, Rambachan A (2026) Large language models: An applied econometric framework. Annual Rev. Econom. 18(1):283–316.Google Scholar
• Montgomery DC (2017) Design and Analysis of Experiments, 9th ed. (John Wiley & Sons, Hoboken, NJ).Google Scholar
• Murphy KP (2012) Machine Learning: A Probabilistic Perspective (MIT Press, Cambridge, MA).Google Scholar
• Netzer O, Feldman R, Goldenberg J, Fresko M (2012) Mine your own business: Market-structure surveillance through text mining. Marketing Sci. 31(3):521–543.Link, Google Scholar
• Qiu L, Singh PV, Srinivasan K (2023) Testing theoretical alignment in large language models: Evidence from insurance choices. Preprint, submitted July 31, https://doi.org/10.2139/ssrn.4526072.Google Scholar
• Qu Y, Wang J (2024) Performance and biases of large language models in public opinion simulation. Humanities Soc. Sci. Comm. 11(1):1–13.Google Scholar
• Salewski L, Alaniz S, Rio-Torto I, Schulz E, Akata Z (2023) In-context impersonation reveals large language models’ strengths and biases. Oh A, Naumann T, Globerson A, Saenko K, Hardt M, Levine S, eds. Adv. Neural Inform. Processing Systems, vol. 36 (Curran Associates, Red Hook, NY), 72044–72057.Google Scholar
• Santurkar S, Durmus E, Ladhak F, Lee C, Liang P, Hashimoto T (2023) Whose opinions do language models reflect? Krause A, Brunskill E, Cho K, Engelhardt B, Sabato S, Scarlett J, eds. Internat. Conf. Machine Learn., vol. 202 (PMLR, New York), 29971–30004.Google Scholar
• Sarstedt M, Adler SJ, Rau L, Schmitt B (2024) Using large language models to generate silicon samples in consumer and marketing research: Challenges, opportunities, and guidelines. Psych. Marketing 41(6):1254–1270.Crossref, Google Scholar
• Schoenegger P, Tuminauskaite I, Park PS, Bastos RVS, Tetlock PE (2024) Wisdom of the silicon crowd: LLM ensemble prediction capabilities rival human crowd accuracy. Sci. Adv. 10(45):eadp1528.Crossref, Google Scholar
• Shalev-Shwartz S, Ben-David S (2014) Understanding Machine Learning: From Theory to Algorithms (Cambridge University Press, Cambridge, UK).Crossref, Google Scholar
• Stromberg M, Moe WW, Reutterer T, Schweidel DA (2025) Blind spots in broad strokes: Caveats for the use of LLMs in marketing research. Preprint, submitted April 21, https://doi.org/10.2139/ssrn.5145114.Google Scholar
• Toubia O, Johnson E, Evgeniou T, Delquié P (2013) Dynamic experiments for estimating preferences: An adaptive method of eliciting time and risk parameters. Management Sci. 59(3):613–640.Link, Google Scholar