Predicting Consumer In-Store Purchase Through Real-Time Video Analytics: An Advanced Computer Vision and Deep Learning Approach

This study introduces a theory-driven video analytics framework to predict purchase decisions in offline retail using consumer shopping video data. The framework creates an “offline clickstream,” enabling real-time behavioral visibility and personalized targeting in physical stores. By integrating person reidentification, trajectory reconstruction, vision-language models, and pose estimation, we extract five categories of features: spatial-temporal trajectory (e.g., speed, path complexity), product interaction (e.g., touch, pickup, visual engagement), body pose and motion (e.g., hand positioning, head orientation), facial expressions, and benchmark features (static demographic/contextual characteristics). Using deep learning models, particularly a transformer model, we show that video-derived features substantially outperform benchmarks, achieving an area under the precision–recall curve of 0.8703 and an area under the receiver operating characteristic curve of 0.9243—improvements of +79.17% and +26.68% over benchmark-only models, respectively. Interpretability analyses highlight product interaction and spatial-temporal trajectory features trajectories as the strongest predictors, providing marketers with actionable insights into moments of engagement. To demonstrate practical utility, we design five real-time targeting policies informed by purchase-intention trajectories; simulations show that policies leveraging persuadability, entropy, and trajectory patterns significantly outperform baselines, with the persuadability policy delivering a 13.1% profit lift over nontargeting. Complementary segmentation and demographic analyses reveal meaningful heterogeneity between purchasers and nonpurchasers, enriching managerial insight. Overall, our framework provides a scalable, privacy-conscious tool to transform in-store video into structured behavioral data, bridging the gap with e-commerce capabilities and enabling timely, personalized interventions that enhance conversion and customer value.

History: Sean Xu, Senior Editor; Tianshu Sun, Associate Editor.

Funding: K. Xu received financial support from the National Social Science Foundation of China [Grant 22VRC174].

Supplemental Material: The online appendices are available at https://doi.org/10.1287/isre.2023.0432.