# Logo Experiment Replication Package

This directory contains the analysis code and data needed to replicate the logo
experiment results. All scripts read from pre-processed datasets and produce
figures, tables, and statistical output.

## Setup

All scripts use `here()` for path management, anchored to the project root
(where the `.Rproj` file lives). R package dependencies are managed via
[rv](https://github.com/A2-ai/rv). Before running any scripts, run from the
project root:

```bash
./setup.sh
```

Input datasets are hashed for anonymization using `shared/hash_userId.R` at the
project root level. This utility produces both the hashed CSV and a reversible
crosswalk mapping.

## Paper Figures

| Figure | Script | Output | Verified |
|--------|--------|--------|----------|
| Fig 3B: Iteration plot (Bradley-Terry scores by DALL-E version + ATE) | `logo_exp_replication/code/analysis/figure1_iterPlots_logoExp_PROD.R` | `logo_exp_replication/outputs/figures/fig1_iterPlot_logoExp_stratified.pdf` | ✅ |
| Fig 3C: Prompt length by iteration (colored by similarity) | `logo_exp_replication/code/analysis/prompting_task2.R` | `logo_exp_replication/outputs/figures/prompt_length_similarity_by_model_task2.pdf` | ✅ |
| Fig 4A: Replay decomposition (Model Effect + Prompting Effect) | `logo_exp_replication/code/analysis/model_replay_task2.R` | `logo_exp_replication/outputs/figures/replay_decomposition_base_model_2_task2.pdf` | ✅ |

## Paper Tables

| Table | Script | Output | Verified |
|-------|--------|--------|----------|
| Distributional Effects of Model Upgrades in the Logo Task | `logo_exp_replication/code/analysis/model_replay_task2.R` | `logo_exp_replication/outputs/tables/qte_table_task2.tex` | ✅ |
| Covariate Balance Across All Three Arms (Logo Experiment) | `logo_exp_replication/code/analysis/prod_balance_tbl_logo_exp.R` | `logo_exp_replication/outputs/tables/balance_table_logo_exp_*.csv` | ✅ |

## Additional Analyses

| Script | Description | Output |
|--------|-------------|--------|
| `logo_exp_replication/code/analysis/prompting_task2.R` | POS tagging analysis (udpipe section) | console |

## Input Data

- `logo_exp_replication/inputs/main_df_logoExp_promptLvl_2026-03-17_PROD_HASHED.csv` — analysis-ready
  prompt-level dataset (18,425 rows, 281 columns). User IDs are hashed for anonymization.
- `logo_exp_replication/inputs/main_df_logoExp_promptLvl_NONPROD_1924sample_HASHED.csv` — smaller
  pre-filtered sample (1,924 users). Used by `prod_balance_tbl_logo_exp.R` to produce the
  covariate balance table shown in the paper.
- `logo_exp_replication/inputs/logo_experiment_mislabel_excluded_image_rankings_20_perc.csv` —
  image-level Bradley-Terry rankings from replay replications (image_id, wins, losses, strength, rank, logo_task).

## Running

From the project root in R or RStudio:

```r
# R scripts
source(here::here("logo_exp_replication", "code", "analysis", "prompting_task2.R"))

# Rmd files
rmarkdown::render(here::here("logo_exp_replication", "code", "analysis", "figure1_iterPlots_logoExp_PROD.R"))
```

Output directories are created automatically where needed.

## Directory Structure

```
logo_exp_replication/
├── LOGO_EXP_README.md
├── inputs/
│   ├── main_df_logoExp_promptLvl_2026-03-17_PROD_HASHED.csv
│   ├── main_df_logoExp_promptLvl_NONPROD_1924sample_HASHED.csv
│   └── logo_experiment_mislabel_excluded_image_rankings_20_perc.csv
├── code/
│   └── analysis/         # 4 scripts (see tables above)
├── outputs/              # generated by running scripts
│   ├── data/
│   ├── figures/
│   └── tables/
└── logs/
```

