match-score-ab-test
Does showing job candidates their AI match score help them? Candidates, postings and match scores come from more than 150,000 real application records. Only the reaction to seeing a score was simulated, and how is documented in src/simulate_behavior.py.
Looking only at people who applied, the offer rate rose from 6.4% to 14.4%. That comparison is misleading: seeing the score changed who applied, and applications dropped 70%. Measured on the full population, the chance of landing any offer fell from 19.1% to 11.6%. CUPED made the selection effect visible, and a causal forest showed candidates with less traditional backgrounds were discouraged most.
Methods: candidate-level randomization with SRM checks, power analysis, CUPED, always-valid sequential testing, causal forest CATE.
mhealth_abtesting
A Streamlit dashboard built on more than 1.2 million readings from the UCI mHealth sensor dataset (10 subjects, 13 activities, 24 channels). It runs a stratified quasi-randomized A/B assignment, Welch’s t-test, Cohen’s d and 95% confidence intervals. The intervention is simulated on observational data, so the effects are illustrative, not clinical outcomes.