What this analysis does
The Random Forest analysis (RF log) found that illusion_of_control and future_ev_influence predict ΔSI more strongly in the T&E arm than in Pure Control. That is a predictive signal, not a causal one. This step asks: does a founder's score on W causally change how much they benefit from T&E?
Framework Chernozhukov, Demirer, Duflo & Fernández-Val (2025, Econometrica 93(4)) — BLP / GATES / CLAN
BLP spec Y = α₁·B(Z) + α₂·T̃ + α₃·T̃·(W−W̄) + ε (eq. 3.3 — no intercept)
GATES spec Y = α₁·B(Z) + Σₖ γₖ·T̃·1(Gₖ) + ε (eq. 3.7 — no intercept, joint regression)
CLAN spec Per-split δ_a(v) = mean(v|G_K) − mean(v|G_1); aggregate via median p-values & β=0.5 CI (eq. 3.10, Defs. 4.2–4.3)
B(Z) nuisance cv.glmnet Lasso fitted on control units of auxiliary sample each split
Propensity p̂ = constant = mean(D) — balanced RCT
Splits NS = 250 (paper recommendation); 250/250 valid in all 8 cells
Aggregation Median p-values (Def. 4.2) & β=0.5 quantile CI bounds (Def. 4.3) — identical rule for BLP, GATES, CLAN
SE estimator HC1 robust (sandwich); vectorized O(nk²)
Implementation Manual R (glmnet + lm.fit); verified line-by-line against paper spec
Label Exploratory pre-registration pending
Why we focus on periods 1–2
| Period | N | ATE (median) | 95% CI | median-p | Interpretation |
|---|---|---|---|---|---|
| 1 | 372–480 | +0.57–0.67 | [0.26, 0.96] | <0.001 | Strong positive effect |
| 2 | 269–357 | +0.61–0.73 | [0.23, 1.12] | <0.002 | Effect persists |
| 3–5 | ~136–194 | ≈ 0 | — | — | Effect gone; no HTE to detect |
Why does the ATE fade? Attrition (N drops from ~450 to ~136 by period 5) and mean-reversion in SI as the program recedes. Investigating the fade-out mechanism is a next step.
BLP Scanner — 8 cells × 250 splits
α₂ = ATE; α₃ ≡ β₂ = HTE coefficient. All results are medians across 250 random splits (250/250 valid). β₂ > 0 → high-W founders benefit more from T&E.
| Moderator (W) | Period | N | ATE | 95% CI | median-p | β₂ | 95% CI | median-p | |
|---|---|---|---|---|---|---|---|---|---|
illusion_of_control | 1 | 372 | +0.566 | [0.259, 0.876] | 0.000 | −0.009 | [−0.344, 0.328] | 0.655 | n.s. |
illusion_of_control | 2 | 269 | +0.679 | [0.268, 1.080] | 0.002 | −0.154 | [−0.565, 0.281] | 0.447 | n.s. |
idea_aspiration_1 | 1 | 438 | +0.665 | [0.369, 0.959] | 0.000 | +0.001 | [−0.013, 0.014] | 0.577 | n.s. |
idea_aspiration_1 | 2 | 321 | +0.734 | [0.355, 1.117] | 0.000 | +0.003 | [−0.015, 0.020] | 0.604 | n.s. |
new_ideas | 1 | 480 | +0.586 | [0.297, 0.879] | 0.000 | +0.059 | [−0.703, 0.825] | 0.592 | n.s. |
new_ideas | 2 | 357 | +0.607 | [0.235, 0.975] | 0.002 | −0.432 | [−1.360, 0.507] | 0.349 | n.s. |
future_ev_influence | 1 | 438 | +0.667 | [0.376, 0.958] | 0.000 | +0.494 | [−0.062, 1.063] | 0.084 | p<0.10 |
future_ev_influence | 2 | 320 | +0.732 | [0.347, 1.115] | 0.000 | +0.178 | [−0.511, 0.890] | 0.516 | n.s. |
Key finding:
future_ev_influence × period 1 — β₂ = +0.494, 95% CI [−0.062, 1.063], median-p = 0.084 over 250 splits. Borderline (p < 0.10), not significant at 5%. The CI barely crosses zero on the left. Direction consistent: founders who believe external factors can shape their venture benefit more from T&E.
All other moderators: null. IoC, idea_aspiration_1, and new_ideas show no significant moderation in either period (all β₂ p > 0.35). ATE is robustly significant in all cells (median-p < 0.002).
GATES — Group Average Treatment Effects
K=5 groups assigned by predicted CATE (= α₂ + α₃·W_c). Joint regression per eq. 3.7; results aggregated via β=0.5 quantile CI across 250 splits. A G1 → G5 gradient corroborates the BLP signal.
GATES: W = future_ev_influence, Period 1 (β₂ p = 0.084)
Clear gradient: G1 (lowest future_ev_influence) is near-zero while G2–G5 are all above 0.6. This is consistent with β₂ > 0, though the CI on β₂ just touches zero.
GATES: W = illusion_of_control, Period 1 (β₂ p = 0.655 — flat)
GATES: W = idea_aspiration_1, Period 1 (β₂ p = 0.577 — flat)
GATES: W = new_ideas, Period 1 (β₂ p = 0.592 — null)
GATES: W = future_ev_influence, Period 2 (β₂ p = 0.516 — null)
CLAN — Who benefits most?
For each split, δ_a(v) = mean(v | G_5) − mean(v | G_1) on the main sample, t-tested. Results aggregated across 250 splits using median δ, β=0.5 CI, and median p-value (same rule as BLP/GATES). Shown for future_ev_influence × period 1.
Site confound warning. The top 4 CLAN variables for
future_ev_influence are site dummies (site_5 vs sites 3/1/4). G5 = founders at site_5 (who score high on FEI); G1 = founders at sites 1 and 3. This means the apparent β₂ may be capturing between-site treatment effect heterogeneity rather than individual-level variation in beliefs. BLP includes site FE in X controls, but if FEI has little within-site variation, the β₂ is still effectively a site × treatment interaction. A within-site analysis is needed.| Variable | Median δ (G5−G1) | 95% CI | Median p | |
|---|---|---|---|---|
site_5 | +0.465 | [+0.310, +0.620] | 0.000 | site! |
site_3 | −0.316 | [−0.461, −0.145] | 0.000 | site! |
site_1 | −0.272 | [−0.419, −0.130] | 0.001 | site! |
site_4 | +0.281 | [+0.115, +0.445] | 0.001 | site! |
prob_unexpectedevents_6M | −0.140 | [−0.240, −0.041] | 0.006 | p<0.01 |
| All other individual-level variables: median-p > 0.10 | ||||
Interpretation and next steps
Summary
- ATE robust. Significant positive treatment effect in periods 1–2 across all cells (median-p < 0.002). T&E improves scientific intensity by ~+0.6–0.7 SI units.
- No moderator significant at 5%. IoC, idea_aspiration_1, new_ideas are clearly null.
future_ev_influenceis borderline (median-p = 0.084), CI [−0.062, 1.063]. - Single-split p was inflated. Stata single-split gave p=0.018 for FEI. Proper 250-split median aggregation gives p=0.084 — the earlier result was split-dependent.
- CLAN dominated by sites. The between-site composition of G5 vs G1 for FEI raises a confound concern even though site FE are in X controls.
Limitations
- Site confound in FEI result. Residualize FEI on site dummies and re-run BLP to test the within-site signal.
- W demeaned, not residualized. Partialling X from W via Lasso (fully rigorous BLP) is the confirmatory step.
- Linear CATE proxy. GATES uses linear BLP proxy for group assignment. Causal forest would detect non-linear heterogeneity.
- Multiple comparisons. 8 BLP tests run; Bonferroni threshold = p<0.006. No result survives. p=0.084 is a candidate for pre-registration, not a confirmed finding.
Next steps
- Residualize
future_ev_influenceon site dummies; re-run BLP on the residual to isolate within-site individual variation. - If within-site signal is consistent: pre-register FEI as primary moderator.
- Causal forest CATE proxy for GATES (ranger now available).
- Investigate ATE fade-out mechanism.