Compute the ATE standard error from its influence function (#1006)

[jeongyoonlee]

Proposed changes

CausalTreeRegressor.estimate_ate reported a confidence interval far too narrow to be useful. Closes #1006.

The standard error was dhat.std() / X.shape[0], which is wrong twice: the standard error of a mean divides by sqrt(n), not n; and dhat.std() measures how the predicted effect varies across units — real heterogeneity — rather than how far the sample mean of that effect would land from the truth on another draw. The second point matters more: no outcome noise enters it at all, so the interval stays too narrow after fixing the first.

Point estimates do not change. Only the interval moves, and it gets wider.

The replacement

The standard error now comes from the estimate's influence function, which adds what the model got wrong on the observed outcomes to the spread of the predicted effects:

psi_i = mean_g (yhat_g,i - yhat_0,i)
        + (y_i - yhat_g(i),i) / (share_g(i) * n_treatments)   if i is treated
        - (y_i - yhat_0,i) / share_0                          if i is control

se = std(psi) / sqrt(n)

With one treatment group this is the AIPW influence function, and its variance is the three-term expression BaseTLearner.estimate_ate already computes. The two agree to within a percent across seeds — the difference is cross terms that vanish when residuals are mean-zero — so this brings the causal tree onto the convention the meta-learners already follow rather than inventing one.

A control unit enters every treatment group's contrast, so its residual is not divided by the number of them, while a treated unit's residual affects only its own.

Measured

20 randomized-trial draws, synthetic_data(mode=2, n=2000) split in half, fitted on one half and estimated on the other (n=1000), nominal 95%. Coverage counts the seeds whose interval contains the true ATE, so 19 of 20 is the target:

standard error	coverage	mean CI width
std(dhat) / n (before)	0/20	0.0033
std(dhat) / sqrt(n)	7/20	0.1026
influence function (this PR)	17/20	0.3923

Over 100 draws the shipped path covers 88%. The gap to 95% is that estimate_ate fits and estimates on the same rows, which makes the residuals optimistic — that is #517, and this PR does not change it.

Tests

Four, in tests/test_causal_trees.py:

• the standard error matches the meta-learners' three-term formula on a binary design (rel=0.05);
• the interval narrows by sqrt(4) = 2, not 4, when the sample quadruples — the scaling error is invisible at a single sample size;
• coverage over 20 seeds is at least 14, against 0 before;
• with two treatment groups, the reported error tracks the spread of the estimate over 40 draws.

The last one exists because of a mutation that survived everything else: dividing the control residual by n_treatments is plausible — each group's contrast is averaged, after all — and is a no-op when there is one treatment group. It takes that ratio from 0.89 to 0.69.

Types of changes

• Bugfix (non-breaking change which fixes an issue)
• New feature (non-breaking change which adds functionality)
• Breaking change (fix or feature that would cause existing functionality to not work as expected)
• Documentation Update (if none of the other choices apply)

Checklist

• I have read the CONTRIBUTING doc
• I have signed the CLA
• Lint and unit tests pass locally with my changes
• I have added tests that prove my fix is effective or that my feature works
• I have added necessary documentation (if appropriate)
• Any dependent changes have been merged and published in downstream modules

Further comments

The interval has been this way since v0.8.0 (2020-07-17); #522 moved the expression into the current file during the scikit-learn rewrite, so git blame points there rather than at the origin.

CausalRandomForestRegressor has no estimate_ate of its own and is unaffected. causaltree.py:431 was the only occurrence of the pattern in the package.