WooldridgeDiD(method="ols", control_group="never_treated") currently builds every treated-cohort × time indicator, absorbs unit and time fixed effects, and relies on generic rank-deficiency handling to remove the resulting collinear columns.
That does not enforce the ETWFE normalization. The dropped column for each treated cohort should be its last pre-treatment period, t = g - 1, but the QR path can remove different cells. This can suppress genuine post-treatment effects and changes the interpretation and value of the remaining coefficients.
I found this while validating the genuine Callaway-Sant'Anna mpdta data for #723.
Reproducer
import warnings
from diff_diff import WooldridgeDiD
from diff_diff.datasets import load_mpdta
mpdta = load_mpdta()
with warnings.catch_warnings(record=True) as caught:
warnings.simplefilter("always")
result = WooldridgeDiD(
method="ols",
control_group="never_treated",
).fit(
mpdta,
outcome="lemp",
unit="countyreal",
time="year",
cohort="first_treat",
)
for warning in caught:
if "Rank-deficient design matrix" in str(warning.message):
print(warning.message)
for key in [(2004, 2004), (2004, 2005), (2007, 2007)]:
print(key, result.group_time_effects.get(key))
Observed on current main plus the source-verified loader branch:
Rank-deficient design matrix: dropping 3 of 15 columns
('g2004_t2004', 'g2006_t2005', 'g2007_t2005').
(2004, 2004) None
(2004, 2005) -0.059919911882184825
(2007, 2007) -0.057141530108885445
For cohorts 2004, 2006, and 2007, the intended reference periods are 2003, 2005, and 2006. Instead, the current solve drops 2004, 2005, and 2005 respectively; the first is a post-treatment cell and the third is not cohort 2007's g-1 reference.
An explicit full-rank dummy OLS on the same observations, omitting t = g - 1 for each treated cohort before solving, gives:
g2004_t2004 -0.010503246221103879
g2004_t2005 -0.07042315810285515
g2007_t2007 -0.026054410719146937
The reference implementation makes this normalization explicit for the never-treated control group by setting .Dtreat = (time != cohort - 1) before forming the interactions:
https://github.com/grantmcdermott/etwfe/blob/2b47cfdf61b90140c8372574eece4ad6d43d37f9/R/etwfe.R#L399-L403
Would you accept a focused PR with this boundary?
Proposed scope
- For OLS with
control_group="never_treated", omit each treated cohort's g-1 interaction deterministically before rank handling.
- Keep all other identified pre- and post-treatment cells and preserve the documented
group_time_effects mapping.
- Apply the same interaction basis across within-transformed and full-dummy OLS paths, including bootstrap refits and covariate interactions.
- Add regression tests that verify:
- each cohort's intended reference cell is absent and no post-treatment cell is dropped;
- coefficients match an independently constructed full-rank dummy OLS on
mpdta;
- coefficient results are invariant to row and interaction-column ordering;
- HC1 and full-dummy coefficient paths use the same normalization;
- anticipation and unbalanced-panel reference semantics are explicit and tested.
- Add a brief changelog entry and update any docstring that currently implies all
(g, t) cells are simultaneously identified.
Out of scope
- No change to
control_group="not_yet_treated".
- No change to logit or Poisson interaction bases without separate methodology evidence.
- No change to generic
solve_ols() rank-deficiency behaviour.
- No unrelated aggregation, inference, or result-object redesign.
WooldridgeDiD(method="ols", control_group="never_treated")currently builds every treated-cohort × time indicator, absorbs unit and time fixed effects, and relies on generic rank-deficiency handling to remove the resulting collinear columns.That does not enforce the ETWFE normalization. The dropped column for each treated cohort should be its last pre-treatment period,
t = g - 1, but the QR path can remove different cells. This can suppress genuine post-treatment effects and changes the interpretation and value of the remaining coefficients.I found this while validating the genuine Callaway-Sant'Anna
mpdtadata for #723.Reproducer
Observed on current
mainplus the source-verified loader branch:For cohorts 2004, 2006, and 2007, the intended reference periods are 2003, 2005, and 2006. Instead, the current solve drops 2004, 2005, and 2005 respectively; the first is a post-treatment cell and the third is not cohort 2007's
g-1reference.An explicit full-rank dummy OLS on the same observations, omitting
t = g - 1for each treated cohort before solving, gives:The reference implementation makes this normalization explicit for the never-treated control group by setting
.Dtreat = (time != cohort - 1)before forming the interactions:https://github.com/grantmcdermott/etwfe/blob/2b47cfdf61b90140c8372574eece4ad6d43d37f9/R/etwfe.R#L399-L403
Would you accept a focused PR with this boundary?
Proposed scope
control_group="never_treated", omit each treated cohort'sg-1interaction deterministically before rank handling.group_time_effectsmapping.mpdta;(g, t)cells are simultaneously identified.Out of scope
control_group="not_yet_treated".solve_ols()rank-deficiency behaviour.