DiD for Repeated Cross-Sections

Coverage

The simulations are based on the the make_did_SZ2020-DGP with \(1000\) observations. Learners are only set to boosting, due to time constraints (and the nonlinearity of some of the DGPs).

DoubleML Version                       0.12.dev0
Script                   did_cs_atte_coverage.py
Date                         2025-12-04 20:44:32
Total Runtime (seconds)               12870.6627
Python Version                            3.12.3

Observational Score

ⓘLearner g Learner m DGP In-sample-norm. Bias CI Length Coverage
LGBM LGBM 1 False 11.670 59.599 0.973
LGBM LGBM 2 False 13.470 70.532 0.977
LGBM LGBM 3 False 12.634 67.469 0.989
LGBM LGBM 4 False 16.709 83.443 0.982
LGBM LGBM 5 False 7.535 38.946 0.973
LGBM LGBM 6 False 7.328 37.244 0.960
LGBM LGBM 1 True 4.470 21.342 0.954
LGBM LGBM 2 True 4.861 24.388 0.965
LGBM LGBM 3 True 4.795 23.936 0.958
LGBM LGBM 4 True 5.634 28.391 0.956
LGBM LGBM 5 True 4.118 19.511 0.943
LGBM LGBM 6 True 3.734 17.859 0.957
Coverage for 95%-Confidence Interval over 1000 Repetitions
ⓘLearner g Learner m DGP In-sample-norm. Bias CI Length Coverage
LGBM LGBM 1 False 11.670 50.017 0.940
LGBM LGBM 2 False 13.470 59.192 0.929
LGBM LGBM 3 False 12.634 56.622 0.945
LGBM LGBM 4 False 16.709 70.028 0.946
LGBM LGBM 5 False 7.535 32.685 0.933
LGBM LGBM 6 False 7.328 31.256 0.922
LGBM LGBM 1 True 4.470 17.911 0.903
LGBM LGBM 2 True 4.861 20.467 0.928
LGBM LGBM 3 True 4.795 20.088 0.916
LGBM LGBM 4 True 5.634 23.827 0.913
LGBM LGBM 5 True 4.118 16.374 0.890
LGBM LGBM 6 True 3.734 14.988 0.891
Coverage for 90%-Confidence Interval over 1000 Repetitions

Experimental Score

Remark that the only two valid DGPs are DGP \(5\) and DGP \(6\). All other DGPs are invalid due to non-experimental treatment assignment.

ⓘLearner g Learner m DGP In-sample-norm. Bias CI Length Coverage
LGBM LGBM 1 False 3.905 12.418 0.796
LGBM LGBM 2 False 3.717 13.290 0.830
LGBM LGBM 3 False 2.981 12.070 0.899
LGBM LGBM 4 False 3.911 12.211 0.789
LGBM LGBM 5 False 2.905 14.243 0.949
LGBM LGBM 6 False 2.476 12.404 0.951
LGBM LGBM 1 True 3.985 12.442 0.774
LGBM LGBM 2 True 3.723 13.283 0.832
LGBM LGBM 3 True 2.994 12.082 0.896
LGBM LGBM 4 True 3.951 12.223 0.782
LGBM LGBM 5 True 2.944 14.277 0.949
LGBM LGBM 6 True 2.562 12.421 0.955
Coverage for 95%-Confidence Interval over 1000 Repetitions
ⓘLearner g Learner m DGP In-sample-norm. Bias CI Length Coverage
LGBM LGBM 1 False 3.905 10.421 0.715
LGBM LGBM 2 False 3.717 11.153 0.746
LGBM LGBM 3 False 2.981 10.129 0.826
LGBM LGBM 4 False 3.911 10.248 0.709
LGBM LGBM 5 False 2.905 11.953 0.897
LGBM LGBM 6 False 2.476 10.410 0.900
LGBM LGBM 1 True 3.985 10.442 0.695
LGBM LGBM 2 True 3.723 11.147 0.769
LGBM LGBM 3 True 2.994 10.140 0.822
LGBM LGBM 4 True 3.951 10.258 0.707
LGBM LGBM 5 True 2.944 11.982 0.894
LGBM LGBM 6 True 2.562 10.424 0.894
Coverage for 90%-Confidence Interval over 1000 Repetitions