DiD for Repeated Cross-Sections
ATTE Coverage
The simulations are based on the the make_did_SZ2020-DGP with \(1000\) observations. Learners are only set to boosting, due to time constraints (and the nonlinearity of some of the DGPs).
Metadata
DoubleML Version 0.9.0
Script did_cs_atte_coverage.py
Date 2024-09-09 12:16:59
Total Runtime (seconds) 17588.891701
Python Version 3.12.5Observational Score
| Learner g | Learner m | DGP | In-sample-norm. | Bias | CI Length | Coverage |
|---|---|---|---|---|---|---|
| LGBM | LGBM | 1 | False | 11.671 | 59.601 | 0.973 |
| LGBM | LGBM | 2 | False | 13.470 | 70.532 | 0.977 |
| LGBM | LGBM | 3 | False | 12.635 | 67.470 | 0.989 |
| LGBM | LGBM | 4 | False | 16.709 | 83.443 | 0.982 |
| LGBM | LGBM | 5 | False | 7.536 | 38.945 | 0.973 |
| LGBM | LGBM | 6 | False | 7.328 | 37.242 | 0.960 |
| LGBM | LGBM | 1 | True | 4.470 | 21.342 | 0.954 |
| LGBM | LGBM | 2 | True | 4.861 | 24.388 | 0.965 |
| LGBM | LGBM | 3 | True | 4.795 | 23.936 | 0.958 |
| LGBM | LGBM | 4 | True | 5.634 | 28.391 | 0.956 |
| LGBM | LGBM | 5 | True | 4.118 | 19.511 | 0.943 |
| LGBM | LGBM | 6 | True | 3.734 | 17.859 | 0.957 |
| Learner g | Learner m | DGP | In-sample-norm. | Bias | CI Length | Coverage |
|---|---|---|---|---|---|---|
| LGBM | LGBM | 1 | False | 11.671 | 50.018 | 0.940 |
| LGBM | LGBM | 2 | False | 13.470 | 59.192 | 0.929 |
| LGBM | LGBM | 3 | False | 12.635 | 56.623 | 0.945 |
| LGBM | LGBM | 4 | False | 16.709 | 70.028 | 0.945 |
| LGBM | LGBM | 5 | False | 7.536 | 32.684 | 0.932 |
| LGBM | LGBM | 6 | False | 7.328 | 31.255 | 0.922 |
| LGBM | LGBM | 1 | True | 4.470 | 17.911 | 0.903 |
| LGBM | LGBM | 2 | True | 4.861 | 20.467 | 0.928 |
| LGBM | LGBM | 3 | True | 4.795 | 20.088 | 0.916 |
| LGBM | LGBM | 4 | True | 5.634 | 23.827 | 0.913 |
| LGBM | LGBM | 5 | True | 4.118 | 16.374 | 0.890 |
| LGBM | LGBM | 6 | True | 3.734 | 14.988 | 0.891 |
Experimental Score
Remark that the only two valid DGPs are DGP \(5\) and DGP \(6\). All other DGPs are invalid due to non-experimental treatment assignment.
| Learner g | Learner m | DGP | In-sample-norm. | Bias | CI Length | Coverage |
|---|---|---|---|---|---|---|
| LGBM | LGBM | 1 | False | 3.906 | 12.418 | 0.796 |
| LGBM | LGBM | 2 | False | 3.717 | 13.290 | 0.830 |
| LGBM | LGBM | 3 | False | 2.981 | 12.070 | 0.900 |
| LGBM | LGBM | 4 | False | 3.911 | 12.212 | 0.788 |
| LGBM | LGBM | 5 | False | 2.905 | 14.243 | 0.950 |
| LGBM | LGBM | 6 | False | 2.476 | 12.404 | 0.951 |
| LGBM | LGBM | 1 | True | 3.985 | 12.442 | 0.774 |
| LGBM | LGBM | 2 | True | 3.723 | 13.283 | 0.832 |
| LGBM | LGBM | 3 | True | 2.994 | 12.082 | 0.896 |
| LGBM | LGBM | 4 | True | 3.951 | 12.223 | 0.782 |
| LGBM | LGBM | 5 | True | 2.944 | 14.277 | 0.949 |
| LGBM | LGBM | 6 | True | 2.562 | 12.421 | 0.955 |
| Learner g | Learner m | DGP | In-sample-norm. | Bias | CI Length | Coverage |
|---|---|---|---|---|---|---|
| LGBM | LGBM | 1 | False | 3.906 | 10.421 | 0.714 |
| LGBM | LGBM | 2 | False | 3.717 | 11.154 | 0.747 |
| LGBM | LGBM | 3 | False | 2.981 | 10.130 | 0.826 |
| LGBM | LGBM | 4 | False | 3.911 | 10.248 | 0.709 |
| LGBM | LGBM | 5 | False | 2.905 | 11.953 | 0.897 |
| LGBM | LGBM | 6 | False | 2.476 | 10.410 | 0.901 |
| LGBM | LGBM | 1 | True | 3.985 | 10.442 | 0.695 |
| LGBM | LGBM | 2 | True | 3.723 | 11.147 | 0.769 |
| LGBM | LGBM | 3 | True | 2.994 | 10.140 | 0.822 |
| LGBM | LGBM | 4 | True | 3.951 | 10.258 | 0.707 |
| LGBM | LGBM | 5 | True | 2.944 | 11.982 | 0.894 |
| LGBM | LGBM | 6 | True | 2.562 | 10.424 | 0.894 |