DiD for Panel Data
ATTE Coverage
The simulations are based on the the make_did_SZ2020-DGP with \(1000\) observations. Learners are only set to boosting, due to time constraints (and the nonlinearity of some of the DGPs).
Metadata
DoubleML Version 0.9.0
Script did_pa_atte_coverage.py
Date 2024-09-09 11:44:15
Total Runtime (seconds) 15625.949723
Python Version 3.12.5Observational Score
| Learner g | Learner m | DGP | In-sample-norm. | Bias | CI Length | Coverage |
|---|---|---|---|---|---|---|
| LGBM | LGBM | 1 | False | 3.379 | 15.002 | 0.953 |
| LGBM | LGBM | 2 | False | 3.622 | 17.536 | 0.966 |
| LGBM | LGBM | 3 | False | 3.413 | 17.144 | 0.977 |
| LGBM | LGBM | 4 | False | 5.765 | 21.602 | 0.932 |
| LGBM | LGBM | 5 | False | 1.901 | 9.180 | 0.960 |
| LGBM | LGBM | 6 | False | 1.799 | 9.020 | 0.972 |
| LGBM | LGBM | 1 | True | 1.030 | 4.966 | 0.967 |
| LGBM | LGBM | 2 | True | 1.224 | 5.982 | 0.963 |
| LGBM | LGBM | 3 | True | 1.133 | 5.844 | 0.969 |
| LGBM | LGBM | 4 | True | 1.370 | 7.100 | 0.971 |
| LGBM | LGBM | 5 | True | 0.828 | 4.110 | 0.965 |
| LGBM | LGBM | 6 | True | 0.800 | 4.053 | 0.970 |
| Learner g | Learner m | DGP | In-sample-norm. | Bias | CI Length | Coverage |
|---|---|---|---|---|---|---|
| LGBM | LGBM | 1 | False | 3.379 | 12.590 | 0.893 |
| LGBM | LGBM | 2 | False | 3.622 | 14.717 | 0.914 |
| LGBM | LGBM | 3 | False | 3.413 | 14.388 | 0.933 |
| LGBM | LGBM | 4 | False | 5.765 | 18.129 | 0.842 |
| LGBM | LGBM | 5 | False | 1.901 | 7.704 | 0.917 |
| LGBM | LGBM | 6 | False | 1.799 | 7.570 | 0.922 |
| LGBM | LGBM | 1 | True | 1.030 | 4.168 | 0.906 |
| LGBM | LGBM | 2 | True | 1.224 | 5.020 | 0.917 |
| LGBM | LGBM | 3 | True | 1.133 | 4.905 | 0.921 |
| LGBM | LGBM | 4 | True | 1.370 | 5.959 | 0.925 |
| LGBM | LGBM | 5 | True | 0.828 | 3.449 | 0.920 |
| LGBM | LGBM | 6 | True | 0.800 | 3.402 | 0.925 |
Experimental Score
Remark that the only two valid DGPs are DGP \(5\) and DGP \(6\). All other DGPs are invalid due to non-experimental treatment assignment.
| Learner g | Learner m | DGP | In-sample-norm. | Bias | CI Length | Coverage |
|---|---|---|---|---|---|---|
| LGBM | LGBM | 1 | False | 2.170 | 2.577 | 0.094 |
| LGBM | LGBM | 2 | False | 1.321 | 2.537 | 0.475 |
| LGBM | LGBM | 3 | False | 1.001 | 2.240 | 0.584 |
| LGBM | LGBM | 4 | False | 1.917 | 2.244 | 0.122 |
| LGBM | LGBM | 5 | False | 0.521 | 2.460 | 0.951 |
| LGBM | LGBM | 6 | False | 0.434 | 2.155 | 0.948 |
| LGBM | LGBM | 1 | True | 2.171 | 2.573 | 0.099 |
| LGBM | LGBM | 2 | True | 1.320 | 2.538 | 0.470 |
| LGBM | LGBM | 3 | True | 1.000 | 2.239 | 0.598 |
| LGBM | LGBM | 4 | True | 1.924 | 2.247 | 0.127 |
| LGBM | LGBM | 5 | True | 0.520 | 2.459 | 0.947 |
| LGBM | LGBM | 6 | True | 0.437 | 2.156 | 0.944 |
| Learner g | Learner m | DGP | In-sample-norm. | Bias | CI Length | Coverage |
|---|---|---|---|---|---|---|
| LGBM | LGBM | 1 | False | 2.170 | 2.162 | 0.051 |
| LGBM | LGBM | 2 | False | 1.321 | 2.130 | 0.377 |
| LGBM | LGBM | 3 | False | 1.001 | 1.880 | 0.470 |
| LGBM | LGBM | 4 | False | 1.917 | 1.883 | 0.078 |
| LGBM | LGBM | 5 | False | 0.521 | 2.064 | 0.889 |
| LGBM | LGBM | 6 | False | 0.434 | 1.809 | 0.908 |
| LGBM | LGBM | 1 | True | 2.171 | 2.160 | 0.049 |
| LGBM | LGBM | 2 | True | 1.320 | 2.130 | 0.373 |
| LGBM | LGBM | 3 | True | 1.000 | 1.879 | 0.477 |
| LGBM | LGBM | 4 | True | 1.924 | 1.886 | 0.084 |
| LGBM | LGBM | 5 | True | 0.520 | 2.064 | 0.891 |
| LGBM | LGBM | 6 | True | 0.437 | 1.809 | 0.897 |