Basic PLR Models

ATE Coverage

The simulations are based on the the make_plr_CCDDHNR2018-DGP with $500$ observations.

Metadata

DoubleML Version                   0.10.dev0
Script                   plr_ate_coverage.py
Date                     2025-01-08 14:10:27
Total Runtime (seconds)          6916.502653
Python Version                        3.12.8

Partialling out

Table 1: Coverage for 95.0%-Confidence Interval over 1000 Repetitions

Learner l	Learner m	Bias	CI Length	Coverage
Lasso	Lasso	0.035	0.175	0.956
Lasso	Random Forest	0.042	0.171	0.888
Random Forest	Lasso	0.036	0.181	0.952
Random Forest	Random Forest	0.037	0.174	0.946

Table 2: Coverage for 90.0%-Confidence Interval over 1000 Repetitions

Learner l	Learner m	Bias	CI Length	Coverage
Lasso	Lasso	0.035	0.146	0.908
Lasso	Random Forest	0.042	0.143	0.816
Random Forest	Lasso	0.036	0.152	0.901
Random Forest	Random Forest	0.037	0.146	0.878

IV-type

For the IV-type score, the learners ml_l and ml_g are both set to the same type of learner (here Learner g).

Table 3: Coverage for 95.0%-Confidence Interval over 1000 Repetitions

Learner g	Learner m	Bias	CI Length	Coverage
Lasso	Lasso	0.035	0.166	0.945
Lasso	Random Forest	0.036	0.175	0.953
Random Forest	Lasso	0.036	0.169	0.951
Random Forest	Random Forest	0.037	0.178	0.948

Table 4: Coverage for 90.0%-Confidence Interval over 1000 Repetitions

Learner g	Learner m	Bias	CI Length	Coverage
Lasso	Lasso	0.035	0.139	0.881
Lasso	Random Forest	0.036	0.147	0.895
Random Forest	Lasso	0.036	0.142	0.883
Random Forest	Random Forest	0.037	0.149	0.893

ATE Sensitivity

The simulations are based on the the make_confounded_plr_data-DGP with $1000$ observations as highlighted in the Example Gallery. As the DGP is nonlinear, we will only use corresponding learners. Since the DGP includes unobserved confounders, we would expect a bias in the ATE estimates, leading to low coverage of the true parameter.

Both sensitivity parameters are set to $c{f}_y=c{f}_d=0.1$, such that the robustness value $RV$ should be approximately $10\mathrm{\%}$. Further, the corresponding confidence intervals are one-sided (since the direction of the bias is unkown), such that only one side should approximate the corresponding coverage level (here only the upper coverage is relevant since the bias is positive). Remark that for the coverage level the value of $\rho$ has to be correctly specified, such that the coverage level will be generally (significantly) larger than the nominal level under the conservative choice of $|\rho |=1$.

Metadata

DoubleML Version                      0.10.dev0
Script                   plr_ate_sensitivity.py
Date                        2025-01-08 16:36:12
Total Runtime (seconds)            15659.224348
Python Version                           3.12.8

Partialling out

Table 5: Coverage for 95.0%-Confidence Interval over 500 Repetitions

Learner l	Learner m	Bias	Bias (Lower)	Bias (Upper)	Coverage	Coverage (Lower)	Coverage (Upper)	RV	RVa
LGBM	LGBM	0.922	1.646	0.283	0.114	1.000	0.962	0.123	0.052
LGBM	Random Forest	0.995	1.810	0.292	0.150	1.000	0.980	0.118	0.045
Random Forest	LGBM	1.573	2.774	0.403	0.008	1.000	0.948	0.128	0.067
Random Forest	Random Forest	1.737	3.061	0.464	0.018	1.000	0.946	0.128	0.064

Table 6: Coverage for 90.0%-Confidence Interval over 500 Repetitions

Learner l	Learner m	Bias	Bias (Lower)	Bias (Upper)	Coverage	Coverage (Lower)	Coverage (Upper)	RV	RVa
LGBM	LGBM	0.922	1.646	0.283	0.052	1.000	0.878	0.123	0.067
LGBM	Random Forest	0.995	1.810	0.292	0.078	1.000	0.918	0.118	0.060
Random Forest	LGBM	1.573	2.774	0.403	0.002	1.000	0.818	0.128	0.081
Random Forest	Random Forest	1.737	3.061	0.464	0.000	1.000	0.824	0.128	0.078

IV-type

For the IV-type score, the learners ml_l and ml_g are both set to the same type of learner (here Learner g).

Table 7: Coverage for 95.0%-Confidence Interval over 500 Repetitions

Learner g	Learner m	Bias	Bias (Lower)	Bias (Upper)	Coverage	Coverage (Lower)	Coverage (Upper)	RV	RVa
LGBM	LGBM	0.643	1.345	0.271	0.650	1.000	1.000	0.088	0.014
LGBM	Random Forest	0.932	1.698	0.267	0.160	1.000	0.994	0.118	0.043
Random Forest	LGBM	0.888	2.120	0.463	0.746	1.000	1.000	0.072	0.009
Random Forest	Random Forest	1.619	2.948	0.400	0.038	1.000	0.972	0.120	0.057

Table 8: Coverage for 90.0%-Confidence Interval over 500 Repetitions

Learner g	Learner m	Bias	Bias (Lower)	Bias (Upper)	Coverage	Coverage (Lower)	Coverage (Upper)	RV	RVa
LGBM	LGBM	0.643	1.345	0.271	0.490	1.000	0.998	0.088	0.025
LGBM	Random Forest	0.932	1.698	0.267	0.072	1.000	0.934	0.118	0.059
Random Forest	LGBM	0.888	2.120	0.463	0.554	1.000	1.000	0.072	0.018
Random Forest	Random Forest	1.619	2.948	0.400	0.012	1.000	0.892	0.120	0.071