APO Models

init_notebook_modetrusted

APO Pointwise Coverage

The simulations are based on the the make_irm_data_discrete_treatments-DGP with \(500\) observations. Due to the linearity of the DGP, Lasso and Logit Regression are nearly optimal choices for the nuisance estimation.

Metadata

DoubleML Version                       0.9.0
Script                   irm_apo_coverage.py
Date                     2024-09-09 10:19:31
Total Runtime (seconds)         10550.167823
Python Version                        3.12.5

Table 1: Coverage for 95.0%-Confidence Interval over 1000 Repetitions

Learner g	Learner m	Treatment Level	Bias	CI Length	Coverage
LGBM	LGBM	0.000	2.076	10.316	0.965
LGBM	LGBM	1.000	9.181	45.558	0.967
LGBM	LGBM	2.000	9.611	44.674	0.952
LGBM	Logistic	0.000	1.335	6.704	0.955
LGBM	Logistic	1.000	1.697	8.845	0.968
LGBM	Logistic	2.000	1.661	8.724	0.971
Linear	LGBM	0.000	1.304	6.552	0.950
Linear	LGBM	1.000	2.126	12.751	0.982
Linear	LGBM	2.000	1.634	8.953	0.965
Linear	Logistic	0.000	1.283	6.358	0.954
Linear	Logistic	1.000	1.281	6.455	0.956
Linear	Logistic	2.000	1.278	6.394	0.958

Table 2: Coverage for 90.0%-Confidence Interval over 1000 Repetitions

Learner g	Learner m	Treatment Level	Bias	CI Length	Coverage
LGBM	LGBM	0.000	2.076	8.658	0.912
LGBM	LGBM	1.000	9.181	38.233	0.915
LGBM	LGBM	2.000	9.611	37.492	0.890
LGBM	Logistic	0.000	1.335	5.626	0.904
LGBM	Logistic	1.000	1.697	7.423	0.924
LGBM	Logistic	2.000	1.661	7.321	0.917
Linear	LGBM	0.000	1.304	5.498	0.901
Linear	LGBM	1.000	2.126	10.701	0.949
Linear	LGBM	2.000	1.634	7.514	0.930
Linear	Logistic	0.000	1.283	5.336	0.902
Linear	Logistic	1.000	1.281	5.418	0.905
Linear	Logistic	2.000	1.278	5.366	0.908

APOS Coverage

The non-uniform results (coverage, ci length and bias) refer to averaged values over all quantiles (point-wise confidende intervals).

Metadata

DoubleML Version                       0.9.0
Script                   irm_apo_coverage.py
Date                     2024-09-09 10:19:31
Total Runtime (seconds)         10550.167823
Python Version                        3.12.5

Table 3: Coverage for 95.0%-Confidence Interval over 1000 Repetitions

Learner g	Learner m	Bias	CI Length	Coverage	Uniform CI Length	Uniform Coverage
LGBM	LGBM	7.121	33.615	0.959	40.868	0.974
LGBM	Logistic	1.571	8.090	0.962	9.592	0.961
Linear	LGBM	1.725	9.418	0.967	11.234	0.975
Linear	Logistic	1.279	6.402	0.956	6.818	0.953

Table 4: Coverage for 90.0%-Confidence Interval over 1000 Repetitions

Learner g	Learner m	Bias	CI Length	Coverage	Uniform CI Length	Uniform Coverage
LGBM	LGBM	7.121	28.210	0.906	36.194	0.924
LGBM	Logistic	1.571	6.789	0.918	8.394	0.925
Linear	LGBM	1.725	7.903	0.928	9.850	0.940
Linear	Logistic	1.279	5.373	0.903	5.796	0.906

Causal Contrast Coverage

The non-uniform results (coverage, ci length and bias) refer to averaged values over all quantiles (point-wise confidende intervals).

Metadata

DoubleML Version                       0.9.0
Script                   irm_apo_coverage.py
Date                     2024-09-09 10:19:31
Total Runtime (seconds)         10550.167823
Python Version                        3.12.5

Table 5: Coverage for 95.0%-Confidence Interval over 1000 Repetitions

Learner g	Learner m	Bias	CI Length	Coverage	Uniform CI Length	Uniform Coverage
LGBM	LGBM	9.789	45.131	0.949	51.424	0.965
LGBM	Logistic	1.267	6.822	0.963	7.765	0.962
Linear	LGBM	1.506	8.855	0.989	10.090	0.992
Linear	Logistic	0.295	1.361	0.936	1.550	0.920

Table 6: Coverage for 90.0%-Confidence Interval over 1000 Repetitions

Learner g	Learner m	Bias	CI Length	Coverage	Uniform CI Length	Uniform Coverage
LGBM	LGBM	9.789	37.875	0.888	44.829	0.898
LGBM	Logistic	1.267	5.725	0.927	6.774	0.927
Linear	LGBM	1.506	7.431	0.958	8.799	0.975
Linear	Logistic	0.295	1.143	0.872	1.351	0.871