.. _algorithms: Double machine learning algorithms ---------------------------------- The DoubleML package comes with two different algorithms to obtain DML estimates. .. note:: The algorithms are argument ``dml_procedure`` is deprecated in the python package. Generally, the second version of the algorithm DML2 is recommended, to obtain more stable estimates. Algorithm DML1 ++++++++++++++ The algorithm ``dml_procedure='dml1'`` can be summarized as #. **Inputs:** Choose a model (PLR, PLIV, IRM, IIVM), provide data :math:`(W_i)_{i=1}^{N}`, a Neyman-orthogonal score function :math:`\psi(W; \theta, \eta)` and specify machine learning method(s) for the nuisance function(s) :math:`\eta`. #. **Train ML predictors on folds:** Take a :math:`K`-fold random partition :math:`(I_k)_{k=1}^{K}` of observation indices :math:`[N] = \lbrace 1, \ldots, N\rbrace` such that the size of each fold :math:`I_k` is :math:`n=N/K`. For each :math:`k \in [K] = \lbrace 1, \ldots, K\rbrace`, construct a high-quality machine learning estimator .. math:: \hat{\eta}_{0,k} = \hat{\eta}_{0,k}\big((W_i)_{i\not\in I_k}\big) of :math:`\eta_0`, where :math:`x \mapsto \hat{\eta}_{0,k}(x)` depends only on the subset of data :math:`(W_i)_{i\not\in I_k}`. #. **Estimate causal parameter:** For each :math:`k \in [K]`, construct the estimator :math:`\check{\theta}_{0,k}` as the solution to the equation .. math:: \frac{1}{n} \sum_{i \in I_k} \psi(W_i; \check{\theta}_{0,k}, \hat{\eta}_{0,k}) = 0. The estimate of the causal parameter is obtain via aggregation .. math:: \tilde{\theta}_0 = \frac{1}{K} \sum_{k=1}^{K} \check{\theta}_{0,k}. #. **Outputs:** The estimate of the causal parameter :math:`\tilde{\theta}_0` as well as the values of the evaluated score function are returned. Algorithm DML2 ++++++++++++++ The algorithm ``dml_procedure='dml2'`` can be summarized as 1. **Inputs:** Choose a model (PLR, PLIV, IRM, IIVM), provide data :math:`(W_i)_{i=1}^{N}`, a Neyman-orthogonal score function :math:`\psi(W; \theta, \eta)` and specify machine learning method(s) for the nuisance function(s) :math:`\eta`. 2. **Train ML predictors on folds:** Take a :math:`K`-fold random partition :math:`(I_k)_{k=1}^{K}` of observation indices :math:`[N] = \lbrace 1, \ldots, N\rbrace` such that the size of each fold :math:`I_k` is :math:`n=N/K`. For each :math:`k \in [K] = \lbrace 1, \ldots, K\rbrace`, construct a high-quality machine learning estimator .. math:: \hat{\eta}_{0,k} = \hat{\eta}_{0,k}\big((W_i)_{i\not\in I_k}\big) of :math:`\eta_0`, where :math:`x \mapsto \hat{\eta}_{0,k}(x)` depends only on the subset of data :math:`(W_i)_{i\not\in I_k}`. 3. **Estimate causal parameter:** Construct the estimator for the causal parameter :math:`\tilde{\theta}_0` as the solution to the equation .. math:: \frac{1}{N} \sum_{k=1}^{K} \sum_{i \in I_k} \psi(W_i; \tilde{\theta}_0, \hat{\eta}_{0,k}) = 0. 4. **Outputs:** The estimate of the causal parameter :math:`\tilde{\theta}_0` as well as the values of the evaluate score function are returned. .. raw:: html
Implementation of the double machine learning algorithms ++++++++++++++++++++++++++++++++++++++++++++++++++++++++ As an example we consider a partially linear regression model (PLR) implemented in ``DoubleMLPLR``. The default version of the :class:`DoubleML` class is based on the DML2 algorithm. .. tab-set:: .. tab-item:: Python :sync: py .. ipython:: python import doubleml as dml from doubleml.datasets import make_plr_CCDDHNR2018 from sklearn.ensemble import RandomForestRegressor from sklearn.base import clone np.random.seed(3141) learner = RandomForestRegressor(n_estimators=100, max_features=20, max_depth=5, min_samples_leaf=2) ml_l = clone(learner) ml_m = clone(learner) data = make_plr_CCDDHNR2018(alpha=0.5, return_type='DataFrame') obj_dml_data = dml.DoubleMLData(data, 'y', 'd') dml_plr_obj = dml.DoubleMLPLR(obj_dml_data, ml_l, ml_m) dml_plr_obj.fit(); .. tab-item:: R :sync: r The DML algorithm can be selected via parameter ``dml_procedure='dml1'`` vs. ``dml_procedure='dml2'``. .. jupyter-execute:: library(DoubleML) library(mlr3) library(mlr3learners) library(data.table) lgr::get_logger("mlr3")$set_threshold("warn") learner = lrn("regr.ranger", num.trees = 100, mtry = 20, min.node.size = 2, max.depth = 5) ml_l = learner$clone() ml_m = learner$clone() set.seed(3141) data = make_plr_CCDDHNR2018(alpha=0.5, return_type='data.table') obj_dml_data = DoubleMLData$new(data, y_col="y", d_cols="d") dml_plr_obj = DoubleMLPLR$new(obj_dml_data, ml_l, ml_m, dml_procedure="dml1") dml_plr_obj$fit() The ``fit()`` method of ``DoubleMLPLR`` stores the estimate :math:`\tilde{\theta}_0` in its ``coef`` attribute. .. tab-set:: .. tab-item:: Python :sync: py .. ipython:: python dml_plr_obj.coef .. tab-item:: R :sync: r .. jupyter-execute:: dml_plr_obj$coef Let :math:`k(i) = \lbrace k: i \in I_k \rbrace`. The values of the score function :math:`(\psi(W_i; \tilde{\theta}_0, \hat{\eta}_{0,k(i)}))_{i \in [N]}` are stored in the attribute ``psi``. .. tab-set:: .. tab-item:: Python :sync: py .. ipython:: python dml_plr_obj.psi[:5] .. tab-item:: R :sync: r For the DML1 algorithm, the estimates for the different folds :math:`\check{\theta}_{0,k}``, :math:`k \in [K]` are stored in attribute ``all_dml1_coef``. .. jupyter-execute:: dml_plr_obj$psi[1:5, ,1] .. jupyter-execute:: dml_plr_obj$all_dml1_coef