{ "cells": [ { "attachments": {}, "cell_type": "markdown", "id": "b736f911", "metadata": { "papermill": { "duration": 0.040312, "end_time": "2021-03-23T13:44:30.492247", "exception": false, "start_time": "2021-03-23T13:44:30.451935", "status": "completed" }, "tags": [] }, "source": [ "# R: Impact of 401(k) on Financial Wealth" ] }, { "attachments": {}, "cell_type": "markdown", "id": "56bfb9d6", "metadata": { "papermill": { "duration": 0.036965, "end_time": "2021-03-23T13:44:30.565334", "exception": false, "start_time": "2021-03-23T13:44:30.528369", "status": "completed" }, "tags": [] }, "source": [ "In this real-data example, we illustrate how the [DoubleML](https://docs.doubleml.org/stable/index.html) package can be used to estimate the effect of 401(k) eligibility and participation on accumulated assets. The 401(k) data set has been analyzed in several studies, among others [Chernozhukov et al. (2018)](https://arxiv.org/abs/1608.00060).\n", "\n", "401(k) plans are pension accounts sponsored by employers. The key problem in determining the effect of participation in 401(k) plans on accumulated assets is saver heterogeneity coupled with the fact that the decision to enroll in a 401(k) is non-random. It is generally recognized that some people have a higher preference for saving than others. It also seems likely that those individuals with high unobserved preference for saving would be most likely to choose to participate in tax-advantaged retirement savings plans and would tend to have otherwise high amounts of accumulated assets. The presence of unobserved savings preferences with these properties then implies that conventional estimates that do not account for saver heterogeneity and endogeneity of participation will be biased upward, tending to overstate the savings effects of 401(k) participation.\n", "\n", "One can argue that eligibility for enrolling in a 401(k) plan in this data can be taken as exogenous after conditioning on a few observables of which the most important for their argument is income. The basic idea is that, at least around the time 401(k)’s initially became available, people were unlikely to be basing their employment decisions on whether an employer offered a 401(k) but would instead focus on income and other aspects of the job. " ] }, { "attachments": {}, "cell_type": "markdown", "id": "f5020e5a", "metadata": { "papermill": { "duration": 0.035502, "end_time": "2021-03-23T13:44:30.636343", "exception": false, "start_time": "2021-03-23T13:44:30.600841", "status": "completed" }, "tags": [] }, "source": [ "## Data\n", "\n", "The preprocessed data can be fetched by calling [fetch_401k()](https://docs.doubleml.org/r/stable/reference/fetch_401k.html). The arguments `polynomial_features` and `instrument` can be used to replicate the models used in [Chernozhukov et al. (2018)](https://arxiv.org/abs/1608.00060). Note that an internet connection is required for loading the data. We start with a baseline specification of the regression model and reload the data later in case we want to use another specification." ] }, { "cell_type": "code", "execution_count": 1, "id": "b5167d86", "metadata": { "execution": { "iopub.execute_input": "2021-03-23T13:44:30.715038Z", "iopub.status.busy": "2021-03-23T13:44:30.712600Z", "iopub.status.idle": "2021-03-23T13:44:31.163126Z", "shell.execute_reply": "2021-03-23T13:44:31.161559Z" }, "papermill": { "duration": 0.491422, "end_time": "2021-03-23T13:44:31.163324", "exception": false, "start_time": "2021-03-23T13:44:30.671902", "status": "completed" }, "tags": [], "vscode": { "languageId": "r" } }, "outputs": [ { "name": "stderr", "output_type": "stream", "text": [ "Warning message:\n", "\"Paket 'mlr3' wurde unter R Version 4.2.3 erstellt\"\n" ] }, { "data": { "text/html": [ "\n", "
  1. 9915
  2. 12
\n" ], "text/latex": [ "\\begin{enumerate*}\n", "\\item 9915\n", "\\item 12\n", "\\end{enumerate*}\n" ], "text/markdown": [ "1. 9915\n", "2. 12\n", "\n", "\n" ], "text/plain": [ "[1] 9915 12" ] }, "metadata": {}, "output_type": "display_data" }, { "name": "stdout", "output_type": "stream", "text": [ "Classes 'data.table' and 'data.frame':\t9915 obs. of 12 variables:\n", " $ net_tfa: num 0 1015 -2000 15000 0 ...\n", " $ age : num 47 36 37 58 32 34 28 54 43 32 ...\n", " $ inc : num 6765 28452 3300 52590 21804 ...\n", " $ educ : num 8 16 12 16 11 16 12 11 14 18 ...\n", " $ fsize : num 2 1 6 2 1 1 3 3 2 3 ...\n", " $ marr : num 0 0 0 1 0 0 1 0 1 1 ...\n", " $ twoearn: num 0 0 0 1 0 0 1 0 1 0 ...\n", " $ db : num 0 0 1 0 0 0 0 0 1 0 ...\n", " $ pira : num 0 0 0 0 0 1 1 0 1 0 ...\n", " $ hown : num 1 1 0 1 1 1 1 1 1 1 ...\n", " $ p401 : int 0 0 0 0 0 0 0 0 0 0 ...\n", " $ e401 : int 0 0 0 0 0 0 0 0 0 0 ...\n", " - attr(*, \".internal.selfref\")= \n" ] } ], "source": [ "# Load required packages for this tutorial\n", "library(DoubleML)\n", "library(mlr3)\n", "library(mlr3learners)\n", "library(data.table)\n", "library(ggplot2)\n", "\n", "# suppress messages during fitting\n", "lgr::get_logger(\"mlr3\")$set_threshold(\"warn\") \n", "\n", "# load data as a data.table\n", "data = fetch_401k(return_type = \"data.table\", instrument = TRUE)\n", "dim(data)\n", "str(data)" ] }, { "attachments": {}, "cell_type": "markdown", "id": "38231f8a", "metadata": { "papermill": { "duration": 0.036035, "end_time": "2021-03-23T13:44:31.235882", "exception": false, "start_time": "2021-03-23T13:44:31.199847", "status": "completed" }, "tags": [] }, "source": [ "See the \"Details\" section on the description of the data set, which can be accessed by typing [help(fetch_401k)](https://docs.doubleml.org/r/stable/reference/fetch_401k.html)." ] }, { "attachments": {}, "cell_type": "markdown", "id": "9406ba1e", "metadata": { "papermill": { "duration": 0.036944, "end_time": "2021-03-23T13:44:31.669355", "exception": false, "start_time": "2021-03-23T13:44:31.632411", "status": "completed" }, "tags": [] }, "source": [ "The data consist of 9,915 observations at the household level drawn from the 1991 Survey of Income and Program Participation (SIPP). All the variables are referred to 1990. We use net financial assets (*net\\_tfa*) as the outcome variable, $Y$, in our analysis. The net financial assets are computed as the sum of IRA balances, 401(k) balances, checking accounts, saving bonds, other interest-earning accounts, other interest-earning assets, stocks, and mutual funds less non mortgage debts. " ] }, { "attachments": {}, "cell_type": "markdown", "id": "f34e7973", "metadata": { "papermill": { "duration": 0.035954, "end_time": "2021-03-23T13:44:31.741375", "exception": false, "start_time": "2021-03-23T13:44:31.705421", "status": "completed" }, "tags": [] }, "source": [ "Among the $9915$ individuals, $3682$ are eligible to participate in the program. The variable *e401* indicates eligibility and *p401* indicates participation, respectively." ] }, { "cell_type": "code", "execution_count": 2, "id": "6182ff56", "metadata": { "execution": { "iopub.execute_input": "2021-03-23T13:44:31.820240Z", "iopub.status.busy": "2021-03-23T13:44:31.818777Z", "iopub.status.idle": "2021-03-23T13:44:32.849032Z", "shell.execute_reply": "2021-03-23T13:44:32.847733Z" }, "papermill": { "duration": 1.07102, "end_time": "2021-03-23T13:44:32.849241", "exception": false, "start_time": "2021-03-23T13:44:31.778221", "status": "completed" }, "tags": [], "vscode": { "languageId": "r" } }, "outputs": [ { "data": { "image/png": "iVBORw0KGgoAAAANSUhEUgAAA0gAAANICAMAAADKOT/pAAAAY1BMVEUAAAAAv8RNTU1oaGh8fHyDg4OMjIyVlZWampqjo6Onp6evr6+ysrK5ubm9vb3BwcHHx8fJycnQ0NDR0dHY2NjZ2dne3t7h4eHk5OTp6enq6urr6+vv7+/w8PD19fX4dm3///8xfEhQAAAACXBIWXMAABJ0AAASdAHeZh94AAAgAElEQVR4nO2d60LbyrKE+4hLCBBInAAhK2T7/Z/yWLLblizJ5Rl66GlR348V60K7V00VksaSkTUh5N2IdwOELAHxboCQJSDeDRCyBMS7AUKWgHg3QMgSEO8GCFkC4t0AIUtAvBsgZAmIdwOELAHxboCQJSDeDSwKmWK/pf/v/M/P73W89XStc3k6rrK6aUQublbjXS/G77cSuZnu5UZkosRiEe8GFoVbkB6a7J6b4Xu9XmjfF69He34fd/W2+em36Y7fRJq37K7CId4NLAqnIK0mDhXncjN8r9em1/kwSa/NuKtrkYdBbz0eRK5zu4qHeDewKJyCdLroSZ5k+MOX/c4v+ntuIzb86c0xqhn21mfzA98z24qHeDewKOYdneb18+KWWXxAMwzSql26ftoErEvU02HH16M99adX8y2s9jH7BIh3A4siXJBuZBiPNj6325ebs7btPELHUyPjIN0eDlqTLVzsiy0f8W5gUUQL0pMM4/HWW+i/7gJ3efw27R567jbZQjs78VnmG8S7gUURLUjNUZCe+kehm8O5Xbvf5ehtbnvL0y3I5zkkiXcDiwIHabDHw+aXfHPzMjeN8NJ+oHP9fbbG9l850B4zDp/dtAeM09cobVQu+h09iM7CDRc2r+7H/3dNL3WHbduLqZW+wWe5ShLvBhZFUpBWejy4n47I7W5zs5qpMQrS9+7AoXxHx4PuxO6l33N7XfSiC+2W3fy1XLyM/+9W/djut/VzNNxl2Yh3A4siJUirQwBupyJye9i+mq4xClL3ev/hT3t86k27jWkGIW6ZC9L3qf+7y/6ibusmJZp+vi7XnwLxbmBRJASpu5a/3E80jyOyOevab32drLGvtX9x2zsIwTO77sRu2PPwI9fj/5uj5UFIdtu63w7N4YPcS5kTZGmIdwOLQiY4bOn/29l49/v+cjpIO5cedsRBeuqFB53ZbU/shvEYRuV0kNqfvz/adpyj9b2Ao+JiEO8GFsX5QeofL96a463dD+rEcbv1darGOEjtJzd6IojO7JpdEHKD1M5FHE2E3Hf5799Y1Kb5Yf0ZEO8GFsX5Qfre/30+fY2kW+/VjGcEqd13eyRDZ3a7E7v8IA2up7YdixxfEvWusxaOeDewKM4P0k3fhk9TEdkfTfZmPCNI3ZVXdygDZ3Z6Yve+IL0Oto1ztH5lkEgGx9Ybb9F/L/p7vh1v3cdBtzZTNSaC1Nm7O9KBMzs9scsP0nhJpH/VNF1jsYh3A4vi/CBNmzJt61SQVrvUgTO7/YkdmLUblDgnSL2zvYm9Fox4N7Ao/IPUZWEFz+xkTLt67nOkqf+7iSBdyNGjFwwSyaKCIN1u7Q/O7IoEadU7YZzea8GIdwOL4vwgtb+8B1dBMrXXfuvF1NbJIHW3/Ly2q07N2c0EaTClPZq6RkHqT2FM77VgxLuBRXF+kEa/++V4r8Gc3s1UjckgdRG9bSue+jR2Jkhzd39P/d+NZ+3Wg2uvDs7akRzOD9Lgc6T7qYgMPmX6PlVjOkhtZXk4DsGZPfcX2tdvs3tOfI7U/tv0o7j9FdFbXDDi3cCiOD9Ig1m1yTsb9Bf74aOh84L0tjvCnPv8wuCHe4/Fti8v5/ecurNhvTu5O0SYdzaQHM4PUnec2Vn2Ro63dit2p2aHp7/PC5I+Pn7uE3WDH+5uljvcSrua37OLzO14280gw7cymhBfKOLdwKKQSXRL/9/tDXbXJ+/+vt5Y8OlCRt/UM6p1sfX07opl93zG4bAwH+/x1v232rWcnsoe7HDYNji5uzj51ktCvBtYFAlBgs8jPew36+3Us7Vuem+03p0p9s7sUoL0JDJ635k67bu+jbf1T+7e5LNcIjFIpqQEaf/FPJuDyeRe+mTf/htPZ2u9Dr/i535bdNDV6Z57iy/7b4gcfdPq0Z7tb4LvE9t6J3ftJRKfkCXJJAVp47Prw3c2TNxNt/3OhtWg+nStt9v2HErnmds55/6cXVKQdt/93Ux89/fxnk1varu/7XByd83vbCAfiZh+2jK+z67MhUp78nny27baRj7HnB2D5MXgt/vgc9D3M7rP7q3McQHmBCZtQYh3A5+Upn/2tf/I1Yb2ImVwffNU6O6CW3Dm1owfq1gs4t3AJ6X76tLd69Xpi5hU3kYnirelrvibk3MJ/O5vUpzu9rrL1oUv3eyc1aXE6/qt/Vxq8CHoqthXYj2cjAr/GgUpT+9r644vad7Dttzwguu23G2jlyd+BTx8mu+0axHvBj4t3w9/0qux+8Xd1fs4/742s3+W760ZfaK7ZMS7gU/M95vukdKpv9aazfWm4EfOOK9m5xv5N2QJIamIdwOELAHxboCQJSDeDRCyBMS7AUKWgHg3QMgSEO8GCFkC4t0AIUtAvBtY/xemKKtysGarSpG3S4Fj89mrBmqVQWLVeqsGapVBYtV6qwZqlUFi1XqrBmqVQWLVeqsGapVBYtV6qwZqlUFi1XqrBmqVQWLVeqsGapVBYtV6qwZqlUFi1XqrBmqVQWLVeqsGapVBYtV6qwZqlUFi1XqrBmqVQWLVeqsGapVBYtV6qwZqlUFi1XqrBmqVQWLVeqsGapVBYtV6qwZqlUFi1XqrBmqVQWLVeqsGapVBYtV6qwZqlUFi1XqrBmqVQWLVeqsGapVBYtV6qwZqlUFi1XqrBmqVQWLVeqsGapVBYtV6qwZqlUFi1XqrBmo1VJD+R9LBspYZrGqLMkgMUg5Y1jKDVW1RBolBygHLWmawqi3KIDFIOWBZywxWtUUZJAYpByxrmcGqtiiDxCDlgGUtM1jVFmWQGKQcsKxlBqvaogwSg5QDlrXMYFVblEFikHLAspYZrGqLMkgMUg5Y1jKDVW1RBolBygHLWmawqi3KIDFIOWBZywxWtUUZJAYpByxrmcGqtmiAIP382jRXj393S893TdPcPa/PWjyrNW9PhgTLagKDNLdBUiv9/dJs2abj627p6/qMxfNa8/ZkSLCsJjBIcxsktdJV8+3PJk7fmqY9Jv1orn7+W//7edU8ruHima15ezIk54/4u2CQ5jZIYqHH5tv2xbcuHNs4rdd/mqs1XDyzNW9PhgTLagKDNLdB0ur8a5p/+moTjuf9oeZb8xstntuatydDgmU1gUGa2yBpdX4NT9IeG51HeG5+oMVzW/P2ZEiwrCYwSHMbJK3O0aHlrvmze/WnuUOL57bm7cmQYFlNYJDmNkhanbvNmV07/f31V7d41ey3tGd6pxfPbc3bkyHBsprAIM1tkLQ6TaPT31+3i70taFFbQXh7MiRQVVKCdwTpy9Xzv/X6912XpJwgHcMjkgVYVhN4RJrbIGl1muZqN2t31/xikOoBy2oCgzS3QdLqNPuJuN/t/AGDVAtYVhMYpLkNklbnKBtfhtMJpxfPbc3bkyHBsprAIM1tkLQ6d8Mgcfq7FrCsJjBIcxskrc5jo3d9dweZH72PXB/R4rmteXsyJFhWExikuQ2SVue5+bl79bu96e4QkO4uhtOL57bm7cmQYFlNYJDmNkhioS96SPraZUNvS/27vXg6vXhma96eDAmW1QQGaW6DJBb63Vy1Afr9dXvV87h/UOIHXjyzNW9PhuT8EX8XDNLcBkmt9JzyJB8f7PsosKwmMEhzGyS51L/HL5tk7K95fg0fJj+9eFZr3p4MCZbVBAZpboMUebsUGCQLsKxlBqvaogwSg5QDlrXMYFVblEFikHLAspYZrGqLMkgMUg5Y1jKDVW1RBolBygHLWmawqi3KIDFIOWBZywxWtUUZJAYpByxrmcGqtiiDxCDlgGUtM1jVFmWQGKQcsKxlBqvaogwSg5QDlrXMYFVblEFikHLAspYZrGqLMkgMUg5Y1jKDVW1RBolBygHLWmawqi3KIDFIOWBZywxWtUUZJAYpByxrmcGqtiiDxCDlgGUtM1jVFmWQGKQcsKxlBqvaogwSg5QDlrXMYFVblEFikHLAspYZrGqLMkgMUg5Y1jKDVW1RBolBygHLWmawqi3KIDFIOWBZywxWtUUZJAYpByxrmcGqtiiDxCDlgGUtM1jVFmWQGKQcsKxlBqvaogwSg5QDlrXMYFVblEFikHLAspYZrGqLMkgMUg5Y1jKDVW1RBolBygHLWmawqi3KIDFIOWBZywxWtUUZJAYpByxrmcGqtiiDxCDlgGUtM1jVFmWQGKQcsKxlBqvaogwSg5QDlrXMYFVblEFikHLAspYZrGqLMkgMUg5Y1jKDVW1RBolBygHLWmawqi3KIDFIOWBZywxWtUUZJAYpByxrmcGqtiiDxCDlgGUtM1jVFmWQGKQcsKxlBqvaogwSg5QDlrXMYFVblEFikHLAspYZrGqLMkgMUg5Y1jKDVW1RBolBygHLWmawqi3KIDFIOWBZywxWtUUZJAYpByxrmcGqtiiDxCDlgGUtM1jVFmWQGKQcsKxlBqvaogwSg5QDlrXMYFVblEFikHLAspYZrGqLMkgMUg5Y1jKDVW1RBolBygHLWmawqi3KIDFIOWBZywxWtUUZJAYpByxrmcGqtiiDxCDlgGUtM1jVFmWQGKQcsKxlBqvaogwSg5QDlrXMYFVblEFikHLAspYZrGqLMkgMUg5Y1jKDVW1RBolBygHLWmawqi3KIDFIOWBZywxWtUUZJAYpByxrmcGqtiiDxCDlgGUtM1jVFmWQGKQcsKxlBqvaogwSg5QDlrXMYFVblEFikHLAspYZrGqLMkgMUg5Y1jKDVW1RBolBygHLWmawqi3KIDFIOWBZywxWtUUZJAYpByxrmcGqtiiDxCDlgGUtM1jVFmWQGKQcsKxlBqvaogwSg5QDlrXMYFVblEFikHLAspYZrGqLMkgMUg5Y1jKDVW1RBolBygHLWmawqi3KIDFIOWBZywxWtUUZJAYpByxrmcGqtiiDxCDlgGUtM1jVFmWQGKQcsKxlBqvaossP0n8Ib0+GBKpKSuAZpGN4RLIAy1pmsKotuvwj0jEMkgVY1jKDVW1RBolBygHLWmawqi3KIDFIOWBZywxWtUUZJAYpByxrmcGqtiiDxCDlgGUtM1jVFmWQGKQcsKxlBqvaogwSg5QDlrXMYFVblEFikHLAspYZrGqLMkgMUg5Y1jKDVW1RBolBygHLWmawqi3KIDFIOWBZywxWtUUZJAYpByxrmcGqtiiDxCDlgGUtM1jVFmWQGKQcsKxlBqvaogwSg5QDlrXMYFVblEFikHLAspYZrGqLMkgMUg5Y1jKDVW1RBolBygHLWmawqi3KIDFIOWBZywxWtUUZJAYpByxrmcGqtiiDxCDlgGUtM1jVFmWQGKQcsKxlBqvaogwSg5QDlrXMYFVblEFikHLAspYZrGqLMkgMUg5Y1jKDVW1RBolBygHLWmawqi3KIDFIOWBZywxWtUUZJAYpByxrmcGqtiiDxCDlgGUtM1jVFmWQGKQcsKxlBqvaogwSg5QDlrXMYFVblEFikHLAspYZrGqLMkgMUg5Y1jKDVW1RBolBygHLWmawqi3KIDFIOWBZywxWtUUZJAYpByxrmcGqtiiDxCDlgGUtM1jVFmWQGKQcsKxlBqvaogwSg5QDlrXMYFVblEFikHLAspYZrGqLMkgMUg5Y1jKDVW1RBolBygHLWmawqi3KIDFIOWBZywxWtUUZJAYpByxrmcGqtiiDxCDlgGUtM1jVFmWQGKQcsKxlBqvaogwSg5QDlrXMYFVblEFikHLAspYZrGqLMkgMUg5Y1jKDVW1RBolBygHLWmawqi3KIDFIOWBZywxWtUUZJAYpByxrmcGqtiiDxCDlgGUtM1jVFmWQGKQcsKxlBqvaogwSg5QDlrXMYFVblEFikHLAspYZrGqLMkgMUg5Y1jKDVW1RBolBygHLWmawqi3KIDFIOWBZywxWtUUZJAYpByxrmcGqtiiDxCDlgGUtM1jVFmWQGKQcsKxlBqvaogwSg5QDlrXMYFVblEFikHLAspYZrGqLMkgMUg5Y1jKDVW1RBolBygHLWmawqi3KIDFIOWBZywxWtUUZJAYpByxrmcGqtiiDxCDlgGUtM1jVFmWQGKQcsKxlBqvaogwSg5QDlrXMYFVblEFikHLAspYZrGqLMkgMUg5Y1jKDVW1RBolBygHLWmawqi3KIDFIOWBZywxWtUUZJAYpByxrmcGqtiiDxCDlgGUtM1jVFmWQGKQcsKxlBqvaogwSg5QDlrXMYFVblEFikHLAspYZrGqLMkgMUg5Y1jKDVW1RBolBygHLWmawqi0aJEi/m2b36vmuaZq75/VZi2e15u3JkGBZTWCQ5jZIVrl/Vxqkr82Wr+csnteatydDgmU1gUGa2yBZ5drjTPfiR3P189/638+r5hEvntmatydDgmU1gUGa2yA51X41X3dBapq/3b9/miu8eGZr3p4MCZbVBAZpboNkFPt7dfV3G6Tn/aHmW/MbLZ7bmrcnQ4JlNYFBmtsgGcW+Nr/W2yA9NjqP8Nz8QIvntubtyZBgWU1gkOY2SHqtn83deheku+bPbuWfduXpxXNb8/ZkSLCsJjBIcxskudTf5uqfBumq2a9uL4NOL57bmrcnQ4JlNYFBmtsgyaW+dNc72yA1vag0aFFbQXh7MiRQVVKCdwTpR/PtkIycIB3DI5IFWFYTeESa2yCJhXQmm0GqCyyrCQzS3AZJLHS1m0BgkOoCy2oCgzS3QdLqfNN57G0yvgynE04vntuatydDgmU1gUGa2yBpdZo+nP6uByyrCQzS3AZJq3MUpB+9j1wf0eK5rXl7MiRYVhMYpLkNklnw+Bah7i6G04vntubtyZBgWU1gkOY2SGbBo5tW/+rkw6nFM1vz9mRIsKwmMEhzGySz4C4aj/sHJX7gxTNb8/ZkSLCsJjBIcxsksyAf7KsLLKsJDNLcBsksuD9Z+zV8mPz04lmteXsyJFhWExikuQ1S5O1SYJAswLKWGaxqizJIDFIOWNYyg1VtUQaJQcoBy1pmsKotyiAxSDlgWcsMVrVFGSQGKQcsa5nBqrYog8Qg5YBlLTNY1RZlkBikHLCsZQar2qIMEoOUA5a1zGBVW5RBYpBywLKWGaxqizJIDFIOWNYyg1VtUQaJQcoBy1pmsKotyiAxSDlgWcsMVrVFGSQGKQcsa5nBqrYog8Qg5YBlLTNY1RZlkBikHLCsZQar2qIMEoOUA5a1zGBVW5RBYpBywLKWGaxqizJIDFIOWNYyg1VtUQaJQcoBy/p/JB0sqyLJxreGQbIAy+rtyZBgWRVJNr41DJIFWFZvT4YEy6pIsvGtYZAswLJ6ezIkWFZFko1vDYNkAZbV25MhwbIqkmx8axgkC7Cs3p4MCZZVkWTjW8MgWYBl9fZkSLCsiiQb3xoGyQIsq7cnQ4JlVSTZ+NYwSBZgWb09GRIsqyLJxreGQbIAy+rtyZBgWRVJNr41DJIFWFZvT4YEy6pIsvGtYZAswLJ6ezIkWFZFko1vDYNkAZbV25MhwbIqMl4jg3WNXM79rA0MkgVYVm9PhgTLqsh4zTBIR4v2MEgWYFm9PRkSLKsi4zWD5KwYpBBgWb09GRIsqyK91zcyycW5kciDQbIAy+rtyZBgWRXpvX6bDtLDuZHIg0GyAMvq7cmQYFkV6S88TMTosnCOGCQTsKzengwJllWR8ZrSF0VHMEgWYFm9PRkSLKsi4zUMUkCwrN6eDAmWVZFk41vDIFmAZfX2ZEiwrIokG98aBskCLKu3J0OCZVVkYt3L7cVgvgEk4Z0wSBZgWb09GRIsqyLjVavjiTsUhffBIFmAZfX2ZEiwrIqM1ryMZsBhFt4Fg2QBltXbkyHBsioyWnPbfgb7igNgBYNkAZbV25MhwbIqMlrTFL+XYQiDZAGW1duTIcGyKjJeI/KG7W8Hg2QBltXbkyHBsioyXsMPZAOCZfX2ZEiwrIqM1lwySAHBsnp7MiRYVkVGax5EVtj+djBIFmBZvT0ZEiyrIuNVjTQfOGnHIJmAZfX2ZEiwrIqMV71ukvSB898MkgVYVm9PhgTLqsh4DT+QDQiW1duTIcGyKjJewyAFBMvq7cmQYFkVGa9hkAKCZfX2ZEiwrIokG98aBskCLKu3J0OCZVUk2fjWMEgWYFm9PRkSLKsiyca3hkGyAMvq7cmQYFkVSTa+NQySBVhWb0+GBMuqyHgNJxsCgmX19mRIsKyKjNcwSAHBsnp7MiRYVkXGaxikgGBZvT0ZEiyrIqc8/vrQNMVvYGWQLMCyensyJFhWRU67/LWR+9N7vBsGyQIsq7cnQ4JlVQTY/EHkBezyThgkC7Cs3p4MCZZVEWDzN5EbsMs7YZAswLJ6ezIkWFZFkM852RACLKu3J0OCZVUE2PyNQQoBltXbkyHBsioCbP7Av9gXAiyrtydDgmVV5KTJX+83B6Tbk7u8GwbJAiyrtydDgmVVZLzmmMLfcscgWYBl9fZkSLCsiozXHFH6E1kGyQIsq7cnQ4JlVWS8ps/FbeFPkRgkG7Cs3p4MCZZVkWTjW8MgWYBl9fZkSLCsiiQb3xoGyQIsq7cnQ4JlVSTZ+NYwSBZgWb09GRIsqyLJxreGQbIAy+rtyZBgWRWZXPtye9lONVyXn2pgkGzAsnp7MiRYVkUm1r1dH6btrot/dzGDZAGW1duTIcGyKjJe9dr0J8CLf6E+g2QBltXbkyHBsioyXtWe1d0+ta+e2r8newmC8F4YJAuwrN6eDAmWVZHRmlX/bobBgg3/Ibw9GRKo6n/engwJEvVUkK4Hf4z5YXOZZBukUbCOV3h7MiRYVm9PhgTLqsh4zeA2VT6PFAMsq7cnQ4JlVWS8ZpgcBikEWFZvT4YEy6rIeA2PSAHBsnp7MiRYVkVGa3iNFBEsq7cnQ4JlVWS0ZjRr9/1UDN4Pg2QBltXbkyHBsioyXnUx/Byp8Fc2MEgmYFm9PRkSLKsi41W8syEgWFZvT4YEy6rIxLrX/r12hb+xgUGyAcvq7cmQYFkVmVz7ct9l6fqed38HAcvq7cmQYFkVSTa+NQySBVhWb0+GBMuqSLLxrWGQLMCyensyJFhWRSbXrrZTdc1N8b+OxCDZgGX19mRIsKyKTKx7u9zdzdBOfvPBvhBgWb09GRIsqyLjVd3093Yjp7+jgGX19mRIsKyKjFdtcnSzm/ReXWySdDIG74dBsgDL6u3JkGBZFRmtGd5ddzG4864EDJIFWFZvT4YEy6rIaM3mAql3Nrcq/qw5g2QBltXbkyHBsioyXsPnkQKCZfX2ZEiwrIqM1zBIAcGyensyJFhWRUZrLgandm/FZxsYJAuwrN6eDAmWVZHRmtvB3+h74F81DwGW1duTIcGyKjJa89J/sG+wUAYGyQIsq7cnQ4JlVWS86ppfEBkPLKu3J0OCZVVkvOpt+GBf6QeSGCQLsKzengwJllWRqZX9B/tADN4Pg2QBltXbkyHBsioyufb1/rrhg32RwLJ6ezIkWFZFko1vDYNkAZbV25MhwbIqkmx8axgkC7Cs3p4MCZZVkWTjW8MgWYBl9fZkSLCsiiQb3xoGyQIsq7cnQ4JlVSTZ+NYwSBZgWb09GRIsqyLJxreGQbIAy+rtyZBgWRVJNr41DJIFWFZvT4YEy6pIsvGtYZAswLJ6ezIkWFZFko1vDYNkAZbV25MhwbIqkmx8axgkC7Cs3p4MCZZVkWTjW8MgWYBl9fZkSLCsiiQb3xoGyQIsq7cnQ4JlVSTZ+NYwSBZgWb09GRIsqyLJxreGQbIAy+rtyZBgWRVJNr41DJIFWFZvT4YEy6pIsvGtYZAswLJ6ezIkWFZFko1vDYNkAZbV25MhwbIqkmx8axgkC7Cs3p4MCZZVkWTjW8MgWYBl9fZkSLCsiiQb3xoGyQIsq7cnQ4JlVSTZ+NYwSBZgWb09GRIsqyLJxreGQbIAy+rtyZBgWRVJNr41DJIFWFZvT4YEy6pIsvGtYZAswLJ6ezIkWFZFko1vDYNkAZbV25MhwbIqkmx8axgkC7Cs3p4MCZZVkWTjW8MgWYBl9fZkSLCsiiQb3xoGyQIsq7cnQ4JlVSTZ+NYwSBZgWb09GRIsqyLJxreGQbIAy+rtyZBgWRVJNr41DJIFWFZvT4YEy6pIsvGtYZAswLJ6ezIkWFZFko1vDYNkAZbV25MhwbIqkmx8axgkC7Cs3p4MCZZVkWTjW8MgWYBl9fZkSLCsiiQb3xoGyQIsq7cnQ4JlVSTZ+NYwSBZgWb09GRIsqyLJxreGQbIAy+rtyZBgWRVJNr41DJIFWFZvT4YEy6pIsvGtYZAswLJ6ezIkWFZFko1vDYNkAZbV25MhwbIqkmx8axgkC7Cs3p4MCZZVkWTjW8MgWYBl9fZkSLCsiiQb3xoGyQIsq7cnQ4JlVSTZ+NYwSBZgWb09GRIsqyLJxreGQbIAy+rtyZBgWRVJNr41DJIFWFZvT4YEy6pIsvGtYZAswLJ6ezIkWFZFko1vDYNkAZbV25MhwbIqkmx8axgkC7Cs3p4MCZZVkWTjW8MgWYBl9fZkSLCsiiQb3xoGyQIsq7cnQ4JlVSTZ+NYwSBZgWb09GRIsqyLJxreGQbIAy+rtyZBgWRVJNr41DJIFWFZvT4YEy6pIsvGtYZAswLJ6ezIkWFZFUn3/7/FL03x5/KfLz3dN09w9n7c4BYNkAZbV25MhwbIqcsrjEzw3W6524fi6W/56zuIkDJIFWFZvT4YEy6rICYtP8KdpHv+u13/vmqvumPSjufr5b/3v51XziBenYZAswLJ6ezIkWFZF5h0+xbfml77ostE0f7vFP80VXpyGQbIAy+rtyZBgWRWZd/gUV83uxd8uG8/7Q8235jdanIFBsgDL6u3JkGBZFZk1OKBpI/XY6DzCc/MDLc7AIFmAZfX2ZEiwrIrMGhzQBemu+bNb/NPcocUZGCQLsKzengwJllWRWYOf5nfzbd0709sE6wotzsAgWYBl9fZkSLCsiswa/DR33Wlb04tKgxa1FYS3J0MCVf3P28pWlJkAABlWSURBVJMhQaK+O0i/t+dqOUE6hkckC7Cs3p4MCZZVkVmDn+Lv1fZjJAapFrCs3p4MCZZVkVmDn+LL7gMiBqkWsKzengwJllWRWYOf4KvOa38ZTiecXpyBQbIAy+rtyZBgWRWZNfg8+xxx+rsasKzengwJllWRWYPP8feQo/WP3keuj2hxBgbJAiyrtydDgmVVZNbgM/y9OuSoF5DuLobTizMwSBZgWb09GRIsqyKzBp/mubn601vU21L/bmcTTi9OwyBZgGX19mRIsKyKzDt8ir9XV3/7y4/7ByV+4MVpGCQLsKzengwJllWReYdP8a050K3gg311gGX19mRIsKyKnLD4BM1xkNa/hg+Tn16cgkGyAMvq7cmQYFkVOeXxD4FBsgDL6u3JkGBZFUk2vjUMkgVYVm9PhgTLqkiy8a1hkCzAsnp7MiRYVkWSjW8Ng2QBltXbkyHBsiqSbHxrGCQLsKzengwJllWRZONbwyBZgGX19mRIsKyKJBvfGgbJAiyrtydDgmVVJNn41jBIFmBZvT0ZEiyrIsnGt4ZBsgDL6u3JkGBZFUk2vjUMkgVYVm9PhgTLqkiy8a1hkCzAsnp7MiRYVkWSjW8Ng2QBltXbkyHBsiqSbHxrGCQLsKzengwJllWRZONbwyBZgGX19mRIsKyKJBvfGgbJAiyrtydDgmVVJNn41jBIFmBZvT0ZEiyrIsnGt4ZBsgDL6u3JkGBZFUk2vjUMkgVYVm9PhgTLqkiy8a1hkCzAsnp7MiRYVkWSjW8Ng2QBltXbkyHBsiqSbHxrGCQLsKzengwJllWRZONbwyBZgGX19mRIsKyKJBvfGgbJAiyrtydDgmVVJNn41jBIFmBZvT0ZEiyrIsnGt4ZBsgDL6u3JkGBZFUk2vjUMkgVYVm9PhgTLqkiy8a1hkCzAsnp7MiRYVkWSjW8Ng2QBltXbkyHBsiqSbHxrGCQLsKzengwJllWRZONbwyBZgGX19mRIsKyKJBvfGgbJAiyrtydDgmVVJNn41jBIFmBZvT0ZEiyrIsnGt4ZBsgDL6u3JkGBZFUk2vjUMkgVYVm9PhgTLqkiy8a1hkCzAsnp7MiRYVkWSjW8Ng2QBltXbkyHBsiqSbHxrGCQLsKzengwJllWRZONbwyBZgGX19mRIsKyKJBvfGgbJAiyrtydDgmVVJNn41jBIFmBZvT0ZEiyrIsnGt4ZBsgDL6u3JkGBZFUk2vjUMkgVYVm9PhgTLqkiy8a1hkCzAsnp7MiRYVkWSjW8Ng2QBltXbkyHBsiqSbHxrGCQLsKzengwJllWRZONbwyBZgGX19mRIsKyKJBvfGgbJAiyrtydDgmVVJNn41jBIFmBZvT0ZEiyrIsnGt4ZBsgDL6u3JkGBZFUk2vjUMkgVYVm9PhgTLqkiy8a1hkCzAsnp7MiRYVkWSjW8Ng2QBltXbkyHBsiqSbHxrGCQLsKzengwJllWRZONbwyBZgGX19mRIsKyKJBvfGgbJAiyrtydDgmVVJNn41jBIFmBZvT0ZEiyrIsnGt4ZBsgDL6u3JkGBZFUk2vjUMkgVYVm9PhgTLqkiy8a1hkCzAsnp7MiRYVkWSjW8Ng2QBltXbkyHBsiqSbHxrGCQLsKzengwJllWRZONbwyBZgGX19mRIsKyKJBvfGgbJAiyrtydDgmVVJNn41jBIFmBZvT0ZEiyrIsnGt4ZBsgDL6u3JkGBZFUk2vjUMkgVYVm9PhgTLqkiy8a1hkCzAsnp7MiRYVkWSjW8Ng2QBltXbkyHBsiqSbHxrGCQLsKzengwJllWRZONbwyBZgGX19mRIsKyKJBvfGgbJAiyrtydDgmVVJNn41jBIFmBZvT0ZEiyrIsnGt4ZBsgDL6u3JkGBZFUk2vjUMkgVYVm9PhgTLqkiy8a1hkCzAsnp7MiRYVkWSjW8Ng2QBltXbkyHBsiqSbHxrGCQLsKzengwJllWRZONbwyBZgGX19mRIsKyKJBvfGgbJAiyrtydDgmVVJNn41jBIFmBZvT0ZEiyrIsnGt4ZBsgDL6u3JkGBZFUk2/nv5D+HtyZBAVf/z9mRIkKieQTqGRyQLsKzengwJllWRZONbwyBZgGX19mRIsKyKJBvfGgbJAiyrtydDgmVVJNn41jBIFmBZvT0ZEiyrIsnGt4ZBsgDL6u3JkGBZFUk2vjUMkgVYVm9PhgTLqkiy8a1hkCzAsnp7MiRYVkWSjW8Ng2QBltXbkyHBsiqSbHxrGCQLsKzengwJllWRZONbwyBZgGX19mRIsKyKJBvfGgbJAiyrtydDgmVVJNn41jBIFmBZvT0ZEiyrIsnGt4ZBsgDL6u3JkGBZFUk2vjUMkgVYVm9PhgTLqkiy8a1hkCzAsnp7MiRYVkWSjW8Ng2QBltXbkyHBsiqSbHxrGCQLsKzengwJllWRZONbwyBZgGX19mRIsKyKJBvfGgbJAiyrtydDgmVVJNn41jBIFmBZvT0ZEiyrIsnGt4ZBsgDL6u3JkGBZFUk2vjUMkgVYVm9PhgTLqkiy8a1hkCzAsnp7MiRYVkWSjW8Ng2QBltXbkyHBsiqSbHxrGCQLsKzengwJllWRZONbwyBZgGX19mRIsKyKJBvfGgbJAiyrtydDgmVVJNn41jBIFmBZvT0ZEiyrIsnGt4ZBsgDL6u3JkGBZFUk2vjUMkgVYVm9PhgTLqkiy8a1hkCzAsnp7MiRYVkWSjW8Ng2QBltXbkyHBsiqSbHxrGCQLsKzengwJllWRZONbwyBZgGX19mRIsKyKJBvfGgbJAiyrtydDgmVVJNn41jBIFmBZvT0ZEiyrIsnGt4ZBsgDL6u3JkGBZFUk2vjUMkgVYVm9PhgTLqkiy8a1hkCzAsnp7MiRYVkWSjW8Ng2QBltXbkyHBsiqSbHxrGCQLsKzengwJllWRZONbwyBZgGX19mRIsKyKJBvfGgbJAiyrtydDgmVVJNn41jBIFmBZvT0ZEiyrIsnGt4ZBsgDL6u3JkGBZFUk2vjUMkgVYVm9PhgTLqkiy8a1hkCzAsnp7MiRYVkWSjW8Ng2QBltXbkyHBsiqSbHxrGCQLsKzengwJllWRZONbwyBZgGX19mRIsKyKJBvfGgbJAiyrtydDgmVVJNn41jBIFmBZvT0ZEiyrIsnGt4ZBsgDL6u3JkGBZFUk2vjUMkgVYVm9PhgTLqkiy8a1hkCzAsnp7MiRYVkWSjW8Ng2QBltXbkyHBsiqSbHxrGCQLsKzengwJllWRZONbwyBZgGX19mRIsKyKJBvfGgbJAiyrtydDgmVVJNn41jBIFmBZvT0ZEiyrIsnGt4ZBsgDL6u3JkGBZFUk2vjUMkgVYVm9PhgTLqkiy8a1hkCzAsnp7MiRYVkWSjW8Ng2QBltXbkyHBsiqSbHxrGCQLsKzengwJllWRZONbwyBZgGX19mRIsKyKJBvfGgbJAiyrtydDgmVVJNn41jBIFmBZvT0ZEiyrIsnGt4ZBsgDL6u3JkGBZFUk2vjUMkgVYVm9PhgTLqkiy8a1hkCzAsnp7MiRYVkWSjW8Ng2QBltXbkyHBsiqSbHxrGCQLsKzengwJllWRZONbwyBZgGX19mRIsKyKJBvfGgbJAiyrtydDgmVVJNn41jBIFmBZvT0ZEiyrIsnGt4ZBsgDL6u3JkGBZFUk2vjUMkgVYVm9PhgTLqkiy8a1hkCzAsnp7MiRYVkWSjW8Ng2QBltXbkyHBsiqSbHxrGCQLsKzengwJllWRZONbwyBZgGX19mRIsKyKJBvfGgbJAiyrtydDgmVVJNn4STzfNU1z93xqFwbJAiyrtydDgmVV5JTH383XZsvXE/swSBZgWb09GRIsqyInLP5ufjRXP/+t//28ah7nd2KQLMCyensyJFhWReYd/n6a5m/375/man4nBskCLKu3J0OCZVVk3uHv5nl/IPrW/J7di0GyAMvq7cmQYFkVmTX4+3lsdJbhufkxuxeDZAGW1duTIcGyKjJr8Pdz1/zZvfrT3M3uxSBZgGX19mRIsKyKzBr8/Vw1+5cnLpIYJAuwrN6eDAmWVZFZg7+fppl6+R8hS8EzSMfMZvw9FCnKqhysio5Ix3BsPnvVQK0ySKxab9VArfoE6UveZIMFixibz1I1UKuhpr8tWMTYfJaqgVr1CdKP3gey8zfbcWw+e9VArXrfInS4x2EMx+azVw3UqvNNq39PzDVwbD591UCtOgXpcf8YxfytdhybT181UKuhHuyzYBFj81mqBmrV7VHzXxmPmluwiLH5LFUDtRrqy0+qLcqqHCwGiVWrrRqoVQaJVeutGqhVBolV660aqFUGiVXrrRqoVQaJVeutGqhVBolV660aqFUGiVXrrRqoVQaJVeutGqhVBolV660aqFUGiVXrrRqoVQaJVeutGqhVBolV660aqFUGiVXrrRqoVQaJVeutGqhVBolV660aqFUGiVXrrRqo1ZqDRMgCEO8GCFkC4t0AIUtAvBsgZAmIdwOELAHxboCQJSDeDRCyBMS7AUKWgHg3QMgSEO8GCFkC4vO2z1PfCf6v2ePT1YjJNmdX+xJEUuXHuKEqZW0Z9Tohq3xsSzum/0rF79pGfeaPaZzzNzY+nCCSKs/jhqqUtWXc64Ss8rE9bfmx/7tJg7+I+bP56dHNLDNtzqz2JYikyvM42VXK2jLR64Ss8jHNDNG/5Pdn+MfO73ara2GmzZnVvgSRdMfPiUNklbKup3udkFU+ppsBh78t+6353Vt/VZeCM23Ode9KEEm3/NmcxH09NmeVss70OiWrfEg7Qx57f+289zcx/zXfHJqZZ6bNmdW+BJF0y+Y3/I/1sTmrlHWm1ylZ5WP6GXDX/Nm9+tPcHVb/bn61f+Dv6tufyZ/6cGbanFntSxBJtzR3f9cjc1Yp60yvU7LKh7a15erQV/98+Od+JqSOq82ZNmdW+xJE0i3/2v8cm7NKWWd6nZJVPrKrHb2++i1uMv6z7fv3XR3DPtPmzGpfgkja41i7KmXdMT56jmWVj22pY0azq/1Z8temhlORBQSpNkl7RA7SlKzygf0oULPnKq6RFxCkA3VI2iNykA4cZJWPaWUA1qyKs+RFBakOSXssI0gHWeVDOtm+p95T8QVeV1Yh5UybuHsHgkja47ifKmXdcSpIukk+pJPde+6ChGc6qxj1BUx/96hC0h5Bpr87KgvSnh+9z94mJ5P+Nl8+sJ05ZtqE3XsQRNIex+asUtYd80E6yCof08qAg1KHj7PX7VzIv92rX1VIOdPmzGpfgkjaY/4WoYpk3TGetRvLKh/Yzx69P/HvoMPH/a0hX+q422q6zbnVvgSR9MDsTatVybplfDvTWFb5wH56jegd8/27qv5tFlstf3+t5JGU6TbnVvsSRNIDo7hUKeuW0b12E7LKx7a0Y/gMl/b592q3upaPPKbbrPMJtCCS7tmrWbesHaNeJ2SVD++q41f/qeJD4H9+resOy5k2f9X4THQQSZVxkOqUtWWi15Gs8rEtEbJMxLsBQpaAeDdAyBIQ7wYIWQLi3QAhS0C8GyBkCYh3A4QsAfFugJAlIN4NELIExLsBQpaAeDdAyBIQ7waIMSuR/cubRuTiZjXY/nrYTuwQ7waILZucyO7lpey4fO3tcMkglUC8GyC2NPsg7XO0OSodtt8Kg1QC8W6AmNKlp3v1sHlxszkUvd5sXuzP7m6FQSqCeDdALOlyIt3LC5Hv+5UX4+3EFPFugBiyEtFTt5feCd0mU91V0svmxSWDVATxboBksmpP2a4HM3Kvm5ysd0HZnNnd6/r73cGpPdlbM0hFEO8GSBZvOpXQn5FrpFlrUDYxe9L1T22ANkjzfc0glUG8GyBZXOxn5A7fcHPZncHtgnIt8qIbNqd51+2/t29rBqkQ4t0AyWFzvGnas7rVxS4j624ioV21C8ogL/0FBqkI4t0AyWBzMdS8bV/qREI30dD+yyC5IN4NkAzuD58MrXaTCt1EQwuD5IJ4N0AyuB4Eozu3a/RiiUFyQbwbIBk00qdN0KWe4TFIPoh3AyQDGbKfaNhta/9pGKQPRbwbIBmMgiTHTE5/6886dLx4xLsBksEoDOMg3QyDdDP/s8QC8W6AZNA/3HSMg/Qg8qBb+68ZpDKIdwMkg/vdZ0ZT7ILyOrxp9eV4O7FFvBsgGWzO1XRyYdU/bWvRoFzsD0P3gyf7GKQiiHcDJIfNuZ08vG2OO+1TE0+DTRqUwYN9D+PtxBTxboDk8Nb7JOnoJG8flMN9rbt7Ho62E0vEuwGSxevlTI4OQXm9mMoRg1QG8W6AZLL9rq3bl+P1vaB0uzRHX8fFIBVBvBsgZAmIdwOELAHxboCQJSDeDRCyBMS7AUKWgHg3QMgSEO8GCFkC4t0AIUtAvBsgZAmIdwOELAHxboCQJSDeDRCyBMS7AUKWgHg3QMgSEO8GCFkC4t0AIUtAvBsgZAmIdwOELAHxboCQJSDeDRCyBMS7AUKWgHg3QMgSEO8GCFkC4t0AIUtAvBsgZAmIdwOELAHxboCQJSDeDRCyBMS7AUKWgHg3QMgSEO8GCFkC4t0AIUtAvBsgZAmIdwOELAHxboC48L950ov93zz2nVeKeDdAXGCQjBHvBogLDJIx4t0AcYFBMka8GyAuMEjGiHcDEXm9aUQuVufs+nZzZs2VXKMdZP+ye/+bYQOvh+03AntLDtLqWkSup+tmBele5rcFRLwbCMiLdJwTpIeDu0/z2sjb6R1kX+pSdly+9na4PLzVmzT9LVOkBknf83JqY06QVucqEwTxbiAgO1O9nLGrnGuXS3k4vUOzL7XP0eaodNh+23+rh2nD90gM0r00D2/rt4dGbie2ZgRpdbYyQRDvBgKy8QBwfX/Xs/b7Ls3pHbr0dK82Bzm5eW3PL/tHxVsZvFUj30/XSwySyPYQ9zLZZ3qQHoRBIgkeOHfXBkSzy8m21IVoSG4Ph6Te9o4HFMy0IK32B6IbeRpvTg3Sy+bXwiWD9OmxDxLy/eZESE/dXnondBe7A8XLRWfM/luhZKYF6XZ/6FvJ/XhzapA2rd6vGaRPjl6fdAur29bAF7eH66WX29bVD+Ndj+a92tVPF5sfbRf6lx6rm9H02Gt7kb+r8yAHK9/vDk7tyd5RZm9BNNOCdL2/IHyZmlxMDtL165pB+uz00vF6uO7fBeHterfcTZv1g/R6MZxq27xqJ+KknR7/LvsTprfL4W4dTZuKXZ2bw77rp+2Pr6X5fnzwe5LTV0lpQWoOpacCmhqkt60Ap/oLh3g3EI5eOprD/Nnu3OewpnkbZm64pdt2rT93cXDnxWG3/Ttedmdwu6Bc96YLN6d53fHh9m09Oots+lN6Y9KC1Cs95f/0yYaZQoER7wYCopbdnGU1q9bDq4vdByxtNG7bGbXr3Yq9u9st3QSy7MzfP4ztDizr7njTrHYV9Rzqdpu2XalBXmTg8N6G7g1PfTDFIBkj3g0ERC17sT84vO0OO/sjU5uInvu7z3C3W9oPUF62WzQp3/fT6a96vDpMJHQTDb13PTdID6fP7RgkY8S7gYDI2APbVff7g0wbrYverr0tt9uXcjD67T6Q99KbHttOKrzq3QSJQXqSyc9OFQbJGPFuICBHln15uN7ddnAtx5+y6K69LU/bZMjhWudw2XM9cGx3xGr0YikxSC9y8ua9tCBd9N7GYLJh1++J9uIh3g0EpGfZ7fS3zig0I3PormPH99Ycfqw/e7GdbrjUM7zUIK3l5AR4WpCMp793/Z1oLx7i3UBA9pbtTX8fh+No15NBOryUIfuJht5ezflBOm6mT1qQ7ntnnBMnjAwSg5TD3qLbA8jF9cOqVJDkmMnp79FbTC0PSQvSqneFN3HTO4PEIOWgFm2nvx9ee6ssgjT6+aMg3QyDdLMeV5mpNSAtSGs9wXydLMogMUg5qEVH09/9yYbtJYruOpxsuF4PfH44W7s+fjhjHKSH3q3n/dcTQbK7RmrvONo9RjFxqx2DtGaQchgfZh7kePp7tT1Y6C6T099a77o//T07Z737gdfhTasvx9sV01m7Ag/2MUikF6Stj1+2FzTdvztnX5zxgazWu90frg67aRLH73qxPwzdD57sOwqS6edIG75bP2rOIH16eudrF5sEvN53v6vf1vsbgbo7fC52u65039EtQlpvdThD0wKv7VMTw8+keldmhwf7HsbbdzycfhY+OUinyArSwhDvBgKill0Nrl66Q1Hv1tTu6rw7I2qvVd6Ot/R8/3Y4C+vtdnxA2f/A4b7WwYnWUZBM77UDMEgMUg57y95qNFb62Or+KYjdt490j1R3e088RrEv2Hta9PDR1PGJ2f4H9pWGFyxHQZq5nCGFEO8GAnKw7PZZve/7+37Wu6/KujxMrF3st3wfPdinfO+fxm2/a+t29NUq/XPBdpfm6Ou4hkF6Ou9bjogV4t0AWbdnhOd+/d253KAvbSC2iHcDZN2dAZ7+VrtU3s7/oiNigng3QFrQd5WkAr9FiBgj3g2QFvi9donA77Ujxoh3A6QDftNqEvibVokx4t0A6XjdP2NuwFsj6Lu/iTHi3QDZsjKcuDvjr1EQY8S7AUKWgHg3QMgSEO8GCFkC4t0AIUtAvBsgZAmIdwOELAHxboCQJSDeDRCyBMS7AUKWgHg3QMgSEO8GCFkC4t0AIUtAvBsgZAmIdwOELAHxboCQJfD/1BCyyT0gvbQAAAAASUVORK5CYII=", "text/plain": [ "plot without title" ] }, "metadata": { "image/png": { "height": 420, "width": 420 } }, "output_type": "display_data" } ], "source": [ "hist_e401 = ggplot(data, aes(x = e401, fill = factor(e401))) +\n", " geom_bar() + theme_minimal() + \n", " ggtitle(\"Eligibility, 401(k)\") +\n", " theme(legend.position = \"bottom\", plot.title = element_text(hjust = 0.5),\n", " text = element_text(size = 20)) \n", "hist_e401" ] }, { "cell_type": "code", "execution_count": 3, "id": "fd3f9739", "metadata": { "vscode": { "languageId": "r" } }, "outputs": [ { "data": { "image/png": "iVBORw0KGgoAAAANSUhEUgAAA0gAAANICAMAAADKOT/pAAAAY1BMVEUAAAAAv8RNTU1oaGh8fHyDg4OMjIyVlZWampqjo6Onp6evr6+ysrK5ubm9vb3BwcHHx8fJycnQ0NDR0dHY2NjZ2dne3t7h4eHk5OTp6enq6urr6+vv7+/w8PD19fX4dm3///8xfEhQAAAACXBIWXMAABJ0AAASdAHeZh94AAAgAElEQVR4nO2dbUPaWtOF54kvtWrV0qq1p/bm///KhwQ2JCRhsbeznZl0XR/OAQLDds1cDYSgsiaEfBixXgAhS0CsF0DIEhDrBRCyBMR6AYQsAbFeACFLQKwXQMgSEOsFELIExHoBhCwBsV4AIUtArBdAyBIQ6wVEQgZc3zx9oE7Jtg+WLuT5uOTqphG5uFmN73oxfvKVyM30wm5EJkpERawXEAk5pnnIevxDc6hz6jkKlnZO6UKaYcm3i/TTX7wd3fNp/OTvm0e/Ty/sfZPfu+5SDRHrBURiJJLI7fmPXu3/vVYX6bzSZdwMS741vZ9+aNJbM37ya5GHuYU9bPbqums1RKwXEIkJkeT8fdJhlNRFOq90Ec8yLHnZ/+Ev+vfcKjZ89GYfdWJXuXlA6atjd4j1AiIxHIbX++7VXeGjValYuhmKtGqvXT9vBOuMej7c8e3onunRq/klrnLic45YLyASx8Pw1t5w9r+pIUW6kaEel4eXs5tXbdvjCB3PjYxFuj3stCaXeJH12tg1Yr2ASIyG4UEyXuZHFOlZhnq89670L3fCXR4v473378zkEtujEws53iDWC4jEaBjeciY4okjNkUjP/b3QzeG1XXu/y9EybnvXp5coi9klifUCIjEehsGQ3bcvduTy8AFLt/VtM2+X97136Ed1VjebFzjN/kFp2/b/r7ebEb18GPyrffw8Z5dOW99vL4c3z9KqctEv2e6CHyaubC7dj/NpetYdtm3fTK3SEyzkXZJYLyASp0R63X+8cviApdvajc3N3LQfHnb5NqjY/X+129Ychn78PHOlJz7w6bbupb5EP2/3wu61/1O3Cr/ulyL7F7Zy8TrOZyW9j1z32/oeDe8SGrFeQCRGIrUv7bbvptPI7wb/fX//bu8h/c39OqvRg/oi9bamaZt4njNK9w+d3R5uRSa1I38/+KnnRHqayueyfzVt6w5KNH2/oM8hEOsFRGIkUjuU3YuX9gN8uWlH7K19wdO+ztndv/23+v1h+Oj9pe49Vncw+SINZV+kRi5W6UDzdnynn2eydPe2JR2nbg67u83j31LR0zuD7oXd8KcefuR6nMfR9YEku22d3s3hg9zLoxJhEesFROJ4cFb7YbyXw9uBp/0AdWP7Pn70/lL7D/zuzXazk6UvUprD6/0//dPPM1f65vjh0rPnUvpHryd43vnb/6mHCZwW6fkg+n7bsUfdD9T7MCouYr2ASAwG5W3VvWrbvrK7mDRGpH9Qajzt7Q4pvdd+2N13IFKauCZdnn4eUPrwcJHD4fpnOToz4ZhmJ0KpSO1P9DTc1r09u+yfWNT+a5B3wqJTxHoBkZAJJv45HYh0fMisf+mh92/2+27y+yLtJbyfHLdTIvVLdw+/P17Q4GOgCW6SaKUiDd5PbX+ezqPBs/TeZ8VGrBcQiQmP7o/v83p/+FxSpH9e53jah6M23DaQcGLc+s8zU/p59PDBgk6LlF7YfUyk4U8/9qjbc1Kkf42RRhf9/dHb0+31/tTo/f2Hjx5eupgY5b5I+zFMu6uZ55ko3d7hffTwwYJOi5Re2JWLNL4mMv6X5/Qq4iDWC4jEQKLm+n6g0fVg6/7+w0cPL00N0fS2w7Wp54GlJ289OcL7F3bgqN3g49RzRDreBVOkf5ATTd9/anN5fzhvqIJIk89TQSQZ09489znSVMUJkS5kdICDIv2DzDe9+3ynuXkavqnQF2n6eaKItOq9YDxrFYEQ6wVEYr7p7XGxy9HdkEgT3yidnvn3VH76eWZKf+Q90oxIg0Pao0PXSKT+IYyzVhEIsV5AJOabftGbj/Nf2g3+fZfx4e/9tv1J19PPM1P6uf/w6+H9Tv4061mR5s7+nqo4Pmq3Hrz32v8QPGr3rzE/ev0tq7NF6n88lEa0L9L+RdBt2hFMP89E6Yfjh98fP/zsfcHsg9rL77P3nPgcqf1/I4MTKl4FnF8RBbFeQCTOE+nwvQMk0uGc1+2wr9ZDkQanJpx4npnS02c2nPHTzP9sx2ceXc7fc+rMhvXuxd1hP8YzG/5F5kfv8jAP27OrJ+4/MfeHc+32g98XKY3s4fvd088zV3ryXLszfpohg/t1u8H9ubBHZ70OKz7L5AlSN33Fu5/i+DPpkIj1AiIxP3rdqdjthK92v2VnvAPo9iCbwXrr3d69zWlPxn5q5OjF27bM/teMNKeeZ7b0YeKHX3Y6unJaqfFPsef0oezBHQ7bBi/upj6TDolYLyASJwau99vebtOJ3Ef3T1/u69/e/9LQ7eA5Og/225q3U88DS0/+Kp8ykZ57VZu3U/fsVjVxhm3/xd27LOQtEkXK4cTA7X9vYvPUDdDD+P7pLoPbn/dmHH21qPt/Oovh+u3k80yXfp35huz4p8kRaf3ajMrO3LNV+WliW+/F3ZPwG7L/ICcH7qHdf1w/vW+PRF1O3P/9dvf1vcHtq/bEucvb9E/3QKR1+1WN5mZwjHnieWZLH/+Sbg2RdmWnfufD8T2b3qHt/rbDi7tr/s4GUpfTw63/bDWqtu/oTv62rXdZyDE7iuSWTxXpvc5+AXoCTYuDWC+ATPOpIj1XOrvgFrxyaya+0BUUsV4AmeZTRbqt9Y6/OXksgb/7m1TnM0VaVfuVWA8nVeFfoyDV+UyRbuudNnp54l3Sw1J+p12LWC+ATPO5R+2q8dbM/lm+92b0iW5gxHoBZJqFiLT/G7Jj+DdkCSFHiPUCCFkCYr0AQpaAWC+AkCUg1gsgZAmI9QIIWQJivQBCloBYL4CQJSDWC1j/F6Yoq7JZs1WlytPlwN7861UDLZUisarfqoGWSpFY1W/VQEulSKzqt2qgpVIkVvVbNdBSKRKr+q0aaKkUiVX9Vg20VIrEqn6rBloqRWJVv1UDLZUisarfqoGWSpFY1W/VQEulSKzqt2qgpVIkVvVbNdBSKRKr+q0aaKkUiVX9Vg20VIrEqn6rBloqRWJVv1UDLZUisarfqoGWSpFY1W/VQEulSKzqt2qgpVIkVvVbNdBSKRKr+q0aaKkUiVX9Vg20VIrEqn6rBloqRWJVv1UDLZUisarfqoGWSpFY1W/VQEulSKzqt2qgpYYS6X8kHxxrnWa5LUqRKFIJONY6zXJblCJRpBJwrHWa5bYoRaJIJeBY6zTLbVGKRJFKwLHWaZbbohSJIpWAY63TLLdFKRJFKgHHWqdZbotSJIpUAo61TrPcFqVIFKkEHGudZrktSpEoUgk41jrNcluUIlGkEnCsdZrltihFokgl4FjrNMttUYpEkUrAsdZpltuiFIkilYBjrdMst0UpEkUqAcdap1lui1IkilQCjrVOs9wWpUgUqQQca51muS1KkShSCTjWOs1yW5QiUaQScKx1muW2KEWiSCXgWOs0y21RikSRSsCx1mmW26IUiSKVgGOt0yy3RSkSRSoBx1qnWW6LUiSKVAKOtU6z3BalSBSpBBxrnWa5Lbp8kf5DWM9kSGCqpAaWIh3DPZIGONY6zXJbdPl7pGMokgY41jrNcluUIlGkEnCsdZrltihFokgl4FjrNMttUYpEkUrAsdZpltuiFIkilYBjrdMst0UpEkUqAcdap1lui1IkilQCjrVOs9wWpUgUqQQca51muS1KkShSCTjWOs1yW5QiUaQScKx1muW2KEWiSCXgWOs0y21RikSRSsCx1mmW26IUiSKVgGOt0yy3RSkSRSoBx1qnWW6LUiSKVAKOtU6z3BalSBSpBBxrnWa5LUqRKFIJONY6zXJblCJRpBJwrHWa5bYoRaJIJeBY6zTLbVGKRJFKwLHWaZbbohSJIpWAY63TLLdFKRJFKgHHWqdZbotSJIpUAo61TrPcFqVIFKkEHGudZrktSpEoUgk41jrNcluUIlGkEnCsdZrltihFokgl4FjrNMttUYpEkUrAsdZpltuiFIkilYBjrdMst0UpEkUqAcdap1lui1IkilQCjrVOs9wWpUgUqQQca51muS1KkShSCTjWOs1yW5QiUaQScKx1muW2KEWiSCXgWOs0y21RikSRSsCx1mmW26IUiSKVgGOt0yy3RSkSRSoBx1qnWW6LUiSKVAKOtU6z3BalSBSpBBxrnWa5LUqRKFIJONY6zXJblCJRpBJwrHWa5bYoRaJIJeBY6zTLbVGKRJFKwLHWaZbbohSJIpWAY63TLLdFKRJFKgHHWqdZbotSJIpUAo61TrPcFqVIFKkEHGudZrktSpEoUgk41jrNcluUIlGkEnCsdZrltihFokgl4FjrNMttUYpEkUrAsdZpltuiFIkilYBjrdMst0UpEkUqAcdap1lui1IkilQCjrVOs9wWpUgUqQQca51muS1KkShSCTjWOs1yW5QiUaQScKx1muW2KEWiSCXgWOs0y21RikSRSsCx1mmW26IUiSKVgGOt0yy3RSkSRSoBx1qnWW6LUiSKVAKOtU6z3BalSBSpBBxrnWa5LUqRKFIJONY6zXJblCJRpBJwrHWa5bYoRaJIJeBY6zTLbVGKRJFKwLHWaZbbogFE+vG1aa4e/+yuvdw1TXP3sj7r6llLs57JkOBYVaBIcxskt9KfL82WrR1fd9e+rs+4et7SrGcyJDhWFSjS3AbJrXTVfPu90elb07T7pO/N1Y+/678/rprHNbx65tKsZzIk53f8Q1CkuQ2SWeix+ba98K2TY6vTev27uVrDq2cuzXomQ4JjVYEizW2QvDp/m+ZvurSR42W/q/nW/EJXz12a9UyGBMeqAkWa2yB5dX4OX6Q9Nuk4wkvzHV09d2nWMxkSHKsKFGlug+TVOdq13DW/d5d+N3fo6rlLs57JkOBYVaBIcxskr87d5pVde/j768/u6lWz39K+0jt99dylWc9kSHCsKlCkuQ2SV6dp0uHvr9urvS3o6rlLs57JkOBYVaBIcxskr07TfLl6+bte/7rrTCoR6T+E9UyGBKZKavABka52R+3ump/cI/kBx6oC90hzGySvTrM/EPerPX5AkbyAY1WBIs1tkLw6R258GR5OOH313KVZz2RIcKwqUKS5DZJX524oEg9/ewHHqgJFmtsgeXUem3TWd7eT+d77yPURXT13adYzGRIcqwoUaW6D5NV5aX7sLv1qT7o7CNKdxXD66rlLs57JkOBYVaBIcxsks9CXtEv62rmRTkv9s33zdPrqmUuznsmQ4FhVoEhzGySz0K/mqhXo19ftu57H/RclvuOrZy7NeiZDcn7HPwRFmtsguZVecr7Jxy/2fRY4VhUo0twGyS719/HLxoz9e56fwy+Tn7561tKsZzIkOFYVKNLcBqnydDlQJA1wrHWa5bYoRaJIJeBY6zTLbVGKRJFKwLHWaZbbohSJIpWAY63TLLdFKRJFKgHHWqdZbotSJIpUAo61TrPcFqVIFKkEHGudZrktSpEoUgk41jrNcluUIlGkEnCsdZrltihFokgl4FjrNMttUYpEkUrAsdZpltuiFIkilYBjrdMst0UpEkUqAcdap1lui1IkilQCjrVOs9wWpUgUqQQca51muS1KkShSCTjWOs1yW5QiUaQScKx1muW2KEWiSCXgWOs0y21RikSRSsCx1mmW26IUiSKVgGOt0yy3RSkSRSoBx1qnWW6LUiSKVAKOtU6z3BalSBSpBBxrnWa5LUqRKFIJONY6zXJblCJRpBJwrHWa5bYoRaJIJeBY6zTLbVGKRJFKwLHWaZbbohSJIpWAY63TLLdFKRJFKgHHWqdZbotSJIpUAo61TrPcFqVIFKkEHGudZrktSpEoUgk41jrNcluUIlGkEnCsdZrltihFokgl4FjrNMttUYpEkUrAsdZpltuiFIkilYBjrdMst0UpEkUqAcdap1lui1IkilQCjrVOs9wWpUgUqQQca51muS1KkShSCTjWOs1yW5QiUaQScKx1muW2KEWiSCXgWOs0y21RikSRSsCx1mmW26IUiSKVgGOt0yy3RSkSRSoBx1qnWW6LUiSKVAKOtU6z3BalSBSpBBxrnWa5LUqRKFIJONY6zXJblCJRpBJwrHWa5bYoRaJIJeBY6zTLbVGKRJFKwLHWaZbbohSJIpWAY63TLLdFKRJFKgHHWqdZbotSJIpUAo61TrPcFqVIFKkEHGudZrktSpEoUgk41jrNcluUIlGkEnCsdZrltihFokgl4FjrNMttUYpEkUrAsdZpltuiFIkilYBjrdMst0UpEkUqAcdap1lui1IkilQCjrVOs9wWpUgUqQQca51muS1KkShSCTjWOs1yW5QiUaQScKx1muW2KEWiSCXgWOs0y21RikSRSsCx1mmW26IUiSKVgGOt0yy3RZcv0n8I65kMCUyV1MBSpGO4R9IAx1qnWW6LLn+PdAxF0gDHWqdZbotSJIpUAo61TrPcFqVIFKkEHGudZrktSpEoUgk41jrNcluUIlGkEnCsdZrltihFokgl4FjrNMttUYpEkUrAsdZpltuiFIkilYBjrdMst0UpEkUqAcdap1lui1IkilQCjrVOs9wWpUgUqQQca51muS1KkShSCTjWOs1yW5QiUaQScKx1muW2KEWiSCXgWOs0y21RikSRSsCx1mmW26IUiSKVgGOt0yy3RSkSRSoBx1qnWW6LUiSKVAKOtU6z3BalSBSpBBxrnWa5LUqRKFIJONY6zXJblCJRpBJwrHWa5bYoRaJIJeBY6zTLbVGKRJFKwLHWaZbbohSJIpWAY63TLLdFKRJFKgHHWqdZbotSJIpUAo61TrPcFqVIFKkEHGudZrktSpEoUgk41jrNcluUIlGkEnCsdZrltihFokgl4FjrNMttUYpEkUrAsdZpltuiFIkilYBjrdMst0UpEkUqAcdap1lui1IkilQCjrVOs9wWpUgUqQQca51muS1KkShSCTjWOs1yW5QiUaQScKx1muW2KEWiSCXgWOs0y21RikSRSsCx1mmW26IUiSKVgGOt0yy3RSkSRSoBx1qnWW6LUiSKVAKOtU6z3BalSBSpBBxrnWa5LUqRKFIJONY6zXJblCJRpBJwrHWa5bYoRaJIJeBY6zTLbVGKRJFKwLHWaZbbohSJIpWAY63TLLdFKRJFKgHHWqdZbotSJIpUAo61TrPcFqVIFKkEHGudZrktSpEoUgk41jrNcluUIlGkEnCsdZrltihFokgl4FjrNMttUYpEkUrAsdZpltuiFIkilYBjrdMst0UpEkUqAcdap1lui1IkilQCjrVOs9wWpUgUqQQca51muS1KkShSCTjWOs1yW5QiUaQScKx1muW2KEWiSCXgWOs0y21RikSRSsCx1mmW26IUiSKVgGOt0yy3Re1FEhnc1sil+moGUCQNcKx1muW2qDuRjq7qQ5E0wLHWaZbbot5EWlGkEOBY6zTLbVFLkW5kkosqK5pfmvVMhgTHWqdZbotaivQ+LdLDxMN+Nc3u0std0zR3L+uzrp61NOuZDMn5Hf8QFGlug/SvPExodDnl0d+rJNLXZsvXc66etzTrmQzJ+R3/EBRpboOMbznjTVG7n+kufG+ufvxd//1x1Tziq2cuzXomQ4JjVYEizW2Q8S1YpJ/N151ITfOn+//v5gpfPXNp1jMZEhyrChRpboMUFPtzdfVnK9LLflfzrfmFrp67NOuZDAmOVQWKNLdBCop9bX6utyI9Nuk4wkvzHV09d2nWMxkSHKsKFGlug0zc9np7MTjecLT5R3O33ol01/ze3fi7vfH01XOXZj2TITm/4x+CIs1tkPFNq+MDd8PNf5qrv0mkq2Z/c/s26PTVc5dmPZMhOb/jH4IizW2Q0S2voyPgw+1fuvc7W5GanioNupqWgrCeyZDAVEkNTol0234G+zYn3vp78+1gRolIx3CPpAGOVQXukeY2yOiWZvpchh3pSDZF8gWOVQWKNLdBxreIvM8XutodQKBIvsCxqkCR5jbI+JZTH8h+S8ext2Z8GR5OOH313KVZz2RIcKwqUKS5DTK65fKUSE0fHv72A45VBYo0t0FGtzyIrGbrHIn0vfeR6yO6eu7SrGcyJOd3/ENQpLkNMr6pkWb+oN3+TkenCHVnMZy+eu7SrGcyJDhWFSjS3AYZ3/S2MenE8e8tRyet/kkHH05dPXNp1jMZEhyrChRpboOMbwEfyG7ZqfG4/6LEd3z1zKVZz2RIzu/4h6BIcxtkfEuOSPxinxPO7/iHoEhzG2R8S5ZI65/DL5OfvnrW0qxnMiQ4VhUo0twGqfJ0OVAkDXCsdZrltihFokgl4FjrNMttUYpEkUrAsdZpltuiFIkilYBjrdMst0XtRTrvYIMeFEkDHGudZrktSpEoUgk41jrNcluUIlGkEnCsdZrltqi9SD3eHppm/gRWJSiSBjjWOs1yW9SVSN15d/fKSzmGImmAY63TLLdFnYnUfqniVXcpx1AkDXCsdZrltqg3kd5FbnSXcgxF0gDHWqdZbot6E+n0N881oEga4FjrNMttUW8ivVOkEOBY6zTLbVFvIj3wL/aFAMdap1lui/oS6e1+s0O61V7MEIqkAY61TrPcFrUXafR57InfcqcBRdIAx1qnWW6L+hOp9ieyFEkDHGudZrkt6kyki9vKnyJRJB1wrHWa5baovUifDUXSAMdap1lui1IkilQCjrVOs9wWpUgUqQQca51muS1KkShSCTjWOs1yW9SHSK+3l+2hhuv6hxookg441jrNclvUg0jv14fDdtf414B/EIqkAY61TrPcFnUg0lvTPwB+xi/U/xgUSQMca51muS3qQKT2Vd3tc3vpuf17spdVFnSAImmAY63TLLdF7UVa9c9mGFypA0XSAMdap1lui9qLdD34Y8wPm7dJNRZ0gCJpgGOt0yy3Re1FGp6myu8jxQDHWqdZbou6EOnEVX0okgY41jrNclvUhUjcI4UDx1qnWW6L2ovE90gRwbHWaZbbovYijY7aPVVZ0R6KpAGOtU6z3Ba1F2l9MfwcqfKvbKBIKuBY6zTLbVEHIvHMhoDgWOs0y21RByKt3/rn2lX+jQ0USQcca51muS3qQaT1+vW+c+n6nmd/BwHHWqdZbov6EOkzoUga4FjrNMttUYpEkUrAsdZpltuiPkRabQ/VNTfV/zoSRdIBx1qnWW6LehDp/XJ3NkN78Jtf7AsBjrVOs9wWdSBSd/h7u5GHv6OAY63TLLdFHYi08ehmd9B7dbExqcqCDlAkDXCsdZrltqi9SMOz6y4GZ97VgCJpgGOt0yy3Re1F2rxB6r2aW1X/rjlF0gDHWqdZbovai8TvI0UEx1qnWW6LUiSKVAKOtU6z3Ba1F+li8NLuvfrRBoqkAY61TrPcFrUX6XbwN/oe+FfNQ4BjrdMst0XtRXrtf7FvcKUOFEkDHGudZrktai9S+11z/oLIaOBY6zTLbVEHIr0Pv9hX+wtJFEkDHGudZrkt6kCk7T4pfbGvynL6UCQNcKx1muW2qAuR1m/31w2/2BcJHGudZrkt6kOkz4QiaYBjrdMst0UpEkUqAcdap1lui1IkilQCjrVOs9wWpUgUqQQca51muS1KkShSCTjWOs1yW5QiUaQScKx1muW2KEWiSCXgWOs0y21RikSRSsCx1mmW26IUiSKVgGOt0yy3RSkSRSoBx1qnWW6LUiSKVAKOtU6z3BalSBSpBBxrnWa5LUqRKFIJONY6zXJbdPki/YewnsmQwFRJDSxFOoZ7JA1wrHWa5bbo8vdIx1AkDXCsdZrltihFokgl4FjrNMttUYpEkUrAsdZpltuiFIkilYBjrdMst0UpEkUqAcdap1lui1IkilQCjrVOs9wWpUgUqQQca51muS1KkShSCTjWOs1yW5QiUaQScKx1muW2KEWiSCXgWOs0y21RikSRSsCx1mmW26IUiSKVgGOt0yy3RSkSRSoBx1qnWW6LUiSKVAKOtU6z3BalSBSpBBxrnWa5LUqRKFIJONY6zXJblCJRpBJwrHWa5bYoRaJIJeBY6zTLbVGKRJFKwLHWaZbbohSJIpWAY63TLLdFKRJFKgHHWqdZbotSJIpUAo61TrPcFqVIFKkEHGudZrktSpEoUgk41jrNcluUIlGkEnCsdZrltihFokgl4FjrNMttUYpEkUrAsdZpltuiFIkilYBjrdMst0UpEkUqAcdap1lui1IkilQCjrVOs9wWpUgUqQQc6/+RfHCsCckefG0okgY4VuuZDAmONSHZg68NRdIAx2o9kyHBsSYke/C1oUga4FitZzIkONaEZA++NhRJAxyr9UyGBMeakOzB14YiaYBjtZ7JkOBYE5I9+NpQJA1wrNYzGRIca0KyB18biqQBjtV6JkOCY01I9uBrQ5E0wLFaz2RIcKwJyR58bSiSBjhW65kMCY41IdmDrw1F0gDHaj2TIcGxJiR78LWhSBrgWK1nMiQ41oRkD742FEkDHKv1TIYEx5qQ7MHXhiJpgGO1nsmQ4FgTkj342lAkDXCs1jMZEhxrQrIHXxuKpAGO1XomQ4JjTUj24GtDkTTAsVrPZEhwrAnJHnxtKJIGOFbrmQwJjjUh2YOvDUXSAMdqPZMhwbEmJHvwtaFIGuBYrWcyJDjWhGQPvjYUSQMcq/VMhgTHmpDswdeGImmAY7WeyZDgWBOSPfjaUCQNcKzWMxkSHGtCsgdfG4qkAY7VeiZDgmNNSPbga0ORNMCxWs9kSHCsCckefG0okgY4VuuZDAmONSHZg68NRdIAx2o9kyHBsSYke/C1oUga4FitZzIkONaEZA++NhRJAxyr9UyGBMeakOzB14YiaYBjtZ7JkOBYE5I9+NpQJA1wrNYzGRIca0KyB18biqQBjtV6JkOCY01I9uBrQ5E0wLFaz2RIcKwJyR58bSiSBjhW65kMCY41IdmDrw1F0gDHaj2TIcGxJiR78LWhSBrgWK1nMiQ41oRkD742FEkDHKv1TIYEx5qQ7MHXhiJpgGO1nsmQ4FgTkjv3fx+/NM2Xx7/p+std0zR3L+ddnYIiaYBjtZ7JkOBYE3Jqxid4abZc7SlTqX4AABkmSURBVOT4urv+9Zyrk1AkDXCs1jMZEhxrQk6M+AS/m+bxz3r956656vZJ35urH3/Xf39cNY/46jQUSQMcq/VMhgTHmpD5CZ/iW/MzXejcaJo/3dXfzRW+Og1F0gDHaj2TIcGxJmR+wqe4anYX/nRuvOx3Nd+aX+jqDBRJAxyr9UyGBMeakNkBBzStUo9NOo7w0nxHV2egSBrgWK1nMiQ41oTMDjigE+mu+b27+ru5Q1dnoEga4FitZzIkONaEzA74aX4139a9V3obsa7Q1RkokgY4VuuZDAmONSGzA36au+5lW9NTpUFXZ6BIGuBYrWcyJDjWhMwO+El+bV+rlYj0H8J6JkMCU/3PeiZDgkL9qEh/rrYfI3GP5AUcq/VMhgTHmpDZAT/Fl90HRBTJCzhW65kMCY41IbMDfoKv6bj2l+HhhNNXZ6BIGuBYrWcyJDjWhMwO+Dx7j3j42w04VuuZDAmONSGzAz7Hn4NH6++9j1wf0dUZKJIGOFbrmQwJjjUhswM+w5+rg0c9QbqzGE5fnYEiaYBjtZ7JkOBYEzI74NO8NFe/e1fTaal/tkcTTl+dhiJpgGO1nsmQ4FgTMj/hU/y5uvrTv/64/6LEd3x1GoqkAY7VeiZDgmNNyPyET/GtOdDdwC/2+QDHaj2TIcGxJuTEiE/QHIu0/jn8Mvnpq1NQJA1wrNYzGRIca0JOzfinQJE0wLFaz2RIcKwJyR58bSiSBjhW65kMCY41IdmDrw1F0gDHaj2TIcGxJiR78LWhSBrgWK1nMiQ41oRkD742FEkDHKv1TIYEx5qQ7MHXhiJpgGO1nsmQ4FgTkj342lAkDXCs1jMZEhxrQrIHXxuKpAGO1XomQ4JjTUj24GtDkTTAsVrPZEhwrAnJHnxtKJIGOFbrmQwJjjUh2YOvDUXSAMdqPZMhwbEmJHvwtaFIGuBYrWcyJDjWhGQPvjYUSQMcq/VMhgTHmpDswdeGImmAY7WeyZDgWBOSPfjaUCQNcKzWMxkSHGtCsgdfG4qkAY7VeiZDgmNNSPbga0ORNMCxWs9kSHCsCckefG0okgY4VuuZDAmONSHZg68NRdIAx2o9kyHBsSYke/C1oUga4FitZzIkONaEZA++NhRJAxyr9UyGBMeakOzB14YiaYBjtZ7JkOBYE5I9+NpQJA1wrNYzGRIca0KyB18biqQBjtV6JkOCY01I9uBrQ5E0wLFaz2RIcKwJyR58bSiSBjhW65kMCY41IdmDrw1F0gDHaj2TIcGxJiR78LWhSBrgWK1nMiQ41oRkD742FEkDHKv1TIYEx5qQ7MHXhiJpgGO1nsmQ4FgTkj342lAkDXCs1jMZEhxrQrIHXxuKpAGO1XomQ4JjTUj24GtDkTTAsVrPZEhwrAnJHnxtKJIGOFbrmQwJjjUh2YOvDUXSAMdqPZMhwbEmJHvwtaFIGuBYrWcyJDjWhGQPvjYUSQMcq/VMhgTHmpDswdeGImmAY7WeyZDgWBOSPfjaUCQNcKzWMxkSHGtCsgdfG4qkAY7VeiZDgmNNSPbga0ORNMCxWs9kSHCsCckefG0okgY4VuuZDAmONSHZg68NRdIAx2o9kyHBsSYke/C1oUga4FitZzIkONaEZA++NhRJAxyr9UyGBMeakOzB14YiaYBjtZ7JkOBYE5I9+NpQJA1wrNYzGRIca0KyB18biqQBjtV6JkOCY01I9uBrQ5E0wLFaz2RIcKwJyR58bSiSBjhW65kMCY41IdmDrw1F0gDHaj2TIcGxJiR78LWhSBrgWK1nMiQ41oRkD742FEkDHKv1TIYEx5qQ7MHXhiJpgGO1nsmQ4FgTkj342lAkDXCs1jMZEhxrQrIHXxuKpAGO1XomQ4JjTUj24GtDkTTAsVrPZEhwrAnJHnxtKJIGOFbrmQwJjjUh2YOvDUXSAMdqPZMhwbEmJHvwtaFIGuBYrWcyJDjWhGQPvjYUSQMcq/VMhgTHmpDswdeGImmAY7WeyZDgWBOSPfjaUCQNcKzWMxkSHGtCsgdfG4qkAY7VeiZDgmNNSPbga0ORNMCxWs9kSHCsCckefG0okgY4VuuZDAmONSHZg68NRdIAx2o9kyHBsSYke/C1oUga4FitZzIkONaEZA/+R/kPYT2TIYGp/mc9kyFBoVqKdAz3SBrgWK1nMiQ41oRkD742FEkDHKv1TIYEx5qQ7MHXhiJpgGO1nsmQ4FgTkj342lAkDXCs1jMZEhxrQrIHXxuKpAGO1XomQ4JjTUj24GtDkTTAsVrPZEhwrAnJHnxtKJIGOFbrmQwJjjUh2YOvDUXSAMdqPZMhwbEmJHvwtaFIGuBYrWcyJDjWhGQPvjYUSQMcq/VMhgTHmpDswdeGImmAY7WeyZDgWBOSPfjaUCQNcKzWMxkSHGtCsgdfG4qkAY7VeiZDgmNNSPbga0ORNMCxWs9kSHCsCckefG0okgY4VuuZDAmONSHZg68NRdIAx2o9kyHBsSYke/C1oUga4FitZzIkONaEZA++NhRJAxyr9UyGBMeakOzB14YiaYBjtZ7JkOBYE5I9+NpQJA1wrNYzGRIca0KyB18biqQBjtV6JkOCY01I9uBrQ5E0wLFaz2RIcKwJyR58bSiSBjhW65kMCY41IdmDrw1F0gDHaj2TIcGxJiR78LWhSBrgWK1nMiQ41oRkD742FEkDHKv1TIYEx5qQ7MHXhiJpgGO1nsmQ4FgTkj342lAkDXCs1jMZEhxrQrIHXxuKpAGO1XomQ4JjTUj24GtDkTTAsVrPZEhwrAnJHnxtKJIGOFbrmQwJjjUh2YOvDUXSAMdqPZMhwbEmJHvwtaFIGuBYrWcyJDjWhGQPvjYUSQMcq/VMhgTHmpDswdeGImmAY7WeyZDgWBOSPfjaUCQNcKzWMxkSHGtCsgdfG4qkAY7VeiZDgmNNSPbga0ORNMCxWs9kSHCsCckefG0okgY4VuuZDAmONSHZg68NRdIAx2o9kyHBsSYke/C1oUga4FitZzIkONaEZA++NhRJAxyr9UyGBMeakOzB14YiaYBjtZ7JkOBYE5I9+NpQJA1wrNYzGRIca0KyB18biqQBjtV6JkOCY01I9uBrQ5E0wLFaz2RIcKwJyR58bSiSBjhW65kMCY41IdmDrw1F0gDHaj2TIcGxJiR78LWhSBrgWK1nMiQ41oRkD742FEkDHKv1TIYEx5qQ7MHXhiJpgGO1nsmQ4FgTkj342lAkDXCs1jMZEhxrQrIHXxuKpAGO1XomQ4JjTUj24GtDkTTAsVrPZEhwrAnJHnxtKJIGOFbrmQwJjjUh2YOvDUXSAMdqPZMhwbEmJHvwtaFIGuBYrWcyJDjWhGQPvjYUSQMcq/VMhgTHmpDswdeGImmAY7WeyZDgWBOSPfjaUCQNcKzWMxkSHGtCsgdfG4qkAY7VeiZDgmNNSPbga0ORNMCxWs9kSHCsCckefG0okgY4VuuZDAmONSHZg68NRdIAx2o9kyHBsSYke/C1oUga4FitZzIkONaEZA++NhRJAxyr9UyGBMeakOzB14YiaYBjtZ7JkOBYE5I9+NpQJA1wrNYzGRIca0KyB18biqQBjtV6JkOCY01I9uBn8XLXNM3dy6m7UCQNcKzWMxkSHGtCTs34h/nabPl64j4USQMcq/VMhgTHmpATI/5hvjdXP/6u//64ah7n70SRNMCxWs9kSHCsCZmf8I/TNH+6//9urubvRJE0wLFaz2RIcKwJmZ/wD/Oy3xF9a37N3osiaYBjtZ7JkOBYEzI74B/nsUlHGV6a77P3okga4FitZzIkONaEzA74x7lrfu8u/W7uZu9FkTTAsVrPZEhwrAmZHfCPc9XsL554k0SRNMCxWs9kSHCsCZkd8I/TNFMX/yNkKViKdMys4x+hSlFWZbMc7ZGOYW/+9aqBlkqRWNVv1UBLtRHpS9nBBg0W0Zt/pWqgpYY6/K3BInrzr1QNtFQbkb73PpCdP9mOvfnXqwZaqvUpQodzHMawN/961UBLNT5p9c+JYw3szT9fNdBSjUR63H+NYv5UO/bmn68aaKmhvtinwSJ6869UDbRUs6+a/yz4qrkGi+jNv1I10FJD/fITt0VZlc2iSKzqtmqgpVIkVvVbNdBSKRKr+q0aaKkUiVX9Vg20VIrEqn6rBloqRWJVv1UDLZUisarfqoGWSpFY1W/VQEulSKzqt2qgpVIkVvVbNdBSKRKr+q0aaKkUiVX9Vg20VIrEqn6rBloqRWJVv1UDLZUisarfqoGWSpFY1W/VQEv1LBIhC0CsF0DIEhDrBRCyBMR6AYQsAbFeACFLQKwXQMgSEOsFELIExHoBhCwBsV4AIUtArBdAyBIQm6d9mfqd4H+bPTarGjG5zNmbbQkSaeL7eEEuY20ZrXUiVvncJe2Y/isVv7x1feaPaZzzNzY+nSCRJl7GC3IZa8t4rROxyueuacv3/d9NGvxFzB/ND4vVzDKzzJmbbQkSaeJlbLbLWFsm1joRq3zOYoakv+T3e/jHzu92N3thZpkzN9sSJNIdPyZ2kS5jXU+vdSJW+ZzVDDj8bdlvza/e7Ve+EpxZ5tzqTQkS6ZbfmxdxX4+H02WsM2udilU+ZTlDHnt/7bz3NzH/Nt8MFjPPzDJnbrYlSKRbNv/Cf18fD6fLWGfWOhWrfM56Btw1v3eXfjd3h5t/NT/bP/B39e335KM+nZllztxsS5BItzR3f9aj4XQZ68xap2KVT13WlqvDuvqvh3/sj4T4eLc5s8yZm20JEumWv+1/jofTZawza52KVT5zVTt66+ovceP4j3bdv+58tH1mmTM32xIk0h7H2bmMdcd47zmOVT53SR0zmV3tXyV/bTy8FFmASN4i7RFZpKlY5RPXk4CZvbh4j7wAkQ74iLRHZJEOHGKVz1nKAJyZi1fJixLJR6Q9liHSIVb5lJVsnzOdU/EFvq90EeXMMvHqDQgSaY/j9biMdccpkdIm+ZSV7J5zJxI+0umi6ws4/N3DRaQ9ghz+7nAm0p7vvc/eJg8m/Wm+fOJy5phZJly9BUEi7XE8nC5j3TEv0iFW+ZylDDgkdfg4e90eC/m7u/TTRZQzy5y52ZYgkfaYP0XIUaw7xkftxrHKJ65nTzo/8c9ghY/7U0O++DjbanqZczfbEiTSA7MnrbqKdcv4dKZxrPKJ6+ktJJ0x3z+r6u/mapvlr69OvpIyvcy5m20JEumBkS4uY90yOtduIlb53CXtGH6HK63zz9XuZi8feUwv0+c30IJEumefpu9YO0ZrnYhVPn1VHT/73yo+CP/jq68zLGeW+dPjd6KDRJoYi+Qz1paJtY5ilc9dEiHLRKwXQMgSEOsFELIExHoBhCwBsV4AIUtArBdAyBIQ6wUQsgTEegGELAGxXgAhS0CsF0DIEhDrBRCyBMR6AUSblcj+4k0jcnGzGmx/O2wnaoj1AogyG09kd/FSdly+9e5wSZEqINYLIMo0e5H2Hm32Softt0KRKiDWCyC6dPZ0lx42F242u6K3m82F/au7W6FINRDrBRBVOk+ku3gh8rS/8WK8nWgi1gsgmqxE0ku3194Luo1T3buk182FS4pUA7FeACmhk2G1ceLyoX/z2+aG9U6UzSu7+3T7/W7n1L7YW1OkGoj1AkgJrQzbV2nS9I7INdKskyibd0bP6fbnVqD2Yc3TmiJVQawXQErYyPCQjsgdTLrsXsHtRLkWeU0bNi/zrtv/376vKVIdxHoBpITOoI0W761O17sbb7cH53aiDHzpX6FINRDrBZASWo+2h+RWsjuQ0B1o2G3r/W9//8nLRAuxXgApYbgf6g4qdAcadtt6/1uvj69QpBqI9QJICb2PWJ93TnUHGnbbev8b3Da6TLQQ6wWQEjYyvPcur9OBhsN1ivS5iPUCSAkjMW4Pu6jdtoYifSZivQBSwkgMOWby8PfosUQLsV4AKWEoUjMl0s1QpJvJxxItxHoBpAQ5WLI92DAW6UFkf/pQ/zJFqoJYL4CUILvPjNbdu6On423t/96GJ62+Hm8nqoj1AkgJst8lrfoH8NK27v8X+93Q/eCbfRSpBmK9AFJC9+rtYXeK0P1oW/f/wRf7HsbbiSZivQBSgmy/V9RxOd62vXBxeMd0ObWdKCLWCyAltK/ndiZdT2zbXni7mHSNItVArBdASuhkeLiQ5uZ5etuW7tdxNUe/josi1UCsF0BKoAzeEOsFkBIokjfEegGkBIrkDbFeACmBInlDrBdASqBI3hDrBZASKJI3xHoBpASK5A2xXgAhS0CsF0DIEhDrBRCyBMR6AYQsAbFeACFLQKwXQMgSEOsFELIExHoBhCwBsV4AIUtArBdAyBIQ6wUQsgTEegGELAGxXgAhS0CsF0DIEhDrBRCyBMR6AYQsAbFeACFLQKwXQMgSEOsFELIExHoBhCwBsV4AIUtArBdAyBIQ6wUQsgTEegGELAGxXgAhS0CsF0DIEhDrBRCyBMR6AcSE/82TX+z/5tFfuVPEegHEBIqkjFgvgJhAkZQR6wUQEyiSMmK9AGICRVJGrBcQkbebRuRidc5d32/OrLmSa3gP2V/sFnAzXMHbYfuNoMVli7S6FpHr6bJFIt3L/LaAiPUCAvIqHeeI9HDuX9Z7a+Qd3OPwV/ouZcflW+8Ol4fnepemv2WCXJHSU15ObSwRaXVuMkEQ6wUEZDdUr2fc9ew/UXkpD+Aezb7W3qPNXumw/bb/XA/TE38gU6R7aR7e1+8PjdxObC0QabW0P94p1gsIyGYG0NT37nrW/Z6kAffo7OkubfZycvPWvsDs7xZvZfBcjTydLJcpksh2D/c6uc58kR6EIpGMGTj3rg1Ss/NkW+tCkiS3h11Sb3vHAzAzT6TVfkd0I8/jzbkivW7+VbikSP88+iKhsW9fCaWXbq+9F3QXuz3F60U3mf3nAmrmiXS73/Ot5H68OVekzUrv1xTpHye9PemurG7b+b24Pbxfer1th/phfNej417tzc8Xm4e2V/ZvPbo7ry73FXa8te/yd4Ue5DDL97udU/ti70ja29Nu5ol0vX9D+Dp1cDFbpOu3NUX61+nZ8XZ4278T4f16d707atYX6e1ieKRtc6k9Dift4fEnSS+Y2jvf9irsaFordoVu5PDq6nn7+LU0T8d7v2c5+S4pT6TmUHnKz1yR3rc/6onlxUOsFxCOnh3N4fDZ7rXP4ZbmfejccEu37To97mI/nZsbHvb325t02b2C24ly3TteuHmZ1+0gbt/Xo5eRTf+Q3og8kXqVp+Y//2DDTKHAiPUCApImdjPyzaod4dXF7gOWVo3b9oDa9e6G/XC3W7oDyLKb/f5ubLdf2d0qt/37rbsDCatDrYEvMhjx3obuGU98MkWRlBHrBQQkTezFft/wvtvt7PdM7Quw3vB3n+Fut7QfoLxutyRRng6H01uPnvb3e0uXbntPe65IDydf21EkZcR6AQGR8Qxsb7rf72RatS56d+1tud1elMOc3x5erA33Q91Bhbd0OkGmSM8y+eHpDoqkjFgvICBHE/v6cL076+Bajj9lSXftbXneiiEHe64HIq0O9+ucatIbqEyRXuXU2Xt5Il30nkXhYMNuufPbAiLWCwhIb2K3h7/TEYVmNBzpruOB793Se5j03ths73GZXuHlirS5fuIAeJ5Iyoe/d8ub3xYQsV5AQPYT2zv8fSzH0V1PijR9cXv59rCL2m1rzhfpeDU98kS6730gO/F6kSJRpBL2E7o9pH1x/bCqJpIcM3n4e/TYqesD8kRa9d7hTZz0TpEoUglpQtvD3w9vvZu0RWqmRLoZinSznnrs1PUBeSKt0+vLt8maFIkilZAmdHT4u3+wYfsOJd11eLDhej0Y8+F7pNfB/cYiPfTOPe9fnhBJ7T1Se8LR7msUE6faUaQ1RSphvJt5kOPD36vtviLdZfLwd6o3PGrXu9/wc6DdI96GJ62+Hm9PaB61q/DFPopEeiJtx7j7yuzu/7vBvjjjA9lU7/awu5J9hfZ+wzMTejvC3W7ofvDNviORND9H2vCk/VVzivTP03u9drEx4O2++7f6fb0/Eag7Z+hid9dVuu/oFKFUbzU8s2F/v6NXUekRgy/2PYy373g4+WX4bJFOUSTSwhDrBQQkTexq8Oal25H0Tk3t3p13r4jatyrvx1t6Y/9+eBEm2+8VTb6K2j/iYl9qcJ8jkTTPtQNQJIpUwn5ib5Maq/St1fekwe7U7e2p3O2lia9R7Asevi3a7tl2JUZvcPaP2JcaunYk0sz7GVIHsV5AQA4Tu/2u3tP+vJ/17jdlHb6W93Cx3/I0+mJfYvh9pPYxzc34G939F4PtkzRHv45rKNLzeb/miCgh1gsg6/YV4e7jIFF7C34Df50K0USsF0DW3SvA7fsZNZHez/9NR0QDsV4AaUm/qkRNJPjrVIguYr0A0pJ+r52aSOj32hFlxHoBpGP3m1a1RIK/aZUoI9YLIB1v29+JoiTSeyPgd38TZcR6AWTLanBu3gfBf42CKCPWCyBkCYj1AghZAmK9AEKWgFgvgJAlINYLIGQJiPUCCFkCYr0AQpaAWC+AkCUg1gsgZAmI9QIIWQJivQBCloBYL4CQJSDWCyBkCYj1AghZAmK9AEKWwP8D239MrUvRtJgAAAAASUVORK5CYII=", "text/plain": [ "plot without title" ] }, "metadata": { "image/png": { "height": 420, "width": 420 } }, "output_type": "display_data" } ], "source": [ "hist_p401 = ggplot(data, aes(x = p401, fill = factor(p401))) +\n", " geom_bar() + theme_minimal() + \n", " ggtitle(\"Participation, 401(k)\") +\n", " theme(legend.position = \"bottom\", plot.title = element_text(hjust = 0.5),\n", " text = element_text(size = 20)) \n", "hist_p401" ] }, { "attachments": {}, "cell_type": "markdown", "id": "24774374", "metadata": { "papermill": { "duration": 0.039347, "end_time": "2021-03-23T13:44:32.932448", "exception": false, "start_time": "2021-03-23T13:44:32.893101", "status": "completed" }, "tags": [] }, "source": [ "Eligibility is highly associated with financial wealth:" ] }, { "cell_type": "code", "execution_count": 4, "id": "512a2b1c", "metadata": { "execution": { "iopub.execute_input": "2021-03-23T13:44:33.016258Z", "iopub.status.busy": "2021-03-23T13:44:33.013504Z", "iopub.status.idle": "2021-03-23T13:44:33.681630Z", "shell.execute_reply": "2021-03-23T13:44:33.681007Z" }, "papermill": { "duration": 0.711317, "end_time": "2021-03-23T13:44:33.681783", "exception": false, "start_time": "2021-03-23T13:44:32.970466", "status": "completed" }, "tags": [], "vscode": { "languageId": "r" } }, "outputs": [ { "name": "stderr", "output_type": "stream", "text": [ "Warning message:\n", "\"\u001b[1m\u001b[22mRemoved 340 rows containing non-finite values (`stat_density()`).\"\n" ] }, { "data": { "image/png": "iVBORw0KGgoAAAANSUhEUgAAA0gAAANICAMAAADKOT/pAAAAilBMVEUAAAAAv8QaGhpNTU1fX19oaGh3d3d8fHyDg4OJiYmMjIyVlZWYmJiampqjo6OlpaWnp6evr6+wsLCysrK5ubm6urq9vb3BwcHExMTHx8fJycnMzMzQ0NDR0dHV1dXY2NjZ2dnc3Nze3t7h4eHk5OTp6enq6urr6+vv7+/w8PDy8vL19fX4dm3////0e5y5AAAACXBIWXMAABJ0AAASdAHeZh94AAAgAElEQVR4nO2dbUPbSrJuRTQcGLLJgSEbJhyYcHlJ4gH//793/SqpJVmusrrpKnk9H3aQWC53lb22bWHZxZwQMjpF7gUQMoUgEiERgkiERAgiERIhiERIhCASIRGCSIRECCIREiGIREiEIBIhEYJIhEQIIhESIYhESIQgEiERgkiERAgiERIhiERIhCASIRGCSIRECCIREiGIdHD+89eXL1/++k/uZUwyf3/JvQJtEOnQ/PPLOv/MvZAJ5j9fEOlY8veXf/z7v/P//vsfX/6VeymTy8IjRDqWfPnye/Xv65d/ZF7J5PLvL4h0NPlP9UD0v1/+X9aVTC2vi+fM/0SkY8m/vmyPMvzny99ZVzK1LB6N/p4j0rHkry+vm59ev/yVdSVTy5e/fs8R6Wjyj/qW5kVS1Px3+R9EOpY0bml/N7r9+JspIh0WREoafzNFpMOCSEnjb6aIdFgQKWn8zRSRDsv/cLAhZRDpWMLh76RBpGPJ340/yPJmu+hBpGNJrU/9HgcSLYh0NNm+afW3v9vcQfwNFZEOzL+q0yh4q138INLxhBP7EgaRjij/x6nmyYJIhBxlEImQCEEkQiIEkQiJEEQiJEIQiZAIQSRCIgSRCIkQRCIkQkyL9HvicLZV2Gjfx6yEMCJlhBHJQGFE8g8jkoHCiOQfRiQDhRHJP4xIBgojkn8YkQwURiT/MCIZKIxI/mFEMlAYkfzDiGSgMCL5hxHJQGFE8g8jkoHCiOQfRiQDhRHJP4xIBgojkn8YkQwURiT/MCIZKIxI/mFEMlAYkfzDiGSgMCL5hxHJQGFE8g8jkoHCiOQfRiQDhRHJP4xIBgojkn8YkQwURiT/MCIZKIxI/mFEMlAYkfzDiGSgMCL5hxHJQGFE8g8jkoHCiOQfRiQDhRHJP4xIBgojUisfySpP785ho30fsxLCiJQRRiQDhRGpFUSSwzba9zErIYxIGWFEMlAYkVpBJDlso30fsxLCiJQRRiQDhRGpFUSSwzba9zErIYxIGWFEMlAYkVpBJDlso30fsxLCiJQRRiQDhRGpFUSSwzba9zErIYxIGWFEMlAYkVpBJDlso30fsxLCiJQRRiQDhRGpFUSSwzba9zErIYxIGWFEMlAYkVpBJDlso30fsxLCkxHpA5HksI32fcxKCCNSRhiRDBRGpDCIpIBttO9jVkIYkTLCiGSgMCKFQSQFbKN9H7MSwoiUEUYkA4URKQwiKWAb7fuYlRBGpIwwIhkojEhhEEkB22jfx6yEsFqk5+uyLK+f9+we3lzmvtz88F5WEa+6J4ikgG2072NWQlgr0rfNff7b4O7hzWWeK21eEenTYRvt+5iVEFaKdF+eP77P3x/Py7uB3cObyzzX2jyWj+pV9wSRFLCN9n3MSggrRSrLP6t/f5XnA7uHN+dLeWqRrje/1ay6J4ikgG2072NWQlgn0nP1mPK9fN25e3hzIdTiqd63SqTzQEnZqnuCSArYRvs+ZiWEdSLdldsDBs/l/c7dw5vLR6jFv1uR3svv+lX3BJEUsI32fcxKCOtEui5/bX76VV7v3D28uRDp+s+8Eum1/PnzuizPv/+ad4JIiWAb7fuYlRDWiXReH1hrvkhq7R7eXDwIrX7c7H2sjtkFhy+GV90TRFLANtr3MSshrBOpcYS6ebC6tXt4s/Xj4tHocWnW63XXJERKBNto38eshHB2kc6rV1DfyvrZ3W91Pj70lzmazEcMlgzEkEh1njuHHXhESgTbaN/HrISwIZHmZftAOCIlgm2072NWQlgn0tf+gw2t3cOb2x97RGrvQqREsI32fcxKCOc4/L0KIiGSi1kJYZ1I942/rd7t3D28uUlXpD/lV+mqe4JICthG+z5mJYQPfYtQ/XaF7u7hzU3qo3bvmz0/O8e/ESkRbKN9H7MSwge+afVP+IDS2j28ub3I+t+76r1GX5tv3xtedU8QSQHbaN/HrISwUqS76oyI+4Hdw5vrVO+1W+xdevb6rX2OEyIlg22072NWQnjciX3VQ4z6xL76on/ON7/tvnkVkRLBNtr3MSshrD7V/GfzrPH6udrP8GTy4c3wovPHb7xp9XNhG+37mJUQ5sNPMsKIZKAwIoVBJAVso30fsxLCExJJY5KNmwWRDBRGpDCIpIBttO9jVkIYkTLCiGSgMCKFQSQFbKN9H7MSwoiUEUYkA4URKQwiKWAb7fuYlRBGpIwwIhkojEhhEEkB22jfx6yEMCJlhBHJQGFECoNICthG+z5mJYQRKSOMSAYKI1IYRFLANtr3MSshjEgZYUQyUBiRwiCSArbRvo9ZCWFEyggjkoHCiBQGkRSwjfZ9zEoII1JGGJEMFEakMIikgG2072NWQhiRMsKIZKAwIoVBJAVso30fsxLCiJQRRiQDhREpDCIpYBvt+5iVEEakjDAiGSiMSGEQSQHbaN/HrIQwImWEEclAYUQKg0gK2Eb7PmYlhBEpI4xIBgojUhhEUsA22vcxKyGMSBlhRDJQGJHCIJICttG+j1kJYUTKCCOSgcKIFAaRFLCN9n3MSggjUkYYkQwURqQwiKSAbbTvY1ZCGJEywohkoDAihUEkBWyjfR+zEsKIlBFGJAOFESkMIilgG+37mJUQRqSMMCIZKIxIYRBJAdto38eshDAiZYQRyUBhRAqDSArYRvs+ZiWEESkjjEgGCiNSGERSwDba9zErIYxIGWFEMlAYkcIgkgK20b6PWQlhRMoII5KBwogUBpEUsI32fcxKCCNSRhiRDBRGpDCIpIBttO9jVkIYkTLCiGSgMCKFQSQFbKN9H7MSwoiUEUYkA4URKQwiKWAb7fuYlRBGpIwwIhkojEhhEEkB22jfx6yEMCJlhBHJQGFECoNICthG+z5mJYQRKSOMSAYKI1IYRFLANtr3MSshjEgZYUQyUBiRwiCSArbRvo9ZCWFEyggjkoHCiBQGkRSwjfZ9zEoII1JGGJEMFEakMIikgG2072NWQhiRMsKIZKAwIoVBJAVso30fsxLCiJQRRiQDhREpDCIpYBvt+5iVEEakjDAiGSiMSGEQSQHbaN/HrIQwImWEEclAYUQKg0gK2Eb7PmYlhBEpI4xIBgojUhhEUsA22vcxKyE8JZEUJtm4WRDJQGFECoNICthG+z5mJYSNivRbnaVI+ksdScYMlgykHqxRkdbhESkRbKN9H7MSwoiUEUYkA4URKQwiKWAb7fuYlRBGpIwwIhkojEhhEEkB22jfx6yEMCJlhBHJQGFECoNICthG+z5mJYQRKSOMSAYKI1IYRFLANtr3MSshjEgZYUQyUBiRwiCSArbRvo9ZCWFEyggjkoHCiBQGkRSwjfZ9zEoII1JGGJEMFEakMIikgG2072NWQhiRMsKIZKAwIoVBJAVso30fsxLCiJQRRiQDhREpDCIpYBvt+5iVEEakjDAiGSiMSGEQSQHbaN/HrIQwImWEEclAYUQK8oFICthG+z5mJYQRKSOMSAYKI1IQRNLANtr3MSshjEgZYUQyUBiRgiCSBrbRvo9ZCWFEyggjkoHCiBQEkTSwjfZ9zEoII1JGGJEMFEakIIikgW2072NWQhiRMsKIZKAwIgVBJA1so30fsxLCiJQRRiQDhREpCCJpYBvt+5iVEEakjDAiGSiMSEEQSQPbaN/HrIQwImWEEclAYUQKgkga2Eb7PmYlhBEpI4xIBgojUhBE0sA22vcxKyGMSBlhRDJQGJGCIJIGttG+j1kJYUTKCCOSgcKIFASRNLCN9n3MSggjUkYYkQwURqQgiKSBbbTvY1ZCGJEywohkoDAiBUEkDWyjfR+zEsKIlBFGJAOFESkIImlgG+37mJUQRqSMMCIZKIxIQRBJA9to38eshDAiZYQRyUBhRAqCSBrYRvs+ZiWEESkjjEgGCiNSEETSwDba9zErIYxIGWFEMlAYkYIgkga20b6PWQlhRMoII5KBwogUBJE0sI32fcxKCCNSRhiRDBRGpCCIpIFttO9jVkIYkTLCiGSgMCIFQSQNbKN9H7MSwoiUEUYkA4URKQgiaWAb7fuYlRBGpIwwIhkojEhBEEkD22jfx6yEMCJlhBHJQGFECoJIGthG+z5mJYQRKSOMSAYKI1IQRNLANtr3MSshjEgZYUQyUBiRgiCSBrbRvo9ZCWFEyggjkoHCiBQEkTSwjfZ9zEoII1JGGJEMFEakIIikgW2072NWQhiRMsKIZKAwIgVBJA1so30fsxLCapGer8uyvH7es3t4c5n7cl9JREoG22jfx6yEsFakb+U63wZ3D28u81yWOy4rWXU3iKSBbbTvY1ZCWCnSfXn++D5/fzwv7wZ2D28us/CoHC45uOpuEEkD22jfx6yEsFKksvyz+vdXeT6we3hzkceyFmlHycFVd4NIGthG+z5mJYR1Ij1Xjxrfy9edu4c3F8osnsx924q0o+TwqrtBJA1so30fsxLCOpHuyu0hgefyfufu4c3lY9Di361IO0oOr7obRNLANtr3MSshrBPpuvy1+elXeb1z9/DmQqTrP/NKpB0lh1fdDSJpYBvt+5iVENaJdF4fs26+omntHt6cz99XP5aDJVdBpESwjfZ9zEoI60Qqy74f27uHN1s/7ii5CiIlgm2072NWQtioSL+1WYmkvtSxZH74YMlQzIu0udXlK+MRSQPbaN/HrIQwImWEEclA4Swife0/MtDaPby5/bEcLLkKIiWCbbTvY1ZCOMfh71U4/I1ILmYlhHUi3Tf+enq3c/fw5iZbkXaUHF51N4ikgW2072NWQvjQtwjVb0jo7h7e3KT7FqGg5PCqu0EkDWyjfR+zEsIHvmn1T3hgoLV7eHN7keGSg6vuBpE0sI32fcxKCCtFuqvOebgf2D28uU6lzY6Sg6vuBpE0sI32fcxKCI87sa+yQX1iX+NYNyf2fTpso30fsxLC6lPNfzbPC69t+BmeLj68GV6077d7Vt0NImlgG+37mJUQ5sNPMsKIZKAwIgVBJA1so30fsxLCiJQRRiQDhREpCCJpYBvt+5iVEJ6ISB+IpIFttO9jVkIYkTLCiGSgMCI1g0gq2Eb7PmYlhBEpI4xIBgojUjOIpIJttO9jVkIYkTLCiGSgMCI1g0gq2Eb7PmYlhBEpI4xIBgojUjOIpIJttO9jVkK4K9LpzZumcsogUiLYRvs+ZiWEuyIVRWHFJURKBNto38eshHCvSFZcQqREsI32fcxKCHdFmv04s+ISIiWCbbTvY1ZCuPdggxWXECkRbKN9H7MSwruO2plwCZESwTba9zErITxw+Lvh0kxzXfGCSIlgG+37mJUQHv470kO5Vqm4eNJcW6wgUiLYRvs+ZiWEB0R6utpqtMyV5uoiBZESwTba9zErIbxLpMqisx+zt9VzvBvN9cUJIiWCbbTvY1ZCuFekt4ZF6z0/iqLzgcLpg0iJYBvt+5iVEO6K9HZz2rJoxRUZ3pWHSIlgG+37mJUQ3vXOhsCi+Xy22KO5vjhBpESwjfZ9zEoI94rUsihbECkRbKN9H7MSwl2RrFg0R6RksI32fcxKCHM+UkYYkQwUTvnUrrlZ5nhxtAkiJYJttO9jVkJ4r0hZDtdtgkiJYBvt+5iVEN4n0gMiTfDOYaN9H7MSwk1LLovenGquKGoQKRFso30fsxLCTZFm/SLdaq4oahApEWyjfR+zEsLB87bbHo3O8nmESKlgG+37mJUQ3nuwIWcQKRFso30fsxLCiJQRRiQDhZOJZCiIlAi20b6PWQlhRMoII5KBwglEWj+p6x5v0FxR1CBSIthG+z5mJYQRKSOMSAYKI1IziKSCbbTvY1ZCmNdIGWFEMlAYkZpBJBVso30fsxLCiJQRRiQDhT9BpKuyOM34BiFESgbbaN/HrIRwn0izq9UbvtcfWHymuZrIQaREsI32fcxKCPeI9FauDtRt38F6obmeuEGkRLCN9n3MSgj3iFSuj3iXy0ejl8V/XzRXFDWIlAi20b6PWQnhrkgPRVG+LR6XFjo9rR6Xcnzq9zqIlAi20b6PWQnhrkiXRfGw+Odm/bg04wzZCd45bLTvY1ZCuOdbzYti+cF2Z5vjDLyzYYJ3Dhvt+5iVEN55PlKxeU6HSBO8c9ho38eshPAukV6K9TM8RJrincNG+z5mJYR3ibQ8+L18hveU8y9JiJQIttG+j1kJ4a5IF6ujdduXSBdZvmFsE0RKBNto38eshHBXpB9LhZ7WH8M1W37UHd9qngpGJAOFk4k0335z7Gx9blK+ByRESgXbaN/HrIRwj0hPa4+WhxryeoRIqWAb7fuYlRDuOyT3tnhCd/a0/On0Kt/7g+aIlAy20b6PWQlhzkfKCCOSgcKI1AwiqWAb7fuYlRBGpIwwIhkonFCkl6tTPkXoM2BEMlA4nUgPfBzXJ8GIZKBwMpFe+Fy7z4IRyUDhZCJdLd/UkO/dDM0gUiLYRvs+ZiWEuyKVOb+jL4xWJLlJNm4WRDJQOJlIm7d9WwgiJYJttO9jVkJ4Ul80hkhC2Eb7PmYlhLvSnCHSZ8GIZKBwMpFuN6fGGggiJYJttO9jVkK49zSK0sZBO0RKBdto38eshHD/J62WNo5/I1Ii2Eb7PmYlhHsPNnj9gywiCWEb7fuYlRBGpIwwIhkojEjNIJIKttG+j1kJYTOHuvuCSIlgG+37mJUQRqSMMCIZKIxIzSCSCrbRvo9ZCWGjIv1WZiOS9mLHksMHSwZTD7ZfpLebi/XX9l0+pRVmODwiJYJttO9jVkK4V6SL6mhdzq8ZQ6RksI32fcxKCO/4DtlapJwmIVIi2Eb7PmYlhHtEOiuK04fN2RTLz2/w8x2yiCSEbbTvY1ZCuPc7ZJdfdrn5Q+yVp++QRSQhbKN9H7MSwr1f67L52O/l1pun75BFJCFso30fsxLCQ1992dzMEkRKBNto38eshDAiZYQRyUDhTxJpVhSl5oqiBpESwTba9zErIdwV6TJ4jXRbFJeaK4oaREoE22jfx6yEcO9Ru3I2bxz+zvcJDoiUCLbRvo9ZCeH+vyOV678jvSw/dfVMcz1xg0iJYBvt+5iVEO4RaRZ8F0XOD0JBpESwjfZ9zEoI9x6Su6w9usj5qauIlAi20b6PWQnhoXd/Fxc3Wb9CFpFSwTba9zErIWz0fKR1ECkRbKN9H7MSwoiUEUYkA4URqRlEUsE22vcxKyHcFKn7QVx8HFdSGJEMFEakZhBJBdto38eshDAiZYQRyUDhhK+RLori8mn596PlOxtuNFcTOYiUCLbRvo9ZCeH+twhVb6974y1CU7xz2Gjfx6yEcO8Xjd3u2vrkIFIi2Eb7PmYlhLsinQZfxjzjVPMJ3jlstO9jVkJ475cxc7BhgncOG+37mJUQ3ifSDJEmeOew0b6PWQnh3k8RCl8jXWiuKGoQKRFso30fsxLCXZF+NM9B4gzZlDAiGSic7vD38ry+q9X5E0+XnCGbEkYkA4XTiVR99vcqGT1CpFSwjfZ9zEoI9x1JmF3UHuX8MgpESgXbaN/HrITwzjNklx/XcHGT80RzREoG22jfx6yEMOcjZYQRyUBhRGoGkVSwjfZ9zEoII1JGGJEMFEakZhBJBdto38eshDAiZYQRyUBhRGoGkVSwjfZ9zEoII1JGGJEMFEakZhBJBdto38eshDAiZYQRyUBhRGoGkVSwjfZ9zEoII1JGGJEMFEakZhBJBdto38eshDAiZYQRyUBhRGoGkVSwjfZ9zEoII1JGGJEMFEakRj4QSQXbaN/HrIQwImWEEclAYURqBJF0sI32fcxKCCNSRhiRDBRGpEYQSQfbaN/HrIQwImWEEclAYURqBJF0sI32fcxKCCNSRhiRDBRGpEYQSQfbaN/HrIQwImWEEclAYURqBJF0sI32fcxKCCNSRhiRDBRGpEYQSQfbaN/HrIQwImWEEclAYURqBJF0sI32fcxKCCNSRhiRDBRGpEYQSQfbaN/HrIQwImWEEclAYURqBJF0sI32fcxKCCNSRhiRDBRGpEYQSQfbaN/HrIQwImWEEclAYURqBJF0sI32fcxKCCNSRhiRDBRGpEYQSQfbaN/HrIQwImWEEclAYURqBJF0sI32fcxKCCNSRhiRDBRGpEYQSQfbaN/HrIQwImWEEclAYURqBJF0sI32fcxKCCNSRhiRDBRGpEYQSQfbaN/HrIQwImWEEclA4VwiPV+XZXn9vGe3YvO9rCJedTuIpINttO9jVkJYK9K3zX3+2+BuzeYrIn06bKN9H7MSwkqR7svzx/f5++N5eTewW7X5WD6qV90OIulgG+37mJUQVopUln9W//4qzwd2qzavN5uaVbeDSDrYRvs+ZiWEdSI9Vw9E38vXnbtVm/PzQEnZqttBJB1so30fsxLCOpHuyu0Bg+fyfudu1eZ7+V2/6nYQSQfbaN/HrISwTqTr8tfmp1/l9c7dqs3X8ufP67I8//5r3gkiJYJttO9jVkJYJ9J5fWCt+SKptVu1+VgdswsOXwyvuh1E0sE22vcxKyGsE6lxhLp5sLq1W7W5eDR6fF/8/HrdNOm3LluRlBc7mhw8WDIcQyKdVy+ZvpXtZ3c8IiWCbbTvY1ZCOLtIdZ47hx0QKRFso30fsxLChkSal+0D4YiUCLbRvo9ZCWGdSF/7Dza0dqs2G+m8RwiREsE22vcxKyGc/fB3I+NFEptk42ZBJAOFs4h03/hj6t3O3arNOn/Kr9JVt4NIOthG+z5mJYQPfYtQ/f6E7m7V5nn5vtn82flLEiIlgm2072NWQvjAN63+CZ+GtXZrNu+q9xp9bb59b3jV7SCSDrbRvo9ZCWGlSHfVKRD3A7s1m++Lf5ZivX5rn+OESMlgG+37mJUQHndiX/W4NOLEvj/nm83um1cRKRFso30fsxLC6lPNfzZPE6+f4P0MTyZXbT5+i/WmVUSSwTba9zErITytDz9BJBlso30fsxLCiJQRRiQDhRGpEUTSwTba9zErIYxIGWFEMlAYkRpBJB1so30fsxLCiJQRRiQDhRGpEUTSwTba9zErITwJkT4QSQfbaN/HrIQwImWEEclAYUSqg0hK2Eb7PmYlhBEpI4xIBgojUh1EUsI22vcxKyGMSBlhRDJQGJHqIJISttG+j1kJYUTKCCOSgcKIVAeRlLCN9n3MSggjUkYYkQwURqQ6iKSEbbTvY1ZCGJEywohkoDAi1UEkJWyjfR+zEsKIlBFGJAOFEakOIilhG+37mJUQnphIUpNs3CyIZKAwItVBJCVso30fsxLCiJQRRiQDhRGpygciKWEb7fuYlRBGpIwwIhkojEhVEEkL22jfx6yEMCJlhBHJQGFEqoJIWthG+z5mJYQRKSOMSAYKI1IVRNLCNtr3MSshjEgZYUQyUBiRqiCSFrbRvo9ZCWFEyggjkoHCiFQFkbSwjfZ9zEoII1JGGJEMFEakKoikhW2072NWQhiRMsKIZKAwIlVBJC1so30fsxLCiJQRRiQDhRGpCiJpYRvt+5iVEEakjDAiGSiMSFUQSQvbaN/HrIQwImWEEclAYUSqgkha2Eb7PmYlhBEpI4xIBgojUhVE0sI22vcxKyGMSBlhRDJQGJGqIJIWVtY6OYl1xYfDNm4xRDqscmLYiUgnJ3KTpjcrIYxIGWEfIp2cKEya3qyEMCJlhBEpDYtIrcha/EAkLYxI0WFEygi7EOnkRGPS9GYlhBEpI4xIaVhEauUAkYQm2bhZECkNi0itIFIi+BCRZCZNb1ZCGJEywh5EOkEkEYxIGWFESsMiUiuIlAiW1zpBJBmMSBlhXyKJTJrerIQwImWEESkNi0itIFIi+DCRJCZNb1ZCGJEywoiUhkWkVhApEYxI0WFEygjbF+kEkYQwImWEESkNi0itIFIi+ECRBCZNb1ZCGJEywoiUhkWkVhApEYxI0WFEyggjUhoWkVpBpESwtNYJIklhRMoIuxNpv0nTm5UQ9i/SByKpYUSKDk9OJJlJNm4WRErDIlIriJQIFtbqeLTfpOnNSggjUkYYkdKwiNQKIiWCESk6bFSk3/K0RVJc9Ghy0GB//+4RKdECvaYerFGR1uERKRHMI1J0GJEywg5F2mfS9GYlhN2L1PZIZpKNmwWR0rCI1AoiJYJltfo8QqQdQaSMMCKlYRGpFURKBCNSdBiRMsIeRdpj0vRmJYQRKSOMSGlYRGoFkRLBiBQdRqSMsEuRhk2a3qyEMCJlhG2LtMMjROoNImWEESkNi0itIFIiGJGiw4iUEUakNCwitYJIiWBJrV0eDZs0vVkJ4QmKJDHJxs2CSGlYRGoFkRLBiBQd9i5Sj0eIhEifDyNSRtipSEMmTW9WQhiRMsKIlIZFpFYQKRE8UqQBk6Y3KyGMSBlhRErDIlIriJQIFtQa8giRukGkjDAipWERqRVESgSPFWm3SdOblRBGpIwwIqVhEakVREoEI1J0GJEywoiUhkWkVg4USWCSjZvFr0g7TZrerIQwImWEDYu0x6OdJk1vVkLYuUi9HiESIn06jEgZYURKwyJSK4iUCI4g0g6TpjcrITxJkfabZONmsSvSfo8QKQwixVjGgTAipWERqRVESgTHEKnfpOnNSggjUoxlHAj7FqnXpOnNSggjUoxlHAgjUhoWkVpBpERwHJH6TJrerIQwIsVYxoGwWZFkHiFSI4gUYxkHwt5F6jFperMSwogUYxkHwu5F6po0vVkJYd8i7fBov0k2bhZESsMiUiuIlAiOJlLHpOnNSggjUoRlHAojUhoWkVpBpERwPJHaKk1vVkLYtUg7PUKkUb/WeIRI60xUpH0m2bhZJiFSaNL0ZiWEpyrSHpNs3CxGRdJ5FJo0vVkJYUQav4yD4YmI1DRperMSwpMVadgkGzfLVERqmDS9WQlhRBq/jIPhyYhUmzS9WQlhzyINevSx/P2hlT8Jno5IlUnTm5UQnq5Ig49LNm4WmyId4lFl0vRmJYSnLtIOk2zcLFMS6ST+Kg9mEakVREoEpxDpJPoqD2YRqZXBFhFI+Z8AABCjSURBVEUa7TLJxs0yLZFOYq/yYBaRWkGkRHAakZYqTW9WQtitSFKPEEn56xEeIZLRIFIiOJVIJxOclRCevki9Jtm4WSyKNMqjk6HvaVav8mAWkVoZaFHuESKpfj1WJI1JLmYlhBFp1DLGwZMUSaGSi1kJYaciKTxCJNWvI4gkNsnFrITwEYjUZ5KNm8WgSDE8EpvkYlZC+BhE6lHJxs0yWZGEKrmYlRA+DpE6Jtm4WSYskkglF7MSwj5F0nrUMcnGzWJPpHgeSVxyMSshjEgjljEWnrpI+2xyMSshfCwitUyycbOYEymBR0M6uZiVED4akUKVbNwsRyRSr00uZiWEXYp0kEerk8/3VVYtYzR8bCK1XXIxKyF8TCI1HpZs3CzWRErvUeiSi1kJYY8ijfBoa5KNm+U4RTo56AMebNxiiIRI+3/9aR5tXXIxKyF8dCJ9rAtEWMZ42JZIn+rRifYsQEQakf5Vj/PoY1Ng9DIiwMct0lqmHO2ngL2J9DFapMqnMcuIA5sSKYdHCpkQaUQ6q14ZEEekj225A5YRC0YkuUyINCK9IkXOIcuIBRsSKatG23xW+ylgVyIl8Gj7mkmzDNWasxR2KlKdtO2ngD2JlMSj2ifpMlRrjsamFSm3OLvSv8JPnZUQVov0fF2W5fXznt1jNnetOqlHlUk9Vk1epFyajMmnzUoIa0X6Vq7zbXD3mM1dq07s0UdwFdVVNpchOtLnUKTcThyaz5mVEFaKdF+eP77P3x/Py7uB3WM2d606tUddraoXT7+rFQgm5E+k3D6MSvpZCWGlSGX5Z/Xvr/J8YPeYzR2r/mSPWg9Qnb3znV9j5k2k3CbETq7B6kR6rh41vpevO3eP2dyx6s+VaCDBStrj+Zh7Eyn33f5Tk3SwOpHuyu0hgefyfufuMZv9q86lzb4Eq13v+h0uOfxtAw76kwSRkqU5hkMHqxPpuvy1+elXeb1z95jN/lXn0USazgqHFzzfvvj6aD+mVdX6gkh5IhysTqTzsvqx+YqmtXvMZv+qExlgKY0m542/EW9+1dw1b00l2G5Prj+IdGBiiVSWfT+2d4/Z3N7Sx5alBK2fP+p/g11VGni1Hf5qm/nxDjZqTk5aO8yLtLnVNWtzCOd/jfTZV3w4bGMV7h6R9qy6Lw5hRDJQGJH8w4hkoHAWkb72Hxlo7R6zKVt1XxzCiGSgcBaRch3+lsQhjEgGCmcR6b7x19O7nbvHbMpW3ReHMCIZKJz5LUL1GxK6u8dsylbdF4cwIhkonEWk6h2mf8IDA63dYzZFq+6LQxiRDBTOI9Jddc7D/cDuMZuiVffFIYxIBgrnEal1Fl71IPIJJ/bti0MYkQwUziTS/GfzvPD62djP8HTxMZuCVffFIYxIBgrnEukzY314Y2FEMlAYkfzDiGSgMCL5hxHJQGFE8g8jkoHCiOQfRiQDhRHJP4xIBgojkn8YkQwURiT/MCIZKIxI/mFEMlAYkfzDiGSgMCL5hxHJQGFE8g8jkoHCiOQfRiQDhRHJP4xIBgojkn8YkQwURiT/MCIZKIxI/mFEMlD4GEQixEsQiZAIQSRCIgSRCIkQRCIkQhCJkAhBJEIiBJEIiRBEIiRCEImQCEEkQiJkGiI97/oY/r15L6v0llJtNnJff9nTmIo9V1BVTrX0ZhistNQkRNr9xTB789oaWqQvpHmuv6cj8jfe1JUTLb0ZBisuNQWR7quvKmt/Ce3+PJaPA6VUm408l9WtMqZizxU0KqdZ+kBJVY5tsFMQafvlmb/Kc/VlrzeX7S+l2qzzWNa3ypiK3StoVk6y9CAMVl5qAiLVX+f8vXzVXvg8uKlapVSbVX4tHuy/VU+vR1TsXEFQOcXSwzBYxRVNQKT629Cfu19Duyfv5feBUqrNKov/td1X32U4pmLnCoLKKZYehsEqrmgCIl2XvzY//SqvlZd9LX8uv3jz/PuvvlKqzSrl9Z/6S0HHVOxcQVA5xdLDMFjFFU1ApPP6gKj6ufxjdXzmrqeUarPK+2pP2bu4cVcQVE6x9DAMVnFFExCpLPt+lGXxf57H5RRfr1dTa5VSbfYvakzF/iuo/5ecaOlDVyrO0Q32yEU6r57xfls+Rnu6vZMtfehKxTm6wR65SHWely8xPd3eyZYuu1JxjmSwiFRd9tzr7R156cIrFec4BjsBkb42WtP/4bC+bNkppdrsVutZXIQr6Lu9oy69CoNVLH0CIo05StvIcmjxjiFvb5WoR2mDysmWXoXBKpY+AZHuG38i078nbJs/5ddOKdVmmO2tMqZi/xV0b+/IS6/CYBVLn4BI9TDqvzpLc16+b376uazSKqXaDLO9VcZU7L+CbeVkS6/CYBVLn4BI8+37Bv/oXxLfVe9B+bp611WrlGqztab+xY2/gu3PyZbevCoGKy01BZHuqje2a98RNn9fXGY5iNdv69NLWqVUm0GqW2hMxd4r2FZOtvQ6DFZeagoijTn/7M/55rLfe0sdfHZc/b+6yOef1ZVTLb0RBisuNQmR5j8PPiN6Pn/8Vr9BsVtKtdlI4znDmIo9V9ConGbpzTBYaalpiERI5iASIRGCSIRECCIREiGIREiEIBIhEYJIhEQIIhESIYhESIQgEiERgkiERAgiTTuzS8Gv3i7Lojh9+ITlTDeINOncFjtv4PpXL8UqiDQmiDTpFLtFqn91thbp5ZPWNM0g0qQjEmnx0+0nrWe6QaRJRyrSJy1nwmGEkw4ifVYYod+sBLhdvMIpr+rXNw+Xi90XD1tglf7Lbn4VQA9XyxdMp1e8XtIGkfxmcZd/K4vgkNtsc+CgOHub60V62166KK4+sY9JBJH8ZnF/L6t7/vox5LTaLud6kepqHAzXBpH8ZuXL4g4/u90+hFyud8wfFkJdbJCBS7d+ul1eera5+FnatU8uiOQ3S4/eVj8tFDhd/PO22DFb/26hwubJ3cClWz+dVn9LmnH8QRvm5Tf133/e1nf8m/oZ2UNR3MwPP2qHSNowL78p1o86mx8X/71o3P3Xz+0OEenl9qJEJG2Yl9807u3dgwXV4Yb9l25A68Pfu45QkN1hXn7TEakIM9eK1Dj8jUjKMC+/iS7S+hHt9OL2AZG0YV5+0yfSbmT3pZuHv2/f9l2O9IZ5+U1HpIvOuRAc/v6sMC+/6Yh003lrj0qkes8tImnDvPymI9JL/daexaucy7lepPUj0gsHG9RhXn7TEWn53K64nc3nb1eLH57W+3e9aa7+1bbO4tKniwu93ayOOcySrn1yQSS/6Yo0a/wlafUk72z7B6Vu6l9t6zwEx/w4k0IVRPKbrkiNvwStXyzd7v6TUP2rirjaXLZcvmv1R9K1Ty6I5Dc9Ii1P7CubZ+bd7n4jd/WrxoWXzw0vFgo98fZvZRCJkAhBJEIiBJEIiRBEIiRCEOkIUnSTe0mTCxM9giBS+jDRIwgipQ8TJSRCEImQCEEkQiIEkQiJEEQiJEIQiZAIQSRCIgSRCIkQRCIkQhCJkAhBJEIiBJEIiRBEIiRCEImQCEEkQiIEkQiJEEQiJEIQiZAIQSRCIgSRCIkQRCIkQhCJkAhBJEIiBJEIiRBEIiRCEImQCEEkQiIEkcgmH7ujL3ayO/FXbiGIRDZBpDFBJLIJIo0JIpFNEGlMEIlsgkhjgkhR8nZZFsXpgwSdXQprPhQX+4Dq1ntYXf9luIC3+veXxf61aUV6uCiK4mJH3UNEuvF8Z/S8djt5WX8NnkSkW+m35b2VxWwYqL9472z7RXxnbw3grL6qWVG+zfdEKdL2Os96ix0g0oPr7xH0vHY72dypXgSo+Gsnz4rbYaCsSp3VX2l5Wv/+qnlVtzvu8I3oRLopytvZfHZbFld9xfQiPfj+Qk7Pa7eTxX1gz72+iYq4H0U5DJxVXwW7eJArLt+Wzy+bj4pX4VfFlsWPPdeoE6ko1g9xL/3rVIt06/ybbT2v3U4U9wEpWu5R86r+TuXTYivJVf2Q1Pj9Krf7xNSJ9FA9EF0WTz3FlCK9LP63cIZIR5/4Iu273y+eCG2fur00ntCdbh4oXk5Xd8zmVe0zUyfSVfXQ91Dc9BRTirRY6s0ckY4829cnq42Hq+Ud+PSqfr30crW8V9920dZxr+Xup9PFRZcbzZceD5edw2Nvyxf5mzq3RX1Xvtk8OC2f7LWcvdr3kKQS6aJ6QfjSe3BRK9LF2xyRjj0NO97q1/0bEWYXm+3VYbOmSG+n4aG2xU/LA3ELA5avkKonTLOzEFulXFqxqXNZs/On9cXnRfmj/eD3VOx5laQSqaxL9wqqFGm2HsDg+mzH89qtpGFHWR8/2zz3qfeUs9C58Der311sL3da3ztPa6y6xrPVM7iNKBeNw4WLp3mrx4er2bzzLLJsHtLriUqkRune+7/6YMOuQl7iee120jh+Vj4s78MPp5s/sCzVuFoeUbvY7Kju3cvfrA4gF5s7f/NhbPPAMl893pQPm4rb51BXa9s2pQJfiuAeHty6iysc/MMUIo2J57XbSeP42ebBYbZ52KkemZZGNO79q7/hrn+z/APKy/o3W1N+VIfT37aPV/WBhNWBhsa1SkW63fPcDpHGxPPa7aTo3gfWu26qB5mlWqcNtPGbq/WPRX1Hv6qEvCkah8fWBxXetu8mUIr0VPT/7XQbRBoTz2u3k9Zd9uX2YvO2g4ui/VeWLdr4zdPajKJ+rVO/7LkI7rGrR6xy+2JJKdJLMfzmPZVIp3EPNmzWO7Q84/G8djtp3GXXh78bBx92oN17fGNPfbHm0Yv14Yaz7TM8rUjzYvgAuEqkyIe/N+sbWp7xeF67nVR32cbh77YcLXRQpPrHIkx1oKFBlXKRBm9tlUg3jWecfU8YEYkckOouun4AOb24fUglUtFO7+HvzlX0bbeiEumh8Qqv703viEQOyPYuujz8ffvW2BVDpM7lWyJdhiJddq5q13YrKpGqN62+9RdFJHJAtnfRzuHv5sGG9UuULRoebLiYB/fz+tnaRfvkjK5It423njd/TvoaafmOo81pFH1vtUMkcki6DzO3Rfvw98P6wWKL9B7+3ta7aB7+3nnMenOBt/BNqy/t328T9ahdghP7EIk0RFrfj182Bxteqj3L+/jeP8hu611VD1c1tjWxe62n1cPQTXBmX0ukp5h/R1rkR+xTzRHp6NN4vna6MODtZvX/6tm8eiPQ6h0+pxv0Yct23iK0rfdQP0PbFnhbnjUR/k2q8cqsPrHvtvv7TW73nAuvFWkwh4jkOogUI9u77EPw6mX1UNR4a+rq1fnqGdHytcqs/ZvG/X5WPwtrYO0HlOoC9ftagydaLZGivtduXxCJHJDqLnu1VeNhe9pqdRbE5tNHboti+9qmexpFVbBxtmj9p6n2E7PqAlWl8AVLS6RdL2e2QaQxQaQYqe+y63P1flTv+5lvPirrrD6wdlr95kfnxL5tfjSfxq0/a+sqPHoXXmCFlK2P4wpFepJ9yhE5LIhkM2XrwML4XO790AYyIohkM7d7XtCoM5N/0BE5IIhkNHs/q0SZ/Z8iRMYEkYxm7+faKbP/c+3ImCCS1ez9pFVVBJ+0SsYEkazmrTrHPEJmZbH3s7/JmCCS2TxEPHAn+TYKMiaIREiEIBIhEYJIhEQIIhESIYhESIQgEiERgkiERAgiERIhiERIhCASIRGCSIRECCIREiGIREiEIBIhEYJIhETI/wfDzcaVYgzr7gAAAABJRU5ErkJggg==", "text/plain": [ "plot without title" ] }, "metadata": { "image/png": { "height": 420, "width": 420 } }, "output_type": "display_data" } ], "source": [ "dens_net_tfa = ggplot(data, aes(x = net_tfa, color = factor(e401), fill = factor(e401)) ) + \n", " geom_density() + xlim(c(-20000, 150000)) + \n", " facet_wrap(.~e401) + theme_minimal() + \n", " theme(legend.position = \"bottom\", text = element_text(size = 20))\n", " \n", "dens_net_tfa" ] }, { "attachments": {}, "cell_type": "markdown", "id": "1633f2bc", "metadata": { "papermill": { "duration": 0.038905, "end_time": "2021-03-23T13:44:33.759588", "exception": false, "start_time": "2021-03-23T13:44:33.720683", "status": "completed" }, "tags": [] }, "source": [ "As a first estimate, we calculate the unconditional average predictive effect (APE) of 401(k) eligibility on accumulated assets. This effect corresponds to the average treatment effect if 401(k) eligibility would be assigned to individuals in an entirely randomized way. The unconditional APE of e401 is about $19559$:" ] }, { "cell_type": "code", "execution_count": 5, "id": "b45661c7", "metadata": { "execution": { "iopub.execute_input": "2021-03-23T13:44:33.843625Z", "iopub.status.busy": "2021-03-23T13:44:33.842058Z", "iopub.status.idle": "2021-03-23T13:44:33.866334Z", "shell.execute_reply": "2021-03-23T13:44:33.864827Z" }, "papermill": { "duration": 0.067968, "end_time": "2021-03-23T13:44:33.866504", "exception": false, "start_time": "2021-03-23T13:44:33.798536", "status": "completed" }, "tags": [], "vscode": { "languageId": "r" } }, "outputs": [ { "data": { "text/html": [ "19559.34" ], "text/latex": [ "19559.34" ], "text/markdown": [ "19559.34" ], "text/plain": [ "[1] 19559.34" ] }, "metadata": {}, "output_type": "display_data" } ], "source": [ "APE_e401_uncond = data[e401==1, mean(net_tfa)] - data[e401==0, mean(net_tfa)]\n", "round(APE_e401_uncond, 2)" ] }, { "attachments": {}, "cell_type": "markdown", "id": "2457ef59", "metadata": { "papermill": { "duration": 0.040553, "end_time": "2021-03-23T13:44:33.950269", "exception": false, "start_time": "2021-03-23T13:44:33.909716", "status": "completed" }, "tags": [] }, "source": [ "Among the $3682$ individuals that are eligible, $2594$ decided to participate in the program. The unconditional APE of p401 is about $27372$:" ] }, { "cell_type": "code", "execution_count": 6, "id": "57d965fb", "metadata": { "execution": { "iopub.execute_input": "2021-03-23T13:44:34.036419Z", "iopub.status.busy": "2021-03-23T13:44:34.034858Z", "iopub.status.idle": "2021-03-23T13:44:34.062248Z", "shell.execute_reply": "2021-03-23T13:44:34.060319Z" }, "papermill": { "duration": 0.07273, "end_time": "2021-03-23T13:44:34.062417", "exception": false, "start_time": "2021-03-23T13:44:33.989687", "status": "completed" }, "tags": [], "vscode": { "languageId": "r" } }, "outputs": [ { "data": { "text/html": [ "27371.58" ], "text/latex": [ "27371.58" ], "text/markdown": [ "27371.58" ], "text/plain": [ "[1] 27371.58" ] }, "metadata": {}, "output_type": "display_data" } ], "source": [ "APE_p401_uncond = data[p401==1, mean(net_tfa)] - data[p401==0, mean(net_tfa)]\n", "round(APE_p401_uncond, 2)" ] }, { "attachments": {}, "cell_type": "markdown", "id": "bd7bffa3", "metadata": { "papermill": { "duration": 0.040806, "end_time": "2021-03-23T13:44:34.144049", "exception": false, "start_time": "2021-03-23T13:44:34.103243", "status": "completed" }, "tags": [] }, "source": [ "As discussed, these estimates are biased since they do not account for saver heterogeneity and endogeneity of participation." ] }, { "attachments": {}, "cell_type": "markdown", "id": "4d5c996f", "metadata": { "papermill": { "duration": 0.040491, "end_time": "2021-03-23T13:44:34.224987", "exception": false, "start_time": "2021-03-23T13:44:34.184496", "status": "completed" }, "tags": [] }, "source": [ "## The `DoubleML` package" ] }, { "attachments": {}, "cell_type": "markdown", "id": "89867310", "metadata": { "papermill": { "duration": 0.041623, "end_time": "2021-03-23T13:44:34.307306", "exception": false, "start_time": "2021-03-23T13:44:34.265683", "status": "completed" }, "tags": [] }, "source": [ "Let's use the package [DoubleML](https://docs.doubleml.org/stable/index.html) to estimate the average treatment effect of 401(k) eligibility, i.e. `e401`, and participation, i.e. `p401`, on net financial assets `net_tfa`." ] }, { "attachments": {}, "cell_type": "markdown", "id": "6f5da000", "metadata": { "papermill": { "duration": 0.042605, "end_time": "2021-03-23T13:47:34.483687", "exception": false, "start_time": "2021-03-23T13:47:34.441082", "status": "completed" }, "tags": [] }, "source": [ "## Estimating the Average Treatment Effect of 401(k) Eligibility on Net Financial Assets" ] }, { "attachments": {}, "cell_type": "markdown", "id": "626c4986", "metadata": { "papermill": { "duration": 0.043776, "end_time": "2021-03-23T13:47:34.573553", "exception": false, "start_time": "2021-03-23T13:47:34.529777", "status": "completed" }, "tags": [] }, "source": [ "We first look at the treatment effect of `e401` on net total financial assets. We give estimates of the ATE in the linear model\n", "\n", "\\begin{equation*}\n", "Y = D \\alpha + f(X)'\\beta+ \\epsilon,\n", "\\end{equation*}\n", "where $f(X)$ is a dictonary applied to the raw regressors. $X$ contains variables on marital status, two-earner status, defined benefit pension status, IRA participation, home ownership, family size, education, age, and income. \n", "\n", "In the following, we will consider two different models, \n", "\n", "* a basic model specification that includes the raw regressors, i.e., $f(X) = X$, and \n", "\n", "* a flexible model specification, where $f(X)$ includes the raw regressors $X$ and the orthogonal polynomials of degree 2 for the variables family size education, age, and income. \n", "\n", "We will use the basic model specification whenever we use nonlinear methods, for example regression trees or random forests, and use the flexible model for linear methods such as the lasso. There are, of course, multiple ways how the model can be specified even more flexibly, for example including interactions of variable and higher order interaction. However, for the sake of simplicity we stick to the specification above. Users who are interested in varying the model can manipulate the model formula below (`formula_flex`), for example implementing the orignal specification in [Chernozhukov et al. (2018)](https://arxiv.org/abs/1608.00060). \n", "\n", "In the first step, we report estimates of the average treatment effect (ATE) of 401(k) eligibility on net financial assets both in the partially linear regression (PLR) model and in the interactive regression model (IRM) allowing for heterogeneous treatment effects. \n" ] }, { "attachments": {}, "cell_type": "markdown", "id": "3261c60c", "metadata": {}, "source": [ "### The Data Backend: `DoubleMLData`" ] }, { "attachments": {}, "cell_type": "markdown", "id": "b195be65", "metadata": {}, "source": [ "To start our analysis, we initialize the data backend, i.e., a new instance of a [DoubleMLData](https://docs.doubleml.org/r/stable/reference/DoubleMLData.html) object. Here, we manually implement the regression model by using R's formula interface. A shortcut would be to directly specify the options `polynomial_features` and `instrument` when calling [fetch_401k()](https://docs.doubleml.org/r/stable/reference/fetch_401k.html).$^{**}$\n", "\n", "To implement both models (basic and flexible), we generate two data backends: `data_dml_base` and `data_dml_flex`.\n", "\n", "$^{**}$ Note that the model specification using `polynomial_features` differs from the one used in our example. " ] }, { "cell_type": "code", "execution_count": 7, "id": "e028de75", "metadata": { "execution": { "iopub.execute_input": "2021-03-23T13:47:34.664674Z", "iopub.status.busy": "2021-03-23T13:47:34.662690Z", "iopub.status.idle": "2021-03-23T13:47:34.941823Z", "shell.execute_reply": "2021-03-23T13:47:34.939905Z" }, "papermill": { "duration": 0.326212, "end_time": "2021-03-23T13:47:34.941985", "exception": false, "start_time": "2021-03-23T13:47:34.615773", "status": "completed" }, "tags": [], "vscode": { "languageId": "r" } }, "outputs": [ { "data": { "text/plain": [ "================= DoubleMLData Object ==================\n", "\n", "\n", "------------------ Data summary ------------------\n", "Outcome variable: net_tfa\n", "Treatment variable(s): e401\n", "Covariates: age, inc, educ, fsize, marr, twoearn, db, pira, hown\n", "Instrument(s): \n", "No. Observations: 9915" ] }, "metadata": {}, "output_type": "display_data" } ], "source": [ "# Set up basic model: Specify variables for data-backend\n", "features_base = c(\"age\", \"inc\", \"educ\", \"fsize\",\n", " \"marr\", \"twoearn\", \"db\", \"pira\", \"hown\")\n", "\n", "# Initialize DoubleMLData (data-backend of DoubleML)\n", "data_dml_base = DoubleMLData$new(data,\n", " y_col = \"net_tfa\",\n", " d_cols = \"e401\",\n", " x_cols = features_base)\n", "data_dml_base" ] }, { "cell_type": "code", "execution_count": 8, "id": "a8726d22", "metadata": { "execution": { "iopub.execute_input": "2021-03-23T13:47:34.664674Z", "iopub.status.busy": "2021-03-23T13:47:34.662690Z", "iopub.status.idle": "2021-03-23T13:47:34.941823Z", "shell.execute_reply": "2021-03-23T13:47:34.939905Z" }, "papermill": { "duration": 0.326212, "end_time": "2021-03-23T13:47:34.941985", "exception": false, "start_time": "2021-03-23T13:47:34.615773", "status": "completed" }, "tags": [], "vscode": { "languageId": "r" } }, "outputs": [ { "data": { "text/plain": [ "================= DoubleMLData Object ==================\n", "\n", "\n", "------------------ Data summary ------------------\n", "Outcome variable: net_tfa\n", "Treatment variable(s): e401\n", "Covariates: poly.age..2..raw...TRUE.1, poly.age..2..raw...TRUE.2, poly.inc..2..raw...TRUE.1, poly.inc..2..raw...TRUE.2, poly.educ..2..raw...TRUE.1, poly.educ..2..raw...TRUE.2, poly.fsize..2..raw...TRUE.1, poly.fsize..2..raw...TRUE.2, marr, twoearn, db, pira, hown\n", "Instrument(s): \n", "No. Observations: 9915" ] }, "metadata": {}, "output_type": "display_data" } ], "source": [ "# Set up a model according to regression formula with polynomials\n", "formula_flex = formula(\" ~ -1 + poly(age, 2, raw=TRUE) +\n", " poly(inc, 2, raw=TRUE) + poly(educ, 2, raw=TRUE) +\n", " poly(fsize, 2, raw=TRUE) + marr + twoearn +\n", " db + pira + hown\")\n", "features_flex = data.frame(model.matrix(formula_flex, data))\n", "\n", "model_data = data.table(\"net_tfa\" = data[, net_tfa],\n", " \"e401\" = data[, e401],\n", " features_flex)\n", "\n", "# Initialize DoubleMLData (data-backend of DoubleML)\n", "data_dml_flex = DoubleMLData$new(model_data,\n", " y_col = \"net_tfa\",\n", " d_cols = \"e401\")\n", "\n", "data_dml_flex" ] }, { "attachments": {}, "cell_type": "markdown", "id": "f7bf775d", "metadata": { "papermill": { "duration": 0.044143, "end_time": "2021-03-23T13:47:35.029558", "exception": false, "start_time": "2021-03-23T13:47:34.985415", "status": "completed" }, "tags": [] }, "source": [ "### Partially Linear Regression Model (PLR)" ] }, { "attachments": {}, "cell_type": "markdown", "id": "e88a8607", "metadata": { "papermill": { "duration": 0.043027, "end_time": "2021-03-23T13:47:35.118878", "exception": false, "start_time": "2021-03-23T13:47:35.075851", "status": "completed" }, "tags": [] }, "source": [ "We start using lasso to estimate the function $g_0$ and $m_0$ in the following PLR model:" ] }, { "attachments": {}, "cell_type": "markdown", "id": "3e99a6d2", "metadata": { "papermill": { "duration": 0.044546, "end_time": "2021-03-23T13:47:35.207536", "exception": false, "start_time": "2021-03-23T13:47:35.162990", "status": "completed" }, "tags": [] }, "source": [ "\\begin{eqnarray}\n", "& Y = D\\theta_0 + g_0(X) + \\zeta, &\\quad E[\\zeta \\mid D,X]= 0,\\\\\n", "& D = m_0(X) + V, &\\quad E[V \\mid X] = 0.\n", "\\end{eqnarray}" ] }, { "attachments": {}, "cell_type": "markdown", "id": "667de94c", "metadata": {}, "source": [ "To estimate the causal parameter $\\theta_0$ here, we use double machine learning with 3-fold cross-fitting. \n", " \n", "Estimation of the nuisance components $g_0$ and $m_0$, is based on the lasso with cross-validated choice of the penalty term , $\\lambda$, as provided by the [glmnet package](https://glmnet.stanford.edu/reference/cv.glmnet.html). We load the learner by using the [mlr3](https://mlr3.mlr-org.com/) function [lrn()](https://mlr3.mlr-org.com/reference/Learner.html). Hyperparameters and options can be set during instantiation of the learner. Here we specify that the lasso should use that value of $\\lambda$ that minimizes the cross-validated mean squared error which is based on 5-fold cross validation.\n", "\n", "In order to use a learner, the underlying R packages have to be installed. In this case, the package [glmnet package](https://glmnet.stanford.edu/reference/cv.glmnet.html) needs to be installed. Moreover, installation of the package [mlr3learners](https://mlr3learners.mlr-org.com/) is required.\n", "\n", "We start by estimation the ATE in the basic model and then repeat the estimation in the flexible model." ] }, { "cell_type": "code", "execution_count": 9, "id": "c3f6a786", "metadata": { "execution": { "iopub.execute_input": "2021-03-23T13:47:35.302091Z", "iopub.status.busy": "2021-03-23T13:47:35.299667Z", "iopub.status.idle": "2021-03-23T13:47:58.447363Z", "shell.execute_reply": "2021-03-23T13:47:58.446093Z" }, "papermill": { "duration": 23.196892, "end_time": "2021-03-23T13:47:58.447523", "exception": false, "start_time": "2021-03-23T13:47:35.250631", "status": "completed" }, "tags": [], "vscode": { "languageId": "r" } }, "outputs": [ { "name": "stdout", "output_type": "stream", "text": [ "Estimates and significance testing of the effect of target variables\n", " Estimate. Std. Error t value Pr(>|t|) \n", "e401 6133 1465 4.185 2.85e-05 ***\n", "---\n", "Signif. codes: 0 '***' 0.001 '**' 0.01 '*' 0.05 '.' 0.1 ' ' 1\n", "\n", "\n" ] } ], "source": [ "# Initialize learners\n", "set.seed(123)\n", "lasso = lrn(\"regr.cv_glmnet\", nfolds = 5, s = \"lambda.min\")\n", "lasso_class = lrn(\"classif.cv_glmnet\", nfolds = 5, s = \"lambda.min\")\n", "\n", "# Initialize DoubleMLPLR model\n", "dml_plr_lasso = DoubleMLPLR$new(data_dml_base, \n", " ml_l = lasso,\n", " ml_m = lasso_class,\n", " n_folds = 3)\n", "dml_plr_lasso$fit()\n", "dml_plr_lasso$summary()" ] }, { "cell_type": "code", "execution_count": 10, "id": "fde8ed43", "metadata": { "vscode": { "languageId": "r" } }, "outputs": [ { "name": "stdout", "output_type": "stream", "text": [ "Estimates and significance testing of the effect of target variables\n", " Estimate. Std. Error t value Pr(>|t|) \n", "e401 9580 1325 7.229 4.87e-13 ***\n", "---\n", "Signif. codes: 0 '***' 0.001 '**' 0.01 '*' 0.05 '.' 0.1 ' ' 1\n", "\n", "\n" ] } ], "source": [ "# Initialize learners\n", "set.seed(123)\n", "lasso = lrn(\"regr.cv_glmnet\", nfolds = 5, s = \"lambda.min\")\n", "lasso_class = lrn(\"classif.cv_glmnet\", nfolds = 5, s = \"lambda.min\")\n", "\n", "# Initialize DoubleMLPLR model\n", "dml_plr_lasso = DoubleMLPLR$new(data_dml_flex, \n", " ml_l = lasso,\n", " ml_m = lasso_class,\n", " n_folds = 3)\n", "dml_plr_lasso$fit()\n", "dml_plr_lasso$summary()" ] }, { "attachments": {}, "cell_type": "markdown", "id": "d922db8d", "metadata": { "papermill": { "duration": 0.046627, "end_time": "2021-03-23T13:47:58.987016", "exception": false, "start_time": "2021-03-23T13:47:58.940389", "status": "completed" }, "tags": [] }, "source": [ "Alternatively, we can repeat this procedure with other machine learning methods, for example a random forest learner as provided by the [ranger](https://github.com/imbs-hl/ranger) package for R. The website of the [mlr3extralearners](https://mlr3extralearners.mlr-org.com/articles/learners/list_learners.html) package has a searchable list of all learners that are available in the [mlr3verse](https://mlr3verse.mlr-org.com/)." ] }, { "cell_type": "code", "execution_count": 11, "id": "9670966e", "metadata": { "execution": { "iopub.execute_input": "2021-03-23T13:47:59.087352Z", "iopub.status.busy": "2021-03-23T13:47:59.086372Z", "iopub.status.idle": "2021-03-23T13:49:02.075109Z", "shell.execute_reply": "2021-03-23T13:49:02.075650Z" }, "papermill": { "duration": 63.042388, "end_time": "2021-03-23T13:49:02.075853", "exception": false, "start_time": "2021-03-23T13:47:59.033465", "status": "completed" }, "tags": [], "vscode": { "languageId": "r" } }, "outputs": [ { "name": "stdout", "output_type": "stream", "text": [ "Estimates and significance testing of the effect of target variables\n", " Estimate. Std. Error t value Pr(>|t|) \n", "e401 9127 1313 6.952 3.61e-12 ***\n", "---\n", "Signif. codes: 0 '***' 0.001 '**' 0.01 '*' 0.05 '.' 0.1 ' ' 1\n", "\n", "\n" ] } ], "source": [ "# Random Forest\n", "randomForest = lrn(\"regr.ranger\", max.depth = 7,\n", " mtry = 3, min.node.size = 3)\n", "randomForest_class = lrn(\"classif.ranger\", max.depth = 5,\n", " mtry = 4, min.node.size = 7)\n", "\n", "set.seed(123)\n", "dml_plr_forest = DoubleMLPLR$new(data_dml_base,\n", " ml_l = randomForest,\n", " ml_m = randomForest_class,\n", " n_folds = 3)\n", "dml_plr_forest$fit() \n", "dml_plr_forest$summary()" ] }, { "attachments": {}, "cell_type": "markdown", "id": "f2cbd323", "metadata": {}, "source": [ "Now, let's use a regression tree as provided by the R package [rpart](https://github.com/bethatkinson/rpart)." ] }, { "cell_type": "code", "execution_count": 12, "id": "76a4b258", "metadata": { "execution": { "iopub.execute_input": "2021-03-23T13:49:02.526135Z", "iopub.status.busy": "2021-03-23T13:49:02.524567Z", "iopub.status.idle": "2021-03-23T13:49:04.092172Z", "shell.execute_reply": "2021-03-23T13:49:04.090774Z" }, "papermill": { "duration": 1.627688, "end_time": "2021-03-23T13:49:04.092395", "exception": false, "start_time": "2021-03-23T13:49:02.464707", "status": "completed" }, "tags": [], "vscode": { "languageId": "r" } }, "outputs": [ { "name": "stdout", "output_type": "stream", "text": [ "Estimates and significance testing of the effect of target variables\n", " Estimate. Std. Error t value Pr(>|t|) \n", "e401 8210 1324 6.203 5.55e-10 ***\n", "---\n", "Signif. codes: 0 '***' 0.001 '**' 0.01 '*' 0.05 '.' 0.1 ' ' 1\n", "\n", "\n" ] } ], "source": [ "# Trees\n", "trees = lrn(\"regr.rpart\", cp = 0.0047, minsplit = 203)\n", "trees_class = lrn(\"classif.rpart\", cp = 0.0042, minsplit = 104)\n", "\n", "set.seed(123)\n", "dml_plr_tree = DoubleMLPLR$new(data_dml_base,\n", " ml_l = trees,\n", " ml_m = trees_class,\n", " n_folds = 3)\n", "dml_plr_tree$fit()\n", "dml_plr_tree$summary()" ] }, { "attachments": {}, "cell_type": "markdown", "id": "dd6800f9", "metadata": {}, "source": [ "We can also experiment with extreme gradient boosting as provided by [xgboost](https://xgboost.readthedocs.io/en/latest/)." ] }, { "cell_type": "code", "execution_count": 13, "id": "a2a0c914", "metadata": { "execution": { "iopub.execute_input": "2021-03-23T13:49:04.206928Z", "iopub.status.busy": "2021-03-23T13:49:04.206179Z", "iopub.status.idle": "2021-03-23T13:49:05.194244Z", "shell.execute_reply": "2021-03-23T13:49:05.192417Z" }, "papermill": { "duration": 1.048317, "end_time": "2021-03-23T13:49:05.194441", "exception": false, "start_time": "2021-03-23T13:49:04.146124", "status": "completed" }, "tags": [], "vscode": { "languageId": "r" } }, "outputs": [ { "name": "stdout", "output_type": "stream", "text": [ "Estimates and significance testing of the effect of target variables\n", " Estimate. Std. Error t value Pr(>|t|) \n", "e401 8700 1360 6.399 1.57e-10 ***\n", "---\n", "Signif. codes: 0 '***' 0.001 '**' 0.01 '*' 0.05 '.' 0.1 ' ' 1\n", "\n", "\n" ] } ], "source": [ "# Boosted trees\n", "boost = lrn(\"regr.xgboost\",\n", " objective = \"reg:squarederror\",\n", " eta = 0.1, nrounds = 35)\n", "boost_class = lrn(\"classif.xgboost\",\n", " objective = \"binary:logistic\", eval_metric = \"logloss\",\n", " eta = 0.1, nrounds = 34)\n", "\n", "set.seed(123)\n", "dml_plr_boost = DoubleMLPLR$new(data_dml_base,\n", " ml_l = boost,\n", " ml_m = boost_class,\n", " n_folds = 3)\n", "dml_plr_boost$fit()\n", "dml_plr_boost$summary()" ] }, { "attachments": {}, "cell_type": "markdown", "id": "cce3aca6", "metadata": { "papermill": { "duration": 0.055456, "end_time": "2021-03-23T13:49:05.305048", "exception": false, "start_time": "2021-03-23T13:49:05.249592", "status": "completed" }, "tags": [] }, "source": [ "Let's sum up the results:" ] }, { "cell_type": "code", "execution_count": 14, "id": "20e3a391", "metadata": { "execution": { "iopub.execute_input": "2021-03-23T13:49:05.421244Z", "iopub.status.busy": "2021-03-23T13:49:05.419710Z", "iopub.status.idle": "2021-03-23T13:49:05.487573Z", "shell.execute_reply": "2021-03-23T13:49:05.486132Z" }, "papermill": { "duration": 0.127098, "end_time": "2021-03-23T13:49:05.487776", "exception": false, "start_time": "2021-03-23T13:49:05.360678", "status": "completed" }, "tags": [], "vscode": { "languageId": "r" } }, "outputs": [ { "data": { "text/html": [ "\n", "\n", "\n", "\t\n", "\t\n", "\n", "\n", "\t\n", "\t\n", "\t\n", "\t\n", "\n", "
A data.table: 4 × 5
modelMLEstimatelowerupper
<chr><chr><dbl><dbl><dbl>
PLRglmnet 9579.7716982.44312177.10
PLRranger 9126.9996553.68411700.31
PLRrpart 8209.7375615.52510803.95
PLRxgboost8700.1346035.28111364.99
\n" ], "text/latex": [ "A data.table: 4 × 5\n", "\\begin{tabular}{lllll}\n", " model & ML & Estimate & lower & upper\\\\\n", " & & & & \\\\\n", "\\hline\n", "\t PLR & glmnet & 9579.771 & 6982.443 & 12177.10\\\\\n", "\t PLR & ranger & 9126.999 & 6553.684 & 11700.31\\\\\n", "\t PLR & rpart & 8209.737 & 5615.525 & 10803.95\\\\\n", "\t PLR & xgboost & 8700.134 & 6035.281 & 11364.99\\\\\n", "\\end{tabular}\n" ], "text/markdown": [ "\n", "A data.table: 4 × 5\n", "\n", "| model <chr> | ML <chr> | Estimate <dbl> | lower <dbl> | upper <dbl> |\n", "|---|---|---|---|---|\n", "| PLR | glmnet | 9579.771 | 6982.443 | 12177.10 |\n", "| PLR | ranger | 9126.999 | 6553.684 | 11700.31 |\n", "| PLR | rpart | 8209.737 | 5615.525 | 10803.95 |\n", "| PLR | xgboost | 8700.134 | 6035.281 | 11364.99 |\n", "\n" ], "text/plain": [ " model ML Estimate lower upper \n", "1 PLR glmnet 9579.771 6982.443 12177.10\n", "2 PLR ranger 9126.999 6553.684 11700.31\n", "3 PLR rpart 8209.737 5615.525 10803.95\n", "4 PLR xgboost 8700.134 6035.281 11364.99" ] }, "metadata": {}, "output_type": "display_data" } ], "source": [ "confints = rbind(dml_plr_lasso$confint(), dml_plr_forest$confint(),\n", " dml_plr_tree$confint(), dml_plr_boost$confint())\n", "estimates = c(dml_plr_lasso$coef, dml_plr_forest$coef,\n", " dml_plr_tree$coef, dml_plr_boost$coef)\n", "result_plr = data.table(\"model\" = \"PLR\", \n", " \"ML\" = c(\"glmnet\", \"ranger\", \"rpart\", \"xgboost\"), \n", " \"Estimate\" = estimates,\n", " \"lower\" = confints[,1],\n", " \"upper\" = confints[,2])\n", "result_plr" ] }, { "cell_type": "code", "execution_count": 15, "id": "50e2a7c9", "metadata": { "vscode": { "languageId": "r" } }, "outputs": [ { "data": { "image/png": "iVBORw0KGgoAAAANSUhEUgAAA0gAAANICAMAAADKOT/pAAAAbFBMVEUAAAAAv8RNTU1oaGh8fHx8rgCDg4OMjIyVlZWampqjo6Onp6evr6+ysrK5ubm9vb2+vr7BwcHHfP/Hx8fJycnQ0NDR0dHY2NjZ2dne3t7h4eHk5OTp6enq6urr6+vv7+/w8PD19fX4dm3////4p3w3AAAACXBIWXMAABJ0AAASdAHeZh94AAAgAElEQVR4nO2d2WLayhZElcYRo83lGLCZAoT//8eLJAYJlRnCVqtL1Ho4uG3nuJJmWdLuKdoJIZ4mqjuAEE1AIglhgEQSwgCJJIQBEkkIAySSEAZIJCEMkEhCGPASIv2pO8CdkORUTIBECgiSnIoJkEgBQZJTMQESKSBIciomQCIFBElOxQRIpIAgyamYgH8TaeyOH21HHec6o+2h5U5kn5gN9h8OZsfvvmj6gqTnWXIqJuCfRJodPUk+SokzORYXIvUOjd4ONb1B0vMsORUT8C8izU4XnJVzo/Vutx64OL0mTd00/41jF0+3u+00diPQ9AdJz7PkVEzAP4g0Pd+5fbiv4wepGwO3zn+nOzRXLgZNf5D0PEtOxQQ8LNJqf3/WO4oUH2/x1pkbccGQ2enK8+EWpaZHSHqeJadiAh4WaX81Gu9Oz0i5T++SWsNH/nMjdywrzPZ/5rLpEZKeZ8mpmIDHRRqsdz+JtHBfXwPn4o9V+rmBWx2+unKDUtMjJD3PklMxAQ+LlBYVSiIt0mvR9FSzS2/i4vN3JXd+F02PkPQ8S07FBPzbOFJJpEF627a/Gk0T0RaD1KTcdyUfXjQP/BGCFmuRFtm9Wnx6Cuolt3F3ieQDkl+hLDkVE2Ai0jqOt8VvmCW3ehLpQUhyKibARKROcfgo/Y5YIj0MSU7FBFiI1HPl2XPJd3SK1YWOig03IMmpmAADkZBH6Xeo/P0gJDkVE/C0SGvo0dp1krl15xHYUanpEZKeZ8mpmIBnRVrHOY9id6w5fCWqnH1JJzVcND1C0vMsORUT8KRIMxevzp8enab+dNLZdMdZquvs+y+a/iDpeZacigl4TqR1HOfrddvYjZP2opctORqd1k2MQdMfJD3PklMxAc+J9OGKK/n2N3oZh8mrWtj3ECQ5FRPwnEjuQqTdbto7T1rd81VcW/6lpebXIMmpmABtfhIQJDkVEyCRAoIkp2ICJFJAkORUTIBECgiSnIoJkEgBQZJTMQHhivQ3TKr8K+sdaolEMqfSd78heodaIpHMkUimKCZAIgWE3qGWSCRzJJIpigmQSAGhd6glEskciWSKYgIkUkDoHWqJRDJHIpmimACJFBB6h1oikcyRSKYoJkAiBYTeoZZIJHMkkimKCciLFP2A10BVIJFMUUyARAoIvUMtkUjmSCRTFBPAb8kdSCRTFBMgkQJC71BLJJI5EskUxQRIpIDQO9SSsETazD/fvASpEolkimICkEiTN1XtakHvUEtqF6mv8ndN6B1qSd0izS80an17DVQFEskUxQSURdpfkFqbnYuiyW4zjJIXeqoU6XeYVPg3lkiIskit1J12FA136eXJ8+l6VcByRar07W+HRAKURcoeivbXonbSeo+iT6+BqkAimSKRAD+JtL8UtZLW8iAUNRLJFIkE+Emk48tOVTt/SCRL6hbpLYo22csy/QaJ5A2JZEndIvWzQl0/qzZ8SyR/SCRL6hZpEkVufnyZt/SM5A+JZEndIiU3ddHhJYV/IEkimSKRAECkpUtFSl4S+l7zVIJEMkUiAeAD0LB1eNnf1/FfjySSMRIJwF9JuAOJZIpEAkikgJBIlkgkcySSKRIJAAZkPzdeE3hAIpkikQBwilDTXJJIpkgkABSpaS5JJFMkEqAs0vLzrWkuSSRTqnyH/hckdwSHxYbNZ/vokvG/Uz1IJFNIrkj3vP3t+LFqN+lmLnUbMCIrkUyRSIBr5e/vvtMuQj6RSJaEI9IuXXEukbwhkSwJR6RJV1ckr0gkS8IQaePrGemPB/76+CEG/K47QKP4z8PPuCHS5lgBb0KlYacrkjG6IgHAONL7waJGrKBIkUimSCSAZjYEhESyJACRGmXRTiIZI5EAmv0dEBLJkrpF6jekwpBDIpkikQBwE32vCTwgkUyRSIAftyxuEhLJFIkE0BUpICSSJXWL9N2IrewKSCRTJBIAbRDZjtqfc68pKkYimSKRAD8NyOow5hqQSJZIJHMkkikSCSCRAkIiWVK3SA1EIpkikQASKSAkkiUSyRyJZIpEAkikgJBIloQg0vK9nZ021v32mqYiJJIpEgkARWqfqnXNmOUgkUyRSIAfjr48i9QEkySSKRIJgA9jbk0Ok8An+xf+2UISyRSJBCiLtHentTutpug34ZIkkXa7X0FS4V+4dpH2D0jJEtmDSMtMK24kkimVvv3tqFukg0HHqUGaIuQPiWSJRDJHIpkikQC3RNpEkfMaqAokkikSCVAWqVt4RhpGUddroCqQSKZIJACs2rnNLlf+5t+dSyKZIpEAeBzJZeNI836y7arXPJUgkUyRSAAg0qaVX9Xnll7zVIJEMkUiAWBJrnv2qN2E/YslkikSCXBt9nfUfuefHpQgkUyRSAD+QaI7kEimSCSARAoIiWRJ3SJdTGVwDSjbSSRTJBLgpkiaIuQPiWRJWCJNJJI/JJIl9YnUjSAtr4GqQCKZIpEAeZE2WKSh10BVIJFMkUiAwn3bEGj0xu+RRLJFIgF0Yl9ASCRLJJI5EskUiQRonDQIiWSKRAJIpICQSJbUL9K8X1hIwe+aRDJFIgGAJZPLwp3XQFUgkUyRSICyJfNSBdxroCqQSKZIJEDZkmR5+bABy2JzSCRTJBKgLJJrwlyGIhLJFIkEgONITVhenkcimSKRABqQDQiJZEndIr1JpLqQSJbULdKwCVtCFpFIpkgkALj6uNt72Y3P+4HPBs65weyfmr6QSKZIJAA++tJdr3/P3Emknsvo/UPTGxLJFIkEgMWGGwOye4+OIo1dPN3uttPYjR5u+kMimSKRAP8g0tSdRXJunb6uXPxw0x8SyRSJBHhYpNX+/qx3FGl2urZ8uMWDTY9IJFMkEuDhUvf+ajTeHUUauWPhYLb/7GNNj0gkUyQS4HGRBuvdSaSBWx0+vXKDB5sekUimSCTAwyJtk/8cRYrPZfDkueehpkckkikSCfBvsxiOIjlX+NRDTY9IJFMkEiAvUlZXuF3+NhXpjwf++vghBvyuO8B9/Ko7wH385+FnBCSSD3RFMkVXJIBECgiJZAnTM1KnWD94qOkRiWSKRAI8J5LK36ZIJEuYRBrnxlhHDzY9IpFMkUiA50Q6G5FOW3io6RGJZIpEAjwn0mke6jr7zENNf0gkUyQS4EmRRqeVEeOHm/6QSKZIJMCTImlhnyUSyRIukXZfxdXjDzV9IZFMkUiAxu0YhJBIpkgkgEQKCIlkiUQyRyKZIpEAEikgJJIlEskciWSKRAJgkZbvbZfO+u5+e01TERLJFIkEgCK1T8snoqjvNU41SCRTJBIA77SaE6kBJv39S2KSRLKkdpHeoqg1OazyS86TnXsNZM/fvywmSSRL6hZp705rdzomqU9/Sfr7l8YkiWRJ3SK1s2NdDiItM62IkUjWSCTAjyf2HfdqoD/ATyJZI5EAzRdJz0jWSCTALZE2UeR5FZ49LB5JJFPqFqlbeEYaRlHXa6AqIPFIIplSt0iT/TVos8uVv/lPlJVIpkgkAB5Hctk40ry//++b1zyVIJFMkUgAINKmld9m9ebBzARIJFMkEgCW5Lpnj9obr3GqQSKZIpEA12Z/R+139ulBGRLJFIkEYB8kuguJZIpEAkikgJBIloQg0qSVvrguf+k7QSKZIpEASKTN23l6UKsBRTuJZItEAvy0sC/7osrfXpFIltQu0t6j7qHoPWk1YKqdRDJGIgHKIg2jqH1u7U0aeoxTDRLJFIkEKIu0f0DK3c1NmjBHSCKZIpEAPy6j+KFJiUQyRSIBJFJASCRL6hapVbi1a8LCPhaRfv/mMEkiAcoiFfcN0sI+b/z+TWKSRAKURZrn1/LNtbDPF79/s5gkkQDgASjZsLif7vn9rYV9/pBIxtQu0sYVFvY1YEGSRDJFIgGubaKfLuzzmqYiKETSM5IxAYikhX21QOKRRELwDxLdAYlIGkcyRSKZI5FMkUgAiRQQEsmS+kWa9wsbcvG7JpFMkUgAYMkkusBroCqQSKZIJACe2SCRakEiWVK3SMlshmED1pfnkEimSCRAWSTXhDWxRSSSKRIJANcjNWBWUAGJZIpEAtxc2NcEJJIpEgkA92zwmsADEskUiQSAuwjxr0AqIpFMkUgAuK9dEzaFzCORTJFIALzTqmtW/VsimSKRALDYoAHZepBIlkgkcySSKRwi/fefV5NqF+mPB/76+CEG/K47wH38qjvAPfyXUPlPuSZSA9EVyRSGK9J/Gf5+oEQKCIlkhkSqAolkikQCSKSAkEh2ePbo6i5C+w+63x6zVIZEMoVCpPqrdrvTvnbJl/P7gNMikUzhEKn2caTsDNmTSE0wSSKZIpEAQKS3KGpNDqspkv0b+DeJlEimSCRAWaS9O63daVlSvwmXJIlkikQClEVqZ8soDiItM624kUimSCTAjytkj1ODNNfOHxLJEolkjkQyRSIBbom00Rmy/pBIltQtUrfwjKQzZD0ikSypW6RJdkrfufzNv4ODRDJFIgHwOJLLxpHmOkPWKxLJktpF2hTOomjCRigSyRSJBIAlue7Zo3YTdl2VSKZIJIDOkA0IiWRJCCI1DIlkikQCSKSAkEiWSCRzJJIpEgmQF6m8EZf2tfOKRLJEIpkjkSz59YvDJIlkjkQy5NcvEpPqf0ZqR1H3Oxk/SmY2vHuNUw0SyY5fv1hMql2kt9z0uqWmCHlEIllSt0jDwmHMwyYczSyR7JBImLJIrcJhzBstNfcHg0h6RsLcPIxZxQZ/UIikqh3klkgbieQPDpE0joSAuwgVn5HaHuNUg0QyRSIByiJ95tcgaYWsTySSJXWLlFQbon66fuK7qxWyPpFIltQu0mnv75QGeCSRbJFIAFRJ2LTPHvHvV7yTSMZIJMCPK2ST7Rra701YaC6RjJFIAP7a9h1IJFMkEkAiBYREskQimSORTJFIgIv1SDu0KslroCqQSKZIJIBECgiJZIlEMkcimSKRAHlLJpNm7AdZQiKZIpEAeZFaDbj4QCSSKRIJAG/tvCbwgEQyRSIBJFJASCRL6hPJpSuRJFJtSCRL6hOpWy7YqWrnE4lkSX0iLSVSvUgkS2qcIjR/k0h1IpEsqXuuXRPMuUAimSKRABIpICSSJXWL1EAkkikSCfCcSC5H0t4Wm7vdbLD/cDA7fv9F0xcSyRSJBDATKU7aiwuReodGb4ea3pBIpkgkABJp3m89WrWbuUXyMnXT/GfHLp5ud9tp7Eag6Q+JZIpEAgBLJo+Xv9dunL4O3Dr/aXdorrLr1UXTHxLJFIkEKFsy/4dxpN7BjLhgyOx05flILlgXTY9IJFMkEqBsSX+vznAJvvVnpi6rH2zdR/7TI3csK8ySK9ZF0yMSyRSJBCiL5B4/WizuZK8L9/U1cC7+WKXNgVsdvmHlBqWmRySSKRIJAAdkH9wW8nSpmZ5qdulNXOxO35I8FV00PSKRTJFIAIuZDe5wQdpfc9x0u39dDFKTXM4cV2oe+OOBvz5+iAG/6w5wH7/qDnAf/3n4GddEentUpKn7OnwUn56Ceslt3F0i+UBXJFN0RQLAw5gfOxGpA6yYJWUHifQgEsmSukXaudxBY3ewKJbqjv+TWCI9jESypHaRlnuTHqh/n6vaeRJVOsXqQkfFhhtIJEvqFunRhX0xvLokIqn8/SASyRIykdZwCuo6qeSNcyOwo1LTIxLJFIkEeFqkr9wshdhtT58d5X1Jb/8umh6RSKZIJMDTC/vyUoxOUnXS2XTHWarrrLhw0fSHRDJFIgGeFun85LPbbWM3TlxZ9LL7vdFp3cQYNP0hkUyRSICnRTrfzu1Zx4cpQoeKuBb2PYREsoRMpIvbtGnvPGl1z1dxbfmXlppfQyJZEoJI8366w12734xzXiSSKRIJgETatM8Vu/ZjK5PCRCKZIpEAeGZDjoemCwWKRDJFIgGASMldXf87+eg7WS375jVPJUgkUyQSoCxSsvfJBDZokUimSCRAWaR2Yan5cP+Y5DFONUgkUyQS4NZS800TtgKXSKZIJMDNpeYSyR8SyZIARNIVqR4kkiV1i6RnpNqQSJbULVKpavfpNVAVSCRTJBIA3Le1iuNILa95KkEimSKRAJrZEBASyZLaRdot83PtHtx1NUgkkikSCfDD7O/31KX2u2Z/+0QiWRKCSCHwN0yq/CtLJEskkjl/bn9LEEgkS0IQadJKX1yXf8JqgkQyRSIB4MK+4z76SfG7AUU7iWSLRAL8VP7OvtiQ8rdEMkUiAeAm+lH3UPSetPYmec1TCRLJFIkEgMe65GbXtR4/CDM8JJIpEgkADxrL3c1NmrDWXCKZIpEAL7EeSSKZIpEAEikgJJIldYvUKtzabZpQbZBIpkgkQFmkfhT1z61hFHU9xqkGiWSKRAKURZrnF/YVGrRIJFMkEgA8ALUbt0GkRDJFIgGASJviwr4GLEiSSKZIJAAsyeUX9nlNUxESyRSJBMC17eV72zVoYZ9EMkUiAfgHie5AIpkikQASKSAkkiUSyRyJZIpEAkikgJBIlkgkcySSKRIJIJGe5HeYVPg3lkgIiRQQJDklEkAiBQRJTokEkEgBQZJTIgEkUkCQ5JRIAIkUECQ5JRLgZ5GasMb8AMkblCWnRAJIpIAgySmRABIpIEhySiSARAoIkpwSCSCRAoIkp0QCSKSAIMkpkQASKSBIckokgEQKCJKcEgkgkQKCJKdEAtQuyx/Bxq+6A9zHfx5+xvl9XLtIPiD5Tc+SU1ckwE8iTZK97d7eG7A75I7mDcqSUyIB8iKdn4qWb8cNIt+9pqkIkjcoS06JBIAi5TctbsDW3yxvUJacEgkARdpfj1qT/V3dcugacU0ieYOy5JRIACTSZO/R4VPLvUn82xaTvEFZckokABKpmzsTaVg4dowUkjcoS06JBEAi7a9Cp2rdsglPSSRvUJacEgmARCrMaWjCBAeSNyhLTokEkEgBQZJTIgGQSG9FkXSquS9IclYp0n9BckdwJNIw94w0acKhfSRvUJacJFckv/+aFyJ9prVuF30fP9fSqeb+IMkpkQAXIu1pdYfdw1VoM3HnISViSN6gLDklEqAsUsbpE27pNU8lkLxBWXJKJEChJDf/zA5hzonUaoBHLG9QlpwSCQBq26lO2Rfb/M9HCSRvUJacEgnAP0h0ByRvUJacEgkgkQKCJKdEApRE+n5vt7KFSO3hN/oThJC8QVlySiTAhUjzVr50F7mh1zBVQfIGZckpkQBFkd6jS1S18whJTokEKIg0Sa5By2RlbLIwdjPpahzJKyQ5JRKgIJI7abOMovSurt+E5Ugsb1CWnBIJkBdpWFwZm05cbWuunT9IckokQF6kt+LK2HSJ+f5ur+s1UBWQvEFZckokAN7XLm20kpdNE2atkrxBWXJKJMAVkcByWVJI3qAsOSUSIG+Ji6JTiW4jkfxDklMiAfKWdHO7QX4eynVL3dr5gySnRALkRZrsrz6HWUHJxpBp/burfe38QZJTIgEK923J1vn9+f6+7tNle55skiFZ7bTqiypz/gqSCv/CdYq0zG2enz4uJa/8FySJZItiAoqVhPNxLtkcu2Z4xNLzLDkVE3BZkpt0W/u7uuPKWNflv6/b0fQ8S07FBPDXtu+ApOdZciomQCIFBElOxQRIpIAgyamYAIkUECQ5FRNwUyRNEfIHSU7FBEikgCDJqZgAiRQQJDkVE8BvyR2Q9DxLTsUESKSAIMmpmACJFBAkORUTIJECgiSnYgJKIi0/u+1s3mq7O2zETDuanmfJqZiAC5EmxS2Lo1Yj9iwm6XmWnIoJKK5HutAo3Uvfa5xqIOl5lpyKCSitkG29fx9u6ObzYasZJpH0PEtOxQRc7tlwsY4v2VVfO636giSnYgIudhEq7aqKPkcHSc+z5FRMQF6kVm5fuyPLbBcUbkh6niWnYgJ+3Gn16ifJIOl5lpyKCXhSpK07kX1iNth/OJgdv3696QuSnmfJqZiAn7YsPnLj1m5xIVLv0Ojt7mh6g6TnWXIqJuDJYsPUTfPNsYun2912GrvR7aY/SHqeJadiAi7L35/FLw9vlL8Hbp1vukNz5eLbTX+Q9DxLTsUE3B6QbV3743FBidnpUvPhFreaHiHpeZacigkoVhLeylOErh5rvnUf+ebIHesIMze+1fQISc+z5FRMwEVJbvjYpNWF+/oaOBd/rNLmwK0OX1i5wa2mR0h6niWnYgJKte358LCMwrW7w2tXo4TpqWaX3rXF5wJf8hh0vekRkp5nyamYgOdGW/dXo+l2/7oYpCa5nCruVtMjJD3PklMxAc+JFJ8ee3rJfdu/iPRHCFqsRDozS8oOuiI9B0lOxQSYTaRLHnsk0nOQ5FRMgJ1Iezc6xXLC9aZHSHqeJadiAkxFUvn7OUhyKibASqS16yST6c5DrqNbTY+Q9DxLTsUEPFu12x4++krcOAuSzmK43vQISc+z5FRMwHMijU5zfTrp9LnjtNR1Vk243vQHSc+z5FRMwHMibWM3TuRY9LI1RqPTQonx7aY/SHqeJadiAp58RlrHhylCh8mrWtj3DCQ5FRPwdLFh2jtPWt3zVVxMfr3pC5KeZ8mpmICLPRsgXgNVAUnPs+RUTIBECgiSnIoJkEgBQZJTMQHIknYUdb83+w/m/Sh69xqnGkh6niWnYgKASG+5/U6W2kTfIyQ5FRNQFmkYRcOfWqSQ9DxLTsUElEVqRdHm3Nrc2EaIApKeZ8mpmICySBfVBRUb/EGSUzEBt0TaSCR/kORUTEDZkvblM1LbY5xqIOl5lpyKCSiL9BlF7rQN10Qn9nmEJKdiAsB9W7JHZD/dtvi7qzNkfUKSUzEBQKSl06nm9UCSUzEBqJKwaZ896oOv00HS8yw5FROAS3LL93a6afH7Bn6ZDZKeZ8mpmAD+2vYdkPQ8S07FBEikgCDJqZgAiRQQJDkVE4BEmveLpyR5DVQFJD3PklMxAcCSiRb21QRJTsUElC2Za4VsXZDkVExA2ZL+Xp2bR/VxQdLzLDkVE1AWyTVhKV8Rkp5nyamYALiMohnDsGdIep4lp2ICbi7sawIkPc+SUzEBZWneJFJdkORUTADc/IR/BVIRkp5nyamYAHD1cbmFfc2ApOdZciomAK9Hcs2qf5P0PEtOxQTAYoMGZOuBJKdiAiRSQJDkVEyARAoIkpyKCeC35A5Iep4lp2ICJFJAkORUTIBECgiSnIoJkEgBQZJTMQG3RNrMP/l3tiPpeZaciglAIk3eVLWrBZKcigkAlvRV/q4JkpyKCbi91Lz17TVQFZD0PEtOxQTApeatTbJOdrLbDKNGTAUn6XmWnIoJgEdfTk6nJO0vT85rnkog6XmWnIoJ+GmF7PGAsfco+vQaqApIep4lp2ICfhJpfjiEeakT+/xBklMxAT/u2XCs1qlq5w+SnIoJgHs2bLKXdHGfRPIHSU7FBMCq3SR7SaoN3xLJHyQ5FRNQtmQSRW5+fJm39IzkD5KcigkAl5u37CJ0mifEP5BE0vMsORUT8MNhzLvzmcwNOEWWpOdZciomAD4ADVuHl/19Hf/1iKbnWXIqJoC/knAHJD3PklMxARIpIEhyKiagdpH+CEHL+X1cu0g+IPkVypJTMQESKSBIciomQCIFBElOxQRIpIAgyamYAIkUECQ5FRMgkQKCJKdiAiRSQJDkVEyARAoIkpyKCbgmUt9FraG3JBVC0vMsORUTgETa9FvJS7aOgn/DYpqeZ8mpmICfl1EMD8uR+Nf1sfQ8S07FBMBTzVORXHI1mu//O/caqApIep4lp2IC8FLzZboPV/SdXpf4V/aR9DxLTsUElEXqZovL37Pr0uawvx01JD3PklMxAXDL4sN2XGmdQbsI+YMkp2ICrm0Q2c81qSHpeZacigm4smVxtn2QRPIHSU7FBFzZRD+9w/tuwkgSSc+z5FRMQFmkdlqtOz4i7VvvXgNVAUnPs+RUTEBZpM9EoWSn4uFut+lGhy3AqSHpeZacign4YUA2yu7skhf+CxJLz7PkVEwAEOn7vFNxMzxi6XmWnIoJQCW55f6G7i09grnV558ftKPpeZacigngr23fAUnPs+RUTIBECgiSnIoJ+HFmwxGncSRvkORUTMBNkTSzwR8kORUTcEukiUTyB0lOxQTkLelGkJbXQFVA0vMsORUTkBdpg0Xi3/+EpOdZciomoHDfNgQavfF7xNLzLDkVE3Cz2NAESHqeJadiAiRSQJDkVExA46RBkPQ8S07FBEikgCDJqZgAJNK83yrUG7wGqgKSnmfJqZgAYMnksnDnNVAVkPQ8S07FBJQtmZcq4F4DVQFJz7PkVExA2ZJ+MgbLv7w8D0nPs+RUTEBZJNeEuQxFSHqeJadiAuA40sZrhOoh6XmWnIoJ0IBsQJDkVExAWZo3iVQXJDkVE1CWZnjYq7hBkPQ8S07FBMB97VyzinYsPc+SUzEB+OhL16z6N0nPs+RUTAAsNmhAth5IciomQCIFBElOxQRIpIAgyamYAH5L7oCk51lyKiZAIgUESU7FBEikgCDJqZgAiRQQJDkVE4BFWr63XVpk6H57TVMRJD3PklMxAVCk9qlaF0X9639+O+o41xltDy13IvvEbLD/cDA7fvdF0xckPc+SUzEBeGZDTqTrJs0O2sSZHIsLkXqHRm+Hmt4g6XmWnIoJACK9RVFrclhNkezfcOXQvpVzo/Vutx64OL0mTd00/+Wxi6fb3XYauxFo+oOk51lyKiagLNIk2zb/MBDbv3pJ+nBfxw9SNwZunf+yOzRXLgZNf5D0PEtOxQSURWqfzmFOWsurp1HEh0eh3TpzIy4YMjtdeT7cotT0CEnPs+RUTMCPK2SPU4PunCKUPhRt3Uf+cyN3LCvM3LjU9AhJz7PkVEyAqUgL9/U1cC7+WKWfG7jV4asrNyg1PULS8yw5FRNwS6RNFLnSt5RZpNei6alml97Exec/mdz5XTQ9QtLzLDkVE1AWqVt4RhpGUUF3inUAABA9SURBVPeO/80gvW3bX42mSfVuMUhNcjlzXKl54I8QtFwTabK/Bm12ufL3HTs4LLJ7tfj0FNRLbuPuEskHJL9CWXIqJgCPI7lsHGme7Lr6dvt/so7jbfEzs+RWTyI9CElOxQQAkTaFsyju2QilUxw+SkiegiTSg5DkVEwALMnljjdv37Hras+VZ88lqnSK1YWOig03IMmpmIBrs7+j9vuV6UEnkEepSCp/PwhJTsUEPL0eaQ09WrtOMrfuPAI7KjU9QtLzLDkVE/CsSOs451HsjjWHr0SVsy/ppIaLpkdIep4lp2ICnhRp5uLVuTU6Tf3ppLPpjrNU11lx4aLpD5KeZ8mpmIC8SKclSHdvx7WO43y9bhu7cdJe9LIlR6PTuokxaPqDpOdZciom4DmRPlxxJd/+Ri/jMHlVC/segiSnYgKeE8ldiLTbTXvnSat7vopry7+01PwaJDkVE6BdhAKCJKdiAiRSQJDkVEyARAoIkpyKCcAiTVrpi+s24+w+kp5nyamYACTS5u28OrbVhBPHSHqeJadiAn7a1y774p2zv0OHpOdZciomAJ4hG3UPc74nrftWmgcOSc+z5FRMADzVvH1u7U0aeoxTDSQ9z5JTMQFlkfYPSLm7ucldS2QDh6TnWXIqJuDHXYR+aFJC0vMsORUTIJECgiSnYgLKlrQKt3Z37msXNiQ9z5JTMQFlkYrb5t+5r13YkPQ8S07FBJRFmue3sis0aCHpeZacigkAD0DJeX399MjL7zv3tQsdkp5nyamYALSvncsvRnJ37McVOiQ9z5JTMQHXzpBN97XzmqYiSHqeJadiAgz2tQsfkp5nyamYAP5Bojsg6XmWnIoJkEgBQZJTMQESKSBIciom4LldhEgg6XmWnIoJkEgBQZJTMQESKSBIciomIG/JZNKMancJkp5nyamYgLxIrQZcfCAkPc+SUzEB8NbOawIPkPQ8S07FBEikgCDJqZiAvDQu3ehEItUGSU7FBOSl6ZYLdqra+YQkp2IC8pYsJVK9kORUTEDBkvmbRKoTkpyKCbi5i1ATIOl5lpyKCSgPyEqk2iDJqZgADcgGBElOxQRoHCkgSHIqJkAiBQRJTsUEaEA2IEhyKiZAA7IBQZJTMQEakA0IkpyKCdCAbECQ5FRMgAZkA4Ikp2ICJFJAkORUTEDjpEGQ9DxLTsUESKSAIMmpmACJFBAkORUTcG0T/f0H3W+vaSqCpOdZciom4NqxLsmX88dg0kLS8yw5FRMARFq6KCdSE0wi6XmWnIoJACK9RVFrcqiCT/Yv/LtGkvQ8S07FBJRF2rvT2p2Gk/pNuCSR9DxLTsUElEVqZ+eYH0RaZlpxQ9LzLDkVE/DjzIbjBIeqJzr8EYKWgETyAcmvUJacigm4JdImipzXQFVA0vMsORUTUBapW3hGGkZR12ugKiDpeZacigmAVTu32eXK3xOvgaqApOdZciomAI8juWwcad7f//fNa55KIOl5lpyKCQAibVr55bFu6TVPJZD0PEtOxQTAklxuF5T2xmucaiDpeZacigm4Nvs7ar/zTw9KIOl5lpyKCeAfJLoDkp5nyamYAIkUECQ5FRMgkQKCJKdiArBI8366w127r2ckn5DkVEwAEmnTzlXtGlD9Zul5lpyKCbi2QlbjSJ4hyamYADyzIeqnm558a2aDV0hyKiYAzrU7T6/TXDufkORUTABcITs8t4b7xySPcaqBpOdZciomAK5Hyk0L2mhhnz9Iciom4OYm+hLJHyQ5FROgK1JAkORUTICekQKCJKdiAu6o2n16DVQFJD3PklMxAeC+rVUcR2p5zVMJJD3PklMxAZrZEBAkORUTgCoJy/xcO62Q9QdJTsUE/DD7+z11SStk/UKSUzEB/LXtOyDpeZacigmQSAFBklMxARIpIEhyKibgmkit9wZU7BJIep4lp2ICSiINW8ePvqMm7PudQNLzLDkVE3Ah0qeLouNB5u9NGUZi6XmWnIoJKIqUzGSI3o+t76QG7howkETS8yw5FRNQEOkzSvfPP5NMteOfs8rS8yw5FROQF2njSg9FOtXcJyQ5FROQF+kTzFBt61Rzf5DkVExAXqQ22OjkuwnbCJH0PEtOxQTkRSqujT1/0l+aiiDpeZacigm4EAl8g0TyBklOxQRIpIAgyamYgLwlb1FUGn6d6xnJHyQ5FROQF6kP9md4b8I0IZKeZ8mpmIC8SN+g/O20ZbE/SHIqJqDwANQqDRrt7/aczzjVQNLzLDkVE1AQaX6xScMyOZji+/KP8EHS8yw5FRNQLMkNk0mr3Uk6KWj+2S5MYSWGpOdZciom4KK2PYmKOP4HpB1Nz7PkVEzA5SDRppvXqN+ANRQ7mp5nyamYgPJo6+azeziJuRFXowSSnmfJqZgA/mkLd0DS8yw5FRMgkQKCJKdiAiRSQJDkVEyARAoIkpyKCZBIAUGSUzEBfkWaDZxzg5nXn7mj6XmWnIoJ8CpSz2X0fP7QHU3Ps+RUTIBPkcYunm5322nsRh5/6o6m51lyKibAp0jOrdPXlYs9/tQdTc+z5FRMgEeRZqcL0Ydb+PuxO5qeZ8mpmACPIo3cscowc2N/P3ZH0/MsORUT4FGkgVsdPlq5gb8fu6PpeZacignwKFJ8Xmvr+SGJpOdZciomwKNIzqEPfUDS8yw5FRNQu0j/E4KWgETyAcmvUJacigmQSAFBklMxAR5F6qjYcAOSnIoJUPk7IEhyKibAo0jj3ICs38l2JD3PklMxAbVMETrPcfADSc+z5FRMQB2TVteeaw0sPc+SUzEBPkUanZZR+J1qx9LzLDkVE6CFfQFBklMxAX6Xmn9pqfk1SHIqJkCbnwQESU7FBEikgCDJqZgAiRQQJDkVEyCRAoIkp2ICJFJAkORUTIBECgiSnIoJkEgBQZJTMQESKSBIciomQCIFBElOxQRIpIAgyamYAIkUECQ5FRMgkQKCJKdiAiRSQJDkVEyARAoIkpyKCZBIAUGSUzEBEikgSHIqJkAiBQRJTsUESKSAIMmpmACJFBAkORUTIJECgiSnYgJeQiQhqkYiCWGARBLCAIkkhAESSQgDJJIQBkgkIQyQSEIYIJFEs8kfxrWt7vBiiSSaTeFUu+qOuJNIotnk3ZlJJCEeZupKfFT1sySSaC69C43iyjySSKLheDr5WyKJZiORzPFUCBWvyKuK5OsX1b8wGq3rjtBAph3X+aruf/+qIlVYCH0a57Z1R2gU087uWMEbVfZDXkQkn4XQpwnYcUZ66b9n7Hq7VexWVf2UFxHJZyH0aUauwluQl2PmOvs75YVzi+RjjSMZQPOb/sONK/vN+XIMEoP2v5zSztdcOwtYRLq4eNYdh5zsH7DjBudGFbySSCxIJEvSf8Ctc9NToxJeTKTVqJP+Ww506/QqpP09c2mZYe06Vf2Y1xLp4/gr3rlZ3VmEHwaJQodHpI/q6t8vJdLAdWan31Aaq3kNvlxvfyVKynXbvU6VDXW/kkiz9MKe/W4auXHNaa6iW1A70pGPeJ0+e1Z3H/JKIg3Sf8dMpPX+91S46BbUklHsBsmVKB5VeBfySiIdhhJcrhEmHLegmgOcRyKFB8ktKMscYD+8kkhxTqSQf4WS3IKyzAE+PHG63qjKB85XEukj9wadBj9pNeQrJ9Uc4KT2XX3IVxJp5eLteYBuUXecHwlfJLI5wNnUxdW4SpNeSaT9P2mcPsSvRlWuTHkaklvQMBUvMUtL3ynruLpfny8l0m58+hUasEcst6AkIn1ks+xSKvzXfC2Rdttxr/LHzqchuQUlWRGfH0DQlsWvBcctaMBDXHm0ZfELQ3ELSnJrV7wiSaTXguEWlGRFvJ6RKmDc0YI5OzhWxBerdpVNXXwpkWKtPDWEZSHvh8aRjBknz/DCChaRDnPpNbPBjA7JmoSLSQODMUV1LGBWo+qfOF9JpJB/a+YpzWOLKQZsXhuJFCBfrjdLrkLbxWh/Ed2Ow50mJI68kkgf1W1Ya8o2dy8/TRzKV3DDZL0Y1B3hClpGYcvChdzbZ0a5LTq2iUOrMBclrT/y9591p/kZLaOwZuQGwc5cy9EpTGo5r5YNjG1+NKET7j+sllFYw1KvLc8OCzLs2HW26ZKP7VeFA51Po2UU5rCI1Mt19yK5q6twitgT9BJ7BtkOpnEcbIleU4Reli93flt2kglt0yAf7lK7p9nbdBryHi1aRvGq9Fw8TW5H1tPkeKy9WEHeOaUirTLHt2GWQxK0jOKFOZfDxukdaYgXJIatJRK0jOKVWY8HycjHOLkuuU6Yo0i99B0aZ7X6cEXSM5IIm1F6w5m9TxfVnZfyLFpGYc96pPVIdqxcvErfp6vdqhPwUl4to7BmrfVIpmSHDmXzBkKeDahlFMZ8uF644+8FSFbyztL7uWnHxeFejxK0jMIWRzJpVSt5CXktkepOcB9ayUvIK4nU49iIjWYlr597JgOymNWuNH4lkWYBl5bysFw5d4PT7WeQI8ZHzjErnMf0SiLtRu4j8N+dKSwiHWcyraZxyCblY1b3m/SlRFp1KB7iSVbyTs9za6sc6XyWr/NYsQZkbViRLKMgWcmbX+0xC/fQjGLMyv5lX0mkHss4EsdKXpIzZDX72xyS4xNYFiCSiNSRSNaE29lFSETq5R7lFuHe2uW3kllUt2zqlURiGUciIb+3USfcpeb5zc062rPBglW4vzYpmbnOV7qQN+jNT5KF+ul2m0nM6gaSXkmkpGhD8BDPggPUnamIz4SvJFLQnf4DAW9hKpHySKQAYdnCVJx5JZFYYNnCVOSQSOFBsoXpaKTTZs5IpPAg2cKUZXzbDxIpPGi2MK07wV0UH4wrO//wRUQKv8KUg2QL01Gym3L4lPq9mvMPJVJ4kGxhuvvItrkKHT/nH76ISFSQbGFK8nvJ0/mHEik8SLYwJRHJ0/mHEik8WLYw5cDT+YcSKUBYtjClwNP5h68kkqdCqAEsW5gy4On8w9cVqbJCqAgKT+cfvpJIngqhT6O5N6b4Of/wlUTyVAh9Gs29scXL+YevJJKnQujThFtKJsXH+YevJJKnQujTkMy9EXleSSRPhdDnIZl7I3K8kkieCqFPQzJlQOR5JZE8FUKfRiJZ0ilc3HWquQV+CqEiKPKHucxibX5igpdCqAiKRewOF6XtoMITkl5LJC+FUBEYH5k/XyejquDFRBKvSHJRWux/g1b5i1MiiRcguafvVTrxSiKJ5rNODmOu8L5uJ5HECzB1brD+qPQsZokkms6q4+JkztVCxQYh/pnx/nJ0GIev8qIkkUSzcfF51H2hAVkh/o2PKy1DJJIQBkgkIQyQSEIYIJGEMEAiCWGARBLCAIkkhAESSQgDJJIQBkgkIQyQSEIYIJGEMEAiCWGARBLCAIkkhAESSQgDJJIQBkgkIQyQSEIYIJGEMEAiCWGARBLCAIkkhAESSQgDJJIQBkgkIQyQSEIYIJGEMEAiCWGARBLCAIkkhAESSQgDJJIQBkgkIQyQSEIYIJGEMEAiCWGARBLCAIkkhAESSQgDJJIQBkgkIQyQSEIYIJGEMEAiCWHA/wGd/QnHGXrR5AAAAABJRU5ErkJggg==", "text/plain": [ "plot without title" ] }, "metadata": { "image/png": { "height": 420, "width": 420 } }, "output_type": "display_data" } ], "source": [ "g_ci = ggplot(result_plr, aes(x = ML, y = Estimate, color = ML)) + \n", " geom_point() +\n", " geom_errorbar(aes(ymin = lower, ymax = upper, color = ML)) +\n", " geom_hline(yintercept = 0, color = \"grey\") +\n", " theme_minimal() + ylab(\"Coefficients and 0.95- confidence interval\") + \n", " xlab(\"\") + \n", " theme(axis.text.x = element_text(angle = 90), legend.position = \"none\",\n", " text = element_text(size = 20))\n", "\n", "g_ci" ] }, { "attachments": {}, "cell_type": "markdown", "id": "6405d1e9", "metadata": { "papermill": { "duration": 0.056069, "end_time": "2021-03-23T13:49:05.604569", "exception": false, "start_time": "2021-03-23T13:49:05.548500", "status": "completed" }, "tags": [] }, "source": [ "### Interactive Regression Model (IRM)" ] }, { "attachments": {}, "cell_type": "markdown", "id": "89212f23", "metadata": { "papermill": { "duration": 0.055247, "end_time": "2021-03-23T13:49:05.714403", "exception": false, "start_time": "2021-03-23T13:49:05.659156", "status": "completed" }, "tags": [] }, "source": [ "Next, we consider estimation of average treatment effects when treatment effects are fully heterogeneous:" ] }, { "attachments": {}, "cell_type": "markdown", "id": "7cc1040d", "metadata": { "papermill": { "duration": 0.057453, "end_time": "2021-03-23T13:49:05.828088", "exception": false, "start_time": "2021-03-23T13:49:05.770635", "status": "completed" }, "tags": [] }, "source": [ "\\begin{eqnarray}\n", "& Y = g_0(D,X) + U, &\\quad E[U\\mid X,D] = 0,\\\\\n", "& D = m_0(X) + V, &\\quad E[V\\mid X] = 0.\n", "\\end{eqnarray}" ] }, { "attachments": {}, "cell_type": "markdown", "id": "dc13bc3d", "metadata": { "papermill": { "duration": 0.055141, "end_time": "2021-03-23T13:49:05.940426", "exception": false, "start_time": "2021-03-23T13:49:05.885285", "status": "completed" }, "tags": [] }, "source": [ "To reduce the disproportionate impact of extreme propensity score weights in the interactive model\n", "we trim the propensity scores which are close to the bounds." ] }, { "cell_type": "code", "execution_count": 16, "id": "7a2a1a88", "metadata": { "execution": { "iopub.execute_input": "2021-03-23T13:49:06.055682Z", "iopub.status.busy": "2021-03-23T13:49:06.054164Z", "iopub.status.idle": "2021-03-23T13:49:24.871519Z", "shell.execute_reply": "2021-03-23T13:49:24.869890Z" }, "papermill": { "duration": 18.876848, "end_time": "2021-03-23T13:49:24.871740", "exception": false, "start_time": "2021-03-23T13:49:05.994892", "status": "completed" }, "tags": [], "vscode": { "languageId": "r" } }, "outputs": [ { "name": "stdout", "output_type": "stream", "text": [ "Estimates and significance testing of the effect of target variables\n", " Estimate. Std. Error t value Pr(>|t|) \n", "e401 8850 1341 6.601 4.08e-11 ***\n", "---\n", "Signif. codes: 0 '***' 0.001 '**' 0.01 '*' 0.05 '.' 0.1 ' ' 1\n", "\n", "\n" ] } ], "source": [ "set.seed(123)\n", "# Initialize DoubleMLIRM model\n", "dml_irm_lasso = DoubleMLIRM$new(data_dml_flex,\n", " ml_g = lasso,\n", " ml_m = lasso_class,\n", " trimming_threshold = 0.01,\n", " n_folds = 3)\n", "dml_irm_lasso$fit()\n", "dml_irm_lasso$summary()" ] }, { "cell_type": "code", "execution_count": 17, "id": "d9d66ac9", "metadata": { "execution": { "iopub.execute_input": "2021-03-23T13:49:25.191466Z", "iopub.status.busy": "2021-03-23T13:49:25.189452Z", "iopub.status.idle": "2021-03-23T13:50:24.952570Z", "shell.execute_reply": "2021-03-23T13:50:24.951094Z" }, "papermill": { "duration": 59.82937, "end_time": "2021-03-23T13:50:24.953063", "exception": false, "start_time": "2021-03-23T13:49:25.123693", "status": "completed" }, "tags": [], "vscode": { "languageId": "r" } }, "outputs": [ { "name": "stdout", "output_type": "stream", "text": [ "Estimates and significance testing of the effect of target variables\n", " Estimate. Std. Error t value Pr(>|t|) \n", "e401 8202 1118 7.334 2.23e-13 ***\n", "---\n", "Signif. codes: 0 '***' 0.001 '**' 0.01 '*' 0.05 '.' 0.1 ' ' 1\n", "\n", "\n" ] } ], "source": [ "# Initialize Learner\n", "randomForest = lrn(\"regr.ranger\")\n", "randomForest_class = lrn(\"classif.ranger\")\n", "\n", "# Random Forest\n", "set.seed(123)\n", "dml_irm_forest = DoubleMLIRM$new(data_dml_base,\n", " ml_g = randomForest,\n", " ml_m = randomForest_class,\n", " trimming_threshold = 0.01,\n", " n_folds = 3)\n", "\n", "# Set nuisance-part specific parameters\n", "dml_irm_forest$set_ml_nuisance_params(\n", " \"ml_g0\", \"e401\", list(max.depth = 6, mtry = 4, min.node.size = 7))\n", "dml_irm_forest$set_ml_nuisance_params(\n", " \"ml_g1\", \"e401\", list(max.depth = 6, mtry = 3, min.node.size = 5))\n", "dml_irm_forest$set_ml_nuisance_params(\n", " \"ml_m\", \"e401\", list(max.depth = 6, mtry = 3, min.node.size = 6))\n", "\n", "dml_irm_forest$fit()\n", "dml_irm_forest$summary()" ] }, { "cell_type": "code", "execution_count": 18, "id": "d93208c7", "metadata": { "vscode": { "languageId": "r" } }, "outputs": [ { "name": "stdout", "output_type": "stream", "text": [ "Estimates and significance testing of the effect of target variables\n", " Estimate. Std. Error t value Pr(>|t|) \n", "e401 8415 1186 7.098 1.26e-12 ***\n", "---\n", "Signif. codes: 0 '***' 0.001 '**' 0.01 '*' 0.05 '.' 0.1 ' ' 1\n", "\n", "\n" ] } ], "source": [ "# Initialize Learner\n", "trees = lrn(\"regr.rpart\")\n", "trees_class = lrn(\"classif.rpart\")\n", "\n", "# Trees\n", "set.seed(123)\n", "dml_irm_tree = DoubleMLIRM$new(data_dml_base,\n", " ml_g = trees,\n", " ml_m = trees_class,\n", " trimming_threshold = 0.01,\n", " n_folds = 3)\n", "\n", "# Set nuisance-part specific parameters\n", "dml_irm_tree$set_ml_nuisance_params(\n", " \"ml_g0\", \"e401\", list(cp = 0.0016, minsplit = 74))\n", "dml_irm_tree$set_ml_nuisance_params(\n", " \"ml_g1\", \"e401\", list(cp = 0.0018, minsplit = 70))\n", "dml_irm_tree$set_ml_nuisance_params(\n", " \"ml_m\", \"e401\", list(cp = 0.0028, minsplit = 167))\n", "\n", "dml_irm_tree$fit()\n", "dml_irm_tree$summary()" ] }, { "cell_type": "code", "execution_count": 19, "id": "a409c687", "metadata": { "vscode": { "languageId": "r" } }, "outputs": [ { "name": "stdout", "output_type": "stream", "text": [ "Estimates and significance testing of the effect of target variables\n", " Estimate. Std. Error t value Pr(>|t|) \n", "e401 8048 1182 6.808 9.91e-12 ***\n", "---\n", "Signif. codes: 0 '***' 0.001 '**' 0.01 '*' 0.05 '.' 0.1 ' ' 1\n", "\n", "\n" ] } ], "source": [ "# Initialize Learners\n", "boost = lrn(\"regr.xgboost\", objective = \"reg:squarederror\")\n", "boost_class = lrn(\"classif.xgboost\", objective = \"binary:logistic\", eval_metric = \"logloss\")\n", "\n", "# Boosted Trees\n", "set.seed(123)\n", "dml_irm_boost = DoubleMLIRM$new(data_dml_base,\n", " ml_g = boost,\n", " ml_m = boost_class,\n", " trimming_threshold = 0.01,\n", " n_folds = 3)\n", "\n", "# Set nuisance-part specific parameters\n", "if (compareVersion(as.character(packageVersion(\"DoubleML\")), \"0.2.1\") > 0) {\n", " dml_irm_boost$set_ml_nuisance_params(\n", " \"ml_g0\", \"e401\", list(nrounds = 8, eta = 0.1))\n", " dml_irm_boost$set_ml_nuisance_params(\n", " \"ml_g1\", \"e401\", list(nrounds = 29, eta = 0.1))\n", " dml_irm_boost$set_ml_nuisance_params(\n", " \"ml_m\", \"e401\", list(nrounds = 23, eta = 0.1))\n", "} else {\n", " # behavior of set_ml_nuisance_params() changed in https://github.com/DoubleML/doubleml-for-r/pull/89\n", " dml_irm_boost$set_ml_nuisance_params(\n", " \"ml_g0\", \"e401\", list(nrounds = 8, eta = 0.1, objective = \"reg:squarederror\", verbose=0))\n", " dml_irm_boost$set_ml_nuisance_params(\n", " \"ml_g1\", \"e401\", list(nrounds = 29, eta = 0.1, objective = \"reg:squarederror\", verbose=0))\n", " dml_irm_boost$set_ml_nuisance_params(\n", " \"ml_m\", \"e401\", list(nrounds = 23, eta = 0.1, objective = \"binary:logistic\", eval_metric = \"logloss\", verbose=0))\n", "}\n", "\n", "dml_irm_boost$fit()\n", "dml_irm_boost$summary()" ] }, { "cell_type": "code", "execution_count": 20, "id": "c388da15", "metadata": { "execution": { "iopub.execute_input": "2021-03-23T13:50:25.095222Z", "iopub.status.busy": "2021-03-23T13:50:25.093677Z", "iopub.status.idle": "2021-03-23T13:50:25.130873Z", "shell.execute_reply": "2021-03-23T13:50:25.129138Z" }, "papermill": { "duration": 0.108326, "end_time": "2021-03-23T13:50:25.131041", "exception": false, "start_time": "2021-03-23T13:50:25.022715", "status": "completed" }, "tags": [], "vscode": { "languageId": "r" } }, "outputs": [ { "data": { "text/html": [ "\n", "\n", "\n", "\t\n", "\t\n", "\n", "\n", "\t\n", "\t\n", "\t\n", "\t\n", "\n", "
A data.table: 4 × 5
modelMLEstimatelowerupper
<chr><chr><dbl><dbl><dbl>
IRMglmnet 8850.0646222.39611477.73
IRMranger 8202.1016010.24510393.96
IRMrpart 8415.2766091.61810738.93
IRMxgboost8047.5985730.68410364.51
\n" ], "text/latex": [ "A data.table: 4 × 5\n", "\\begin{tabular}{lllll}\n", " model & ML & Estimate & lower & upper\\\\\n", " & & & & \\\\\n", "\\hline\n", "\t IRM & glmnet & 8850.064 & 6222.396 & 11477.73\\\\\n", "\t IRM & ranger & 8202.101 & 6010.245 & 10393.96\\\\\n", "\t IRM & rpart & 8415.276 & 6091.618 & 10738.93\\\\\n", "\t IRM & xgboost & 8047.598 & 5730.684 & 10364.51\\\\\n", "\\end{tabular}\n" ], "text/markdown": [ "\n", "A data.table: 4 × 5\n", "\n", "| model <chr> | ML <chr> | Estimate <dbl> | lower <dbl> | upper <dbl> |\n", "|---|---|---|---|---|\n", "| IRM | glmnet | 8850.064 | 6222.396 | 11477.73 |\n", "| IRM | ranger | 8202.101 | 6010.245 | 10393.96 |\n", "| IRM | rpart | 8415.276 | 6091.618 | 10738.93 |\n", "| IRM | xgboost | 8047.598 | 5730.684 | 10364.51 |\n", "\n" ], "text/plain": [ " model ML Estimate lower upper \n", "1 IRM glmnet 8850.064 6222.396 11477.73\n", "2 IRM ranger 8202.101 6010.245 10393.96\n", "3 IRM rpart 8415.276 6091.618 10738.93\n", "4 IRM xgboost 8047.598 5730.684 10364.51" ] }, "metadata": {}, "output_type": "display_data" }, { "data": { "image/png": "iVBORw0KGgoAAAANSUhEUgAAA0gAAANICAMAAADKOT/pAAAAbFBMVEUAAAAAv8RNTU1oaGh8fHx8rgCDg4OMjIyVlZWampqjo6Onp6evr6+ysrK5ubm9vb2+vr7BwcHHfP/Hx8fJycnQ0NDR0dHY2NjZ2dne3t7h4eHk5OTp6enq6urr6+vv7+/w8PD19fX4dm3////4p3w3AAAACXBIWXMAABJ0AAASdAHeZh94AAAgAElEQVR4nO2de0PaytdG08FyV16qqKAesXz/7/iScEtkC/zKzmSesNYfR8a2x6fdLJLMNVsBwNVkTQcAaAOIBODAv4n0FHavvqb9EPrTr117Pg4hjOeXNQHawj+JNA/h8Kqgu5VjuG0PL2kCtIZ/ESm3Z/PqI4Tp52r1OQ7d4pr0FLrPX6uv526Ynm8CtId/EOk57EV6CC+7F4UbIXwWzY/QPd8EaA//s0gf6/uz4U6k7u4W77NwY76/1DyEt3NNgBbxP4u0vho9rfbPSKVvr/8zDbt+hPn6N51pArSI/12k8efqJ5HG4WPb/Ajjc82I/Bf1p/07IjmJafA/i1R0KhyJ9BYeVqU7vfXv6J5rRkSk8io5iWnwb+NIRyKNi9u20rfzl6ebW/4DkMVbpLfNvdq/iBQDkY9QlZzENHAR6bPb/X7Hh0j/gEhOYhq4iNTfDhAh0nWI5CSmgYdIw12/dr/anXC6GRGRyqvkJKaBg0h7j+j+vhKRnMQ0uFqkz4NHq6fSkOv0XDMiIpVXyUlMg2tF+uwePCoJUsxiON2MiEjlVXIS0+BKkeah+1H5/qbX4XPzG0434yFSeZWcxDS4TqTPbvez/P3pfqHE0/lmPEQqr5KTmAbXifQQDhTfSHNhn0jlVXIS0+A6kcJ3kVYv1cXkp5uxEKm8Sk5iGtzE5icilVfJSUwDREoIkZzENECkhBDJSUwDREoIkZzENECkhBDJSUwDREoIkZzENECkhBDJSUwDREoIkZzENECkhBDJSUwDREoIkZzENECkhBDJSUwDREoIkZzENECkhBDJSUwDREoIkZzENECkhBDJSUwDREoIkZzENECkhBDJSUyDdEX6myZ1/pV5h3qCSO7U+u53hHeoJ4jkDiK5QkwDREoI3qGeIJI7iOQKMQ0QKSF4h3qCSO4gkivENECkhOAd6gkiuYNIrhDTAJESgneoJ4jkDiK5QkwDREoI3qGeIJI7iOQKMQ3KImU/EDVQHSCSK8Q0QKSE4B3qCSK5g0iuENNA35ILQCRXiGmASAnBO9QTRHIHkVwhpgEiJQTvUE/SEmm5eLyLEqROEMkVYhpYIs3u6LVrBN6hnjQu0oju74bgHepJ0yItvmnUeY0aqA4QyRViGhyLtL4gdZarkGWz1XKS5V/kQSRXiGlwLFKncKeXZZNVcXkKUfPUAiK5QkyDY5E2D0Xra1Evb91n2WPUQHWASK4Q0+AnkdaXok7eet8KJQ0iuUJMg59E2n1Z0WsXD96hnjQt0l2WLTdf3ovfgEjR4B3qSdMijTYddaNNb8MrIsWDd6gnTYs0y7Kw2H1ZdHhGigfvUE+aFim/qcu2Xwr0B5IQyRViGhgivYdCpPxLzihqnlpAJFeIaWA+AE062y/r+zr96xEiOUNMA/2ehAtAJFeIaYBICcE71BNEcgeRXCGmgTEg+7iMmiACiOQKMQ3MKUJtcwmRXCGmgSlS21xCpNXqV5LU+BduXKT3x7u2uYRIrtT69vejaZHWLB97O5eihqkLRHIFkQx+7LWbDTYuDVowIotIriCSwanu79dRYBehmCCSJ+mItCpWnCNSNBDJk3REmg24IkUFkTxJQ6Qlz0jxQSRPEhBpuesBb4NFK0RyBpEMjHGk+61FrVhBUYBIriCSATMbEgKRPElApFZZtEIkZxDJgNnfCYFInjQt0ihuD8N/Efgb44fcDr+aDpAOh/exuYl+RI+iwBXJFa5IBj9uWdwmEMkVRDLgipQQiORJ0yK9tmIruwqI5AoiGVgbRPay3uMiaoqaqVOk32lS498YkSx+GpDlMOYGqPXt7wciGSBSQiDS6k+SXBAckRICkTy55O3vh74lF4BIriCSASIlBCJ5gkjuIJIriGSASAmBSJ6kINL7fW9z2tjgNWqamkAkVxDJwBSpt++ta8csB0RyBZEMfjj68iBSG0xCJFcQycA+jLkz204Cn62/6M8WQiRXEMngWKS1O53VfjXFqA2XJERyBZEMjkVaPyDlS2S3Ir1vtNIGkVxBJIMfF/btpgYxRSgeiOQJIrmDSK4gksE5kZZZFqIGqgNEcgWRDI5FGlSekSZZNogaqA4QyRVEMjB77cJyVer+1t+5GJFcQSQDexwpbMaRFqN829WoeWoBkVxBJANDpGWnvKovvEfNUwuI5AoiGZhdcoODR7027F+MSK4gksGp2d9Z715/elAOIrmCSAb6g0QXgEiuIJIBIiUEInnStEjfpjKEFnTbIZIriGRwViSmCMUDkTxJS6QZIsUDkTxpTqRBZtKJGqgOEMkVRDIoi7S0RZpEDVQHiOQKIhlU7tsmhkZ3+h4hki+IZMCJfQmBSJ4gkjuI5AoiGbROGgtEcgWRDBApIRDJk+ZFWowqCyn0XUMkVxDJwLBk9r3jLmqgOkAkVxDJ4NiSxVEPeNRAdYBIriCSwbEl+fLySQuWxZZAJFcQyeBYpNCGuQxVEMkVRDIwx5HasLy8DCK5gkgGDMgmBCJ50rRId4jUFIjkSdMiTdqwJWQVRHIFkQyMq09oxV52ZRDJFUQysI++DO3q/0YkVxDJwOxsYEC2GRDJE0RyB5FcQSQDREoIRPKkaZFaCCK5gkgGiJQQiOQJIrmDSK4gkgEiJQQiedKcSJt+BTobGgORPEEkdxDJFUQyQKSEQCRPeEZyB5FcQSQDREoIRPIEkdxBJFcQyQCREgKRPEEkdxDJFUQyQKSEQCRPEMkdRHIFkQwQKSEQyRNEcgeRXEEkA0RKCETyBJHcQSRXEMngapHm4xD6069yM4znlzVjgUiuIJLBtSINw4Z5tTm8pBkNRHIFkQxskd7ve6GY9T14Pf3Hh6GfK/Q2DG958yl0n79WX8/dMD3fjAciuYJIBqZIvf3yiSwbnfrT89DdvhoWL0L4LFof4YJmPBDJFUQysHdaLYl00qTx7pZu9Za/mu8vNQ/5Fep0MyKI5AoiGRgi3WVZZ7Zd5ZefJ7v4+U+HsO9lyC2Z7r2ah6dzzYggkiuIZHAs0tqdzmp/TNLo5CUphMPLcX6B+ti2Ps43I4JIriCSwbFIvc2xLluR3jda/UBZpPXLbqnZPdeMCCK5gkgGP57Yt9ur4eSeDeP9w848F+mbV6ebEUEkVxDJ4DqRXvZDQv1/Fem/CPyN8UMc+N10gMv41XSAy/gT4WdcLtIyy05dPYZhmF+T5v0uV6Tr4YrkSdNXpEHlGWmSZYNTf347WaH/iUjXg0ieNC3SbH0NWq5K3d+nT5R9Was0fF67Mczv7/bfzrsTTjcjgkiuIJKBPY4UNuNIi9H6v3cX/X/ewgPd39eCSJ40LtKyU95m9cKDmafhJZ9MdxhynZ5rxuPvXxGTEMmTxkUqHpN29Jan/nR3P7OheHUQZFqdMWQ1o/H3r4pJiORJAiJtZ39nvfsT04Nypru5Pi8bSXbTUj83vQmnm5H4+1fGJETyJAWRLmWtxNP6SvQxDd3i0jTdL5R4Ot+MBCJ5g0gGVy7sm4dd9/emneDCPkTyBpEMbJFmneJLGJzu+l7zOe2uxXjZt1+qi8lPN+Mg4xEiudK8SMu7w/SgzmWddkmj4hEiudK4SMXCvs0vXt79nTQiHiGSK42LtPZosO30nnVOT7UTAZFcQSSDY5EmWdY7tNYmTSLGqQdEcgWRDI5FWj8gle7mZpfOEUoZRHIFkQx+XEbxQ1MSRHIFkQwQKSEQyZOmRepUbu3OLOzTAJFcQSSDY5Gq+wadW9gnASK5gkgGxyItymv5FmcX9imASK4gkoHxAJRvWDwq9vx+vXxhX9IgkiuIZGAt7AuVhX0nFyRpgEiuIJLBqU30i4V9UdPUBCK5gkgG1y3sEwGRXEEkA/1BogtAJFcQyQCREgKRPEEkdxDJFUQysERajCobcum7hkie/PqlYVLjIs2yb0QNVAeI5MivXyImNS3S4rtHiBQLBZF+/VIxqWmR8tkMkxasLy+BSH4gks2xSKENa2KrIJIfiGRjrkdqwaygCojkiIpHKYgUNUEEEMkTEY8aF+kOkZpCQyTGkSzMXYT0VyBVQSRXEMnA3NeuDZtClkEkVxDJwN5pNbSr/xuRXEEkA7OzgQHZZkAkTxDJHRGRfv/WMAmRDBApGX7/FjEJkQz0LbkACZF+/1YxCZEMECkVEMkZRHIHkVxBJANESgYVjxDJ4tQuQusXg9eoaWpCQyR67XxJQKTevreusg+4LCIiMY7kSuMivYesJFIbTEIkVzRE+vMnqkmGSHdZ1pltV1Pk+zfobxKJSK5IiPTnT1yTjkVau9NZ7ZcljdpwSUIkVxRE+vMnsknHIvU2yyi2Ir1vtNIGkVxBJIMfV8jupgYxRSgeiOQGItUBIrmiIFICz0hVkZacIRsPRHKk8V67QeUZiTNkI4JInjQ9jjTbnNJ36P7W38EBkVxBJAN7HClsxpEWnCEbFUTypHGRlpWzKNqwEQoiuYJIBmaX3ODgUa8Nu64ikiuIZMAZsgmBSJ6kIFLLQCRXEMkAkRICkTxBJHcQyRVEMiiLdLwRF9txRQWRPEEkdxDJFUQyQKSEQCRPmn9G6mXZ4DUfP8pnNtxHjVMPiOQKIhnYU4T20+vemSIUEUTypGmRJpXDmCdtOJoZkVxBJINjkTqVw5iXLDWPByJ50rRI33oX6GyIByJ5kpZIS0SKByJ50rRIve/PSL2IceoBkVxBJINjkR7La5BYIRsTRPKkaZHy3oZsVKyfeB2wQjYmiORJ4yLt9/4uaIFHiOQLIhlYPQnL3sEj/f2KV4jkDCIZ/LhCNt+uoXffhoXmiOQMIhno921fACK5gkgGiJQQiOTJjYn0XwT+xvghDvxuOsBl/Go6wGX8ifAzDu/jb+uRVtaqpNhuucMVyRWuSAaIlBCI5AkiuYNIriCSQdmS2awd+0EegUiuIJJBWaROCy4+JojkCiIZmLd2URNEAJFcQSQDREoIRPKkOZFCsRIJkRoDkTxpTqTBcYcdvXYxQSRPmhPpHZGaBZE8aXCK0OIOkZoEkTxpeq5dG8z5BiK5gkgGiJQQiORJ0yK1EERyBZEMECkhEMkTRHIHkVxBJANLpMWoQ69dEyCSJ42LNKP7uyEQyZOmRVowjtQUiORJ0yKN1upM3o3fqgsiuYJIBscihTYcLVYFkVxBJANzQLYd20IeQCRXEMmAmQ0JgUieNC3SHSI1BSJ50rRIkzaciFQFkVxBJAPj6hNKB421A0RyBZEM7PORQrv6vxHJFUQyMDsbGJBtBkTyBJHcQSRXEMkAkRICkTxpWqQWgkiuIJIBIiUEInmCSO4gkiuIZIBICYFInqQg0mJU7HDXG7XjnBdEcgWRDCyRlr1Dj12vDSOziOQKIhnYMxtKtGG6ECK5gkgGhkj5Xd3oNX/1mq+WvYuapxYQyRVEMjgWKd/7ZGY2ZEEkVxDJ4FikXmWp+WT9mBQxTj0gkiuIZHBuqfmSKULxQCRPEhDpRFMSRHIFkQy4IiUEInnStEg8IzUGInnStEhHvXaPUQPVASK5gkgGxn1bpzqO1ImapxYQyRVEMmBmQ0IgkieNi7R6L8+1a8Ouq4jkCiIZ/DD7+75wqXfP7O+YIJInKYjUMhDJFUQyQKSEQCRPUhBp1im+hIH+hNUcRHIFkQzMhX27ffTzzu8WdNohki+IZPBT9/fmF+n+jgoiedK4SGuPBttO71lnbVLUPLWASK4gkoF5rEtpdl2nDQdhIpIriGRgHjRWupubtWGtOSK5gkgGrEdKCETyBJHcQSRXEMng2JJO5dZu2YbeBkRyBZEMjkUaZdno0Jpk2SBinHpAJFcQyeBYpEV5YV+lIQsiuYJIBsYDUI8NIhsCkTxpXKRldWFfCxYkIZIriGRgdsmVF/ZFTVMTiOQKIhnYfdvv973Awr7oIJInKYjUMhDJFUQyQKSEQCRPEGnD3zSp86+MSJ4gkjv/NR3gQhDJE0RyB5FcQSQDREoIRPJETaTnYQjd6ee2NR+HEMbz1UXNWCCSK4hkcK1In/2wYWPHcNsari5oRgORXEEkg2tF6oaHj7VODyHk16Sn0H3+Wn09d8N0dbYZD0RyBZEMrhRpGh42Lx4KOTY6rVYfobs624wHIq3f/UlS419YSqSvEL52r9ZyzPeXmofwdq4ZEURyReSfMxWRLllj/lK9SZuGXT/CPDyda0ZEpPKI5IqSSN8uLePwsX31EcbnmhERqTwiuaIk0nh9Z5d3fw9fimb3sL1Dfqd3uhkRkcojkitKIoWw6/4ebpqlXznX3PKfOL/TpOl/lgT4E+Fn+InU786/Vqu3cWHSv4gUA5GPUJWcIjG1rkjdba/dOLwg0rWI5BSJqSXSriPuLe8/QKTrEMkpElNLpMrLfrU74XQzIiKVV8kpElNJpHFVJLq/r0Mkp0jMVES6hGnYzfouLjJPpSHX6blmREQqr5JTJKaSSPPwvH31lk+6OwhSzGI43YyISOVVcorETEOkWb633d39ud0h+7tL0rBwYzct9XPz8HS6GQ+RyqvkFInZnEiHp6L3u90Gkfen//hb6OYCvQ03Tz3T/UKJp/PNeIhUXiWnSMwERCpvWnxm6+/5/7KSj4V9pxHJKRIzAZHW16PObH1X9z4JZ69JX9P+2oz9M89LdTH56WYsRCqvklMkZvMizdYebb/1vjZJf9tikcqr5BSJ2bxIg9KZSJPKsWOiiFReJadIzOZFWl+F9r117204IEmk8io5RWI2L1JlTkMbDmMWqbxKzjpj/kmSC4IjUkKI5CSmgSXSXVUk/VPNRSqvkpOYBpZIk9Iz0qwNh/aJVF4lJzENvon0WPR1h+x1971OG041F6m8Sk5iGnwTaU1nMBlsr0LLWTgMKQkjUnmVnMQ0OBZpw/4b4T1qnloQqbxKTmIaVLrkFo+bQ5hLInVa4JFK5VVyEtPA6NsudNr8Yk//+ShHpPIqOYlpoD9IdAEilVfJSUwDREoIkZzENDgS6fW+19ksROpNXq0/IYhI5VVyEtPgm0iLTrnrLguTqGHqQqTyKjmJaVAV6T77Dr12ERHJSUyDikiz/Br0nq+MzRfGLmcDxpGiIpKTmAYVkcJem/csK+7qRm1YjqRSeZWcxDQoizSprowtJq72mGsXD5GcxDQoi3RXXRlbLDFf3+0NogaqA5HKq+QkpoG9r13R6ORflm2YtSpSeZWcxDQ4IZKxXFYUkcqr5CSmQdmSkGX7LrolIsVHJCcxDcqWDEq7QT5uu+veubWLh0hOYhqURZqtrz7bWUH5xpBF//eAfe3iIZKTmAaV+7Z86/zRYn1f9xg2e54s8yFZdlqNhUhOYhpURHovbZ5fPC7lX/UvSCqVV8lJTINqT8LhOJfNHLt2eKRSeZWcxDT43iU3G3TWd3W7lbFhoH9ft5KpvEpOYhro921fgEjlVXIS0wCREkIkJzENECkhRHIS0wCREkIkJzENzorEFKF4iOQkpgEiJYRITmIaIFJCiOQkpoG+JRcgUnmVnMQ0QKSEEMlJTANESgiRnMQ0QKSEEMlJTIMjkd4fB73NvNXeYNKKmXYylVfJSUyDbyLNqlsWZ51W7FksUnmVnMQ0qK5H+qZRsZd+1Dj1IFJ5lZzENDhaIdu5f93e0C0Wk047TBKpvEpOYhp837Ph2zq+fFd9dlqNhUhOYhp820XoaFdV63tyiFReJScxDcoidUr72u143+yCoo1I5VVyEtPgx51WT35TDJHKq+QkpgEiJYRITmIa/LRl8Q5u7SIikpOYBnQ2JIRITmIafO/+fqz+8oTu74iI5CSmwfkB2U7UPLUgUnmVnMQ0qPYk3B1PEWrDseYilVfJSUyDb11yEyatNohITmIaHPVtLybbZRShN5i04GqUI1J5lZzENNAfJLoAkcqr5CSmASIlhEhOYhogUkKI5CSmASIlhEhOYhogUkKI5CSmASIlhEhOYhogUkKI5CSmASIlhEhOYhogUkKI5CSmASIlhEhOYhogUkKI5CSmASIlhEhOYhp827PBpNYA/wHIkpBIMRD5CFXJSUwDREoIkZzENLAs6WXZ4HW5frEYZdl91Dj1IFJ5lZzENDBEuivtd/LOJvoREclJTINjkSZZNvmpJYpI5VVyEtPgWKROli0PrWUbthESqbxKTmIaHIv0rXeBzoZ4iOQkpsE5kZaIFA+RnMQ0OLak9/0ZqRcxTj2IVF4lJzENjkV6zLKw34ZrxpbFERHJSUwD474t3yNyVGxb/DrgDNmYiOQkpoEh0nvgVPNmEMlJTAOrJ2HZO3g0Mn5dDpHKq+QkpoHdJfd+3ys2Lb5fmr+shkjlVXIS00C/b/sCRCqvkpOYBoiUECI5iWmASAkhkpOYBpZIi1H1lKSogepApPIqOYlpYFgyY2FfQ4jkJKbBsSULVsg2hUhOYhocWzJaq9OWo/q2iFReJScxDY5FCm1YyldFpPIqOYlpYC6jaMcw7AGRyqvkJKbB2YV9bUCk8io5iWlwLM0dIjWFSE5iGpibn+ivQKoiUnmVnMQ0MK4+obSwrx2IVF4lJzEN7PVIoV393yKVV8lJTAOzs4EB2WYQyUlMA0RKCJGcxDRApIQQyUlMA31LLkCk8io5iWmASAkhkpOYBoiUECI5iWmASAkhkpOYBudEWi4e9Xe2E6m8Sk5iGlgize7otWsEkZzENDAsGdH93RAiOYlpcH6peec1aqA6EKm8Sk5iGphLzTvLfJ3sbLWcZK2YCi5SeZWcxDQwj76c7U9JWl+eQtQ8tSBSeZWcxDT4aYXs7oCx+yx7jBqoDkQqr5KTmAY/ibTYHsL8zol98RDJSUyDH/ds2PXW0WsXD5GcxDQw92xYbr4Ui/sQKR4iOYlpYPbazTZf8t6GV0SKh0hOYhocWzLLsrDYfVl0eEaKh0hOYhoYl5u7zUVoP09IfyBJpPIqOYlp8MNhzKvDmcwtOEVWpPIqOYlpYD4ATTrbL+v7Ov3rkUzlVXIS00C/J+ECRCqvkpOYBoiUECI5iWmASAkhkpOYBoiUECI5iWmASAkhkpOYBoiUECI5iWmASAkhkpOYBoiUECI5iWmASAkhkpOYBoiUECI5iWmASAkhkpOYBqdEGoWsM4mWpEZEKq+Sk5gGlkjLUSf/sllHob9hsUzlVXIS0+DnZRST7XIk/XV9KpVXyUlMA/NU80KkkF+NFuv/LqIGqgORyqvkJKaBvdT8vdiHK3strkv6K/tEKq+Sk5gGxyINNovL7zfXpeV2fztpRCqvkpOYBuaWxdvtuIp+BnYRiodITmIanNogclRqSiNSeZWcxDQ4sWXxZvsgRIqHSE5iGpzYRL+4w3ttw0iSSOVVchLT4FikXtFbt3tEWrfuowaqA5HKq+QkpsGxSI+5QvlOxZPVajnItluA/8jzMIT+9GvXnI9DCOP5Zc1YiFReJScxDX4YkM02d3b5l5MXpM9u2LCVY7htDi9pRkOk8io5iWlgiPR62Kn4nEerbnj4WK2+nkP4yJtPofv8tW52w/R8Mx4ilVfJSUwDq0vufX1Dd1ccwdwZnZ4f9BQeNi9ewjj/EsJn0fwI3fPNeIhUXiUnMQ2u69vuh93DUciPmp3vLzUP4e1cMyIilVfJSUwDr0GiQqTp7lFpLc3TuWZERCqvkpOYBj/ObNgRLhpHei6uNuPNk9Iqv3sbn2tGRKTyKjmJaXBWpEtmNrw9hG5+j9cN+2/lj0GnmxERqbxKTmIanBNpdl6kokP7a/Oq9M1zzS3/AchiizTITDqnPfoYjvMx2c9/FCkGIh+hKjmJaVAWaWmLdMH+J1+beztEug6RnMQ0qNy3TQyN7i7bR+gh721ApOsQyUlMg7OdDRfyGfr5sNK+nXcnnG5GRKTyKjmJaeAlUnGRofv7OkRyEtPAbdVeLtJTach1eq4ZEZHKq+QkpsF1InX3U4Te8ovMQZBiFsPpZkREKq+Sk5gGlkiLUafS3/Dzn57uJq2uhoUbu2mpn5vehNPNeIhUXiUnMQ0MS2bfO+5+/tNf3fCQy/E23Bg13S+UeDrfjIdI5VVyEtPg2JLFUQ/4iT++X9i3vWtjYd81iOQkpsGxJaN8DPb08vIy+VLz7sOuP271Ul1MfroZC5HKq+QkpsGxSOGiuQxSiFReJScxDcxxpGXUCPUjUnmVnMQ0cBuQTRmRyqvkJKbBsTR3iNQUIjmJaXAszWS7V3GLEKm8Sk5iGpj72oXLO+0kEKm8Sk5iGthHX4b/of9bAJHKq+QkpoHZ2XD5gKwGIpVXyUlMA0RKCJGcxDRApIQQyUlMA31LLkCk8io5iWmASAkhkpOYBoiUECI5iWmASAkhkpOYBrZI7/e9UHQyDF6jpqkJkcqr5CSmgSlSb99bl2WjqHHqQaTyKjmJaWDPbCiJ1AaTRCqvkpOYBoZId1nWmW1XU+T7N5w+tE8Bkcqr5CSmwbFIs822+duB2FEbLkkilVfJSUyDY5F6+3OY89b72dMoBBCpvEpOYhr8uEJ2NzWIKULxEMlJTANESgiRnMQ0OCfSMssib4taAyKVV8lJTINjkQaVZ6RJlg2iBqoDkcqr5CSmgdlrF5arUve3/g4OIpVXyUlMA3scKWzGkRb5rqt3UfPUgkjlVXIS08AQaVk5i6ING6GIVF4lJzENzC650vHmvTbsuipSeZWcxDQ4Nfs7693rTw/KEam8Sk5iGugPEl2ASOVVchLTAJESQiQnMQ0QKSFEchLToCzSfgkS23E1g0hOYhogUkKI5CSmASIlhEhOYhroW3IBIpVXyUlMA0RKCJGcxDRApIQQyUlMA1ukWaf4Egb6M79zRCqvkpOYBpZIy7vD6thOC+asqlReJScxDX7a127zi8z+jopITmIamGfIZoPtnO9Zpw0rzVUqr5KTmAbmqea9Q2tt0iRinHoQqbxKTmIaHIu0fkAq3c3N2rBEVqTyKjmJafDjLkI/NCURqbxKTmIaIFJCiOQkpsGxJZ3KrR372kVEJGJX8+UAAA4YSURBVCcxDY5Fqm6bz752ERHJSUyDY5EW5a3sKg1ZRCqvkpOYBsYDUH5e36g48vKVfe2iIpKTmAbWvnahvBgptGA/LpHKq+QkpsGpM2SLfe2ipqkJkcqr5CSmAfvaJYRITmIa6A8SXYBI5VVyEtMAkRJCJCcxDRApIURyEtOAXYQSQiQnMQ0QKSFEchLTAJESQiQnMQ3Klsxm7ejtPkKk8io5iWlQFqnTgouPiUjlVXIS08C8tYuaIAIilVfJSUwDREoIkZzENChLE4qNThCpMURyEtOgLM3guMOOXruYiOQkpkHZkndEahaRnMQ0qFiyuEOkJhHJSUyDs7sItQGRyqvkJKbB8YAsIjWGSE5iGjAgmxAiOYlp0Pg40n8AsiQkUgxEPkJVchLTgAHZhBDJSUwDBmQTQiQnMQ0YkE0IkZzENGBANiFEchLTgAHZhBDJSUwDREoIkZzENGidNBYilVfJSUwDREoIkZzENECkhBDJSUyDU5vor18MXqOmqQmRyqvkJKbBqWNd8l8uH4Mpi0jlVXIS08AQ6T1kJZHaYJJI5VVyEtPAEOkuyzqzbS/4bP1Ff9dIkcqr5CSmwbFIa3c6q/1w0qgNlySRyqvkJKbBsUi9zTnmW5HeN1ppI1J5lZzENPhxZsNugkMbJjqIVF4lJzENECkhRHIS0+CcSMssC1ED1YFI5VVyEtPgWKRB5RlpkmWDqIHqQKTyKjmJaWD22oXlqtT9PYsaqA5EKq+Sk5gG9jhS2IwjLUbr/95FzVMLIpVXyUlMA0OkZae8PDa8R81TCyKVV8lJTAOzS660C0pvGTVOPYhUXiUnMQ1Ozf7Oevf604NyRCqvkpOYBvqDRBcgUnmVnMQ0QKSEEMlJTANESgiRnMQ0sEVajIod7nojnpFiIpKTmAaWSMteqdeuBb3fKpVXyUlMg1MrZBlHioxITmIa2DMbslGx6ckrMxuiIpKTmAbmXLvD9Drm2sVEJCcxDcwVspNDa7J+TIoYpx5EKq+Sk5gG5nqk0rSgJQv74iGSk5gGZzfRR6R4iOQkpgFXpIQQyUlMA56REkIkJzENLui1e4waqA5EKq+Sk5gGxn1bpzqO1ImapxZEKq+Sk5gGzGxICJGcxDSwehLey3PtWCEbD5GcxDT4Yfb3feESK2TjIpKTmAb6fdsXIFJ5lZzENECkhBDJSUwDREoIkZzENDglUue+BT12OSKVV8lJTIMjkSad3avXrA37fueIVF4lJzENvon0GLJsd5D5fVuGkVQqr5KTmAZVkfKZDNn9rvWa94GHFgwkiVReJScxDSoiPWbF/vkH8ql2+nNWVSqvkpOYBmWRluHooYhTzWMikpOYBmWRHo0Zqj1ONY+HSE5iGpRF6hkbnby2YRshkcqr5CSmQVmk6trYwzfjpakJkcqr5CSmwTeRjN+ASNEQyUlMA0RKCJGcxDQoW3KXZUfDrwuekeIhkpOYBmWRRsb+DPdtmCYkUnmVnMQ0KIv0anR/B7YsjodITmIaVB6AOkeDRuu7vRAzTj2IVF4lJzENKiItvm3S8J4fTPH6/Y/oIVJ5lZzENKh2yU3ySauDWTEpaPHYq0xhFUak8io5iWnwrW97llUJ+g9IK5nKq+QkpsH3QaLloKzRqAVrKFYylVfJSUyD49HW5eNgexJzK65GOSKVV8lJTAP9aQsXIFJ5lZzENECkhBDJSUwDREoIkZzENECkhBDJSUwDREoIkZzENIgr0nwcQhjPo/7MlUzlVXIS0yCqSMOwYRjzh65kKq+Sk5gGMUV6Ct3nr9XXczdMI/7UlUzlVXIS0yCmSCF8Fl8/QjfiT13JVF4lJzENIoo031+IHsJbvB+7kqm8Sk5iGkQUaRp2vQzz8BTvx65kKq+Sk5gGEUUah4/tq48wjvdjVzKVV8lJTIOIInUPa20jPySJVF4lJzENIooUgvUyBiKVV8lJTIPGRfo/AFkSEikGIh+hKjmJaYBICSGSk5gGEUXq09lwBpGcxDSg+zshRHIS0yCiSE+lAdm4k+1EKq+Sk5gGjUwROsxxiINI5VVyEtOgiUmrn5H7GlQqr5KTmAYxRZrul1HEnWqnUnmVnMQ0YGFfQojkJKZB3KXmLyw1P4VITmIasPlJQojkJKYBIiWESE5iGiBSQojkJKYBIiWESE5iGiBSQojkJKYBIiWESE5iGiBSQojkJKYBIiWESE5iGiBSQojkJKYBIiWESE5iGiBSQojkJKYBIiWESE5iGiBSQojkJKYBIiWESE5iGiBSQojkJKYBIiWESE5iGiBSQojkJKYBIiWESE5iGiBSQojkJKYBIiWESE5iGtyESAB1g0gADiASgAOIBOAAIgE4gEgADiASgAOIBOAAIkG7KR/G9VXf4cWIBO2mcqpdfUfcIRK0m7I7c0QC+J95Dkc81PWzEAnay/CbRt3aPEIkaDmRTv5GJGg3iOROpI5QuEVuVaRYH1T/wnT62XSEFvLcD/2X+v73typSjR2hVxPCV9MRWsVzf7XrwZvW9kNuRKSYHaFXk7DjigyLf89uGK4+uuGjrp9yIyLF7Ai9mmmo8Rbk5piH/vpO+S2Et/w140gOyHzSP4Sn2j45b45xbtD6w6koPnPtPFAR6dvFs+k44mz+AfthfGjUwS2JpAIieVL8A36F8Lxv1MKNifQx7Rf/lmNunW6Fot7zUHQzfIZ+XT/mtkR62H3EhzBvOgvEYZwrtH1Eeqiv//umRBqH/nz/CcVYzW3wEobrK1HeXfe11qm2oe5bEmleXNg3n03T8NRwmpNwC+pHMfLR/SyePeu7D7klkcbFv+NGpM/151S6cAvqybQbxvmVqDut8S7klkTaDiWEUiNNNG5BmQNcBpHSQ+QWVGUOcBxuSaRuSaSUP0JFbkFV5gBvnzjDcFrnA+ctifRQeoM+Jz9pNeUrp9Qc4Lzvu/6QtyTSR+h+HQbo3pqO8yPpiyQ2B3gzdfHjqU6Tbkmk9T9pt3iI/5jWuTLlakRuQdNU/Ih50fVd8Nmt7+PzpkRaPe0/QhP2SOUWVESkh80su4Ia/zVvS6TV19Ow9sfOqxG5BRVZEV8eQGDL4ttC4xY04SGuMmxZfMNI3IKK3NpVr0iIdFso3IKKrIjnGakGnvosmPNDY0V8tdeutqmLNyVSl5Wnjqgs5H1gHMmZp/wZHrxQEWk7l56ZDW70RdYkfJs0MH6S6B1LmI9p/U+ctyRSyp+aZY7msXUlBmxuG0RKkJcwnOdXoa+36foi+vWU7jQh2HFLIj3Ut2GtK1+le/nn3KFyD26afL6Nm45wApZR+PIWUq72gWlpi46v3KGPNBclfT6U7z+bTvMzLKPwZhrGyc5cK9GvTGo5rJZNjK/yaEI/3X9YllF4o9Jfezw7LMmwT6H/VSz5+HqpcaDzalhG4Y6KSMNSud/yu7oap4hdwTC3Z7zZwbTbTbaLnilCN8tLOLwt+/mEtuckH+4Ku583b9PnlPdoYRnFrTIM3ef8duTzOT8eay1WkndOhUgfG8e/0uwOyWEZxQ1z6A57Ku5IU7wgKWwtkcMyilvm82mcj3w85del0E9zFGlYvEO7m776dEXiGQnSZlrccG7ep2/1nZdyLSyj8OdzynokPz5C96N4n36sPvoJL+VlGYU3n6xHcmVz6NBm3kDKswFZRuHMQximO/5eQWQl77y4n3vuh26616McllH4EkQmrbKSV5DbEqnpBJfBSl5BbkmkocZGbDIreePcMzmwiVnvSuNbEmmecNdSGZUr52q8v/1McsR4xyFmjfOYbkmk1TQ8JP7ZWaAi0m4m08dzN2WTyjHr+yS9KZE++hIP8SIreZ8Pc2vrHOm8lpfDWDEDsj58iCyjEFnJW17tMU/30IxqzNr+ZW9JpKHKOJLGSl6RM2SZ/e2OyPEJKgsQRUTqI5I36Ra7iohIw9Kj3Fu6t3blrWTe6ls2dUsiqYwjiVDe26if7lLz8uZmffZs8OAj3Y9NSeah/1Is5E1685N8oX6x3WYes76BpFsSKe+0EXiIVyEYNJ2pSsyEtyRS0kX/gYS3MEWkMoiUICpbmMKBWxJJBZUtTKEEIqWHyBam0ymnzRxApPQQ2cJUZXw7DoiUHjJbmDad4CKqD8a1nX94IyKl38NUQmQL02m+m3L6HNW9nvMPESk9RLYwXT1strlKnTjnH96ISFKIbGEq8rkU6fxDREoPkS1MRUSKdP4hIqWHyhamGkQ6/xCREkRlC1MJIp1/eEsiReoIdUBlC1MFIp1/eLsi1dYRCkkR6fzDWxIpUkfo1TD3xpU45x/ekkiROkKvhrk3vkQ5//CWRIrUEXo16XYlixLj/MNbEilSR+jViMy9gTK3JFKkjtDrEZl7AyVuSaRIHaFXIzJlAMrckkiROkKvBpE86Vcu7pxq7kGcjlBIivJhLvMum5+4EKUjFJLirRu2F6WvcY0nJN2WSFE6QiExHjb+vOyNqoMbEwlukfyi9Lb+BK3zgxOR4AbI7+mHtU68QiRoP5/5Ycw13tetEAlugOcQxp8PtZ7FjEjQdj76oZvPuXqjswHgn3laX4624/B1XpQQCdpN6B5G3d8YkAX4Nx5OtBxBJAAHEAnAAUQCcACRABxAJAAHEAnAAUQCcACRABxAJAAHEAnAAUQCcACRABxAJAAHEAnAAUQCcACRABxAJAAHEAnAAUQCcACRABxAJAAHEAnAAUQCcACRABxAJAAHEAnAAUQCcACRABxAJAAHEAnAAUQCcACRABxAJAAHEAnAAUQCcACRABxAJAAHEAnAAUQCcACRABxAJAAHEAnAAUQCcACRABxAJAAH/h+gZVuOAnrJwgAAAABJRU5ErkJggg==", "text/plain": [ "plot without title" ] }, "metadata": { "image/png": { "height": 420, "width": 420 } }, "output_type": "display_data" } ], "source": [ "confints = rbind(dml_irm_lasso$confint(), dml_irm_forest$confint(),\n", " dml_irm_tree$confint(), dml_irm_boost$confint())\n", "estimates = c(dml_irm_lasso$coef, dml_irm_forest$coef,\n", " dml_irm_tree$coef, dml_irm_boost$coef)\n", "result_irm = data.table(\"model\" = \"IRM\", \n", " \"ML\" = c(\"glmnet\", \"ranger\", \"rpart\", \"xgboost\"), \n", " \"Estimate\" = estimates,\n", " \"lower\" = confints[,1],\n", " \"upper\" = confints[,2])\n", "result_irm\n", "\n", "g_ci = ggplot(result_irm, aes(x = ML, y = Estimate, color = ML)) + \n", " geom_point() +\n", " geom_errorbar(aes(ymin = lower, ymax = upper, color = ML)) +\n", " geom_hline(yintercept = 0, color = \"grey\") +\n", " theme_minimal() + ylab(\"Coefficients and 0.95- confidence interval\") + \n", " xlab(\"\") + \n", " theme(axis.text.x = element_text(angle = 90), legend.position = \"none\",\n", " text = element_text(size = 20))\n", "\n", "g_ci" ] }, { "attachments": {}, "cell_type": "markdown", "id": "16a2088c", "metadata": { "papermill": { "duration": 0.067511, "end_time": "2021-03-23T13:50:25.267772", "exception": false, "start_time": "2021-03-23T13:50:25.200261", "status": "completed" }, "tags": [] }, "source": [ "These estimates that flexibly account for confounding are\n", "substantially attenuated relative to the baseline estimate (*19559*) that does not account for confounding. They suggest much smaller causal effects of 401(k) eligiblity on financial asset holdings." ] }, { "attachments": {}, "cell_type": "markdown", "id": "286f8dfa", "metadata": { "papermill": { "duration": 0.06959, "end_time": "2021-03-23T13:50:25.563570", "exception": false, "start_time": "2021-03-23T13:50:25.493980", "status": "completed" }, "tags": [] }, "source": [ "## Local Average Treatment Effects of 401(k) Participation on Net Financial Assets" ] }, { "attachments": {}, "cell_type": "markdown", "id": "33169af9", "metadata": { "papermill": { "duration": 0.068243, "end_time": "2021-03-23T13:50:25.700083", "exception": false, "start_time": "2021-03-23T13:50:25.631840", "status": "completed" }, "tags": [] }, "source": [ "### Interactive IV Model (IIVM)" ] }, { "attachments": {}, "cell_type": "markdown", "id": "197b1a47", "metadata": { "papermill": { "duration": 0.070268, "end_time": "2021-03-23T13:50:25.842409", "exception": false, "start_time": "2021-03-23T13:50:25.772141", "status": "completed" }, "tags": [] }, "source": [ "In the examples above, we estimated the average treatment effect of *eligibility* on financial asset holdings. Now, we consider estimation of local average treatment effects (LATE) of *participation* using eligibility as an instrument for the participation decision. Under appropriate assumptions, the LATE identifies the treatment effect for so-called compliers, i.e., individuals who would only participate if eligible and otherwise not participate in the program. \n", "\n", "As before, $Y$ denotes the outcome `net_tfa`, and $X$ is the vector of covariates. We use `e401` as a binary instrument for the treatment variable `p401`. Here the structural equation model is:\n", "\n", "\\begin{eqnarray}\n", "& Y = g_0(Z,X) + U, &\\quad E[U\\mid Z,X] = 0,\\\\\n", "& D = r_0(Z,X) + V, &\\quad E[V\\mid Z, X] = 0,\\\\\n", "& Z = m_0(X) + \\zeta, &\\quad E[\\zeta \\mid X] = 0.\n", "\\end{eqnarray}" ] }, { "cell_type": "code", "execution_count": 21, "id": "803215b0", "metadata": { "vscode": { "languageId": "r" } }, "outputs": [ { "data": { "text/plain": [ "================= DoubleMLData Object ==================\n", "\n", "\n", "------------------ Data summary ------------------\n", "Outcome variable: net_tfa\n", "Treatment variable(s): p401\n", "Covariates: age, inc, educ, fsize, marr, twoearn, db, pira, hown\n", "Instrument(s): e401\n", "No. Observations: 9915" ] }, "metadata": {}, "output_type": "display_data" } ], "source": [ "# Initialize DoubleMLData with an instrument\n", "\n", "# Basic model\n", "data_dml_base_iv = DoubleMLData$new(data,\n", " y_col = \"net_tfa\",\n", " d_cols = \"p401\",\n", " x_cols = features_base,\n", " z_cols = \"e401\")\n", "data_dml_base_iv" ] }, { "cell_type": "code", "execution_count": 22, "id": "fe2617d3", "metadata": { "execution": { "iopub.execute_input": "2021-03-23T13:50:25.988471Z", "iopub.status.busy": "2021-03-23T13:50:25.986744Z", "iopub.status.idle": "2021-03-23T13:50:26.049870Z", "shell.execute_reply": "2021-03-23T13:50:26.048200Z" }, "papermill": { "duration": 0.139252, "end_time": "2021-03-23T13:50:26.050042", "exception": false, "start_time": "2021-03-23T13:50:25.910790", "status": "completed" }, "tags": [], "vscode": { "languageId": "r" } }, "outputs": [], "source": [ "# Flexible model\n", "model_data = data.table(\"net_tfa\" = data[, net_tfa],\n", " \"e401\" = data[, e401],\n", " \"p401\" = data[, p401],\n", " features_flex)\n", "data_dml_flex_iv = DoubleMLData$new(model_data,\n", " y_col = \"net_tfa\",\n", " d_cols = \"p401\",\n", " z_cols = \"e401\")" ] }, { "cell_type": "code", "execution_count": 23, "id": "1beb1a78", "metadata": { "execution": { "iopub.execute_input": "2021-03-23T13:50:26.199177Z", "iopub.status.busy": "2021-03-23T13:50:26.197932Z", "iopub.status.idle": "2021-03-23T13:50:47.781356Z", "shell.execute_reply": "2021-03-23T13:50:47.779723Z" }, "papermill": { "duration": 21.662252, "end_time": "2021-03-23T13:50:47.781590", "exception": false, "start_time": "2021-03-23T13:50:26.119338", "status": "completed" }, "tags": [], "vscode": { "languageId": "r" } }, "outputs": [ { "name": "stdout", "output_type": "stream", "text": [ "Estimates and significance testing of the effect of target variables\n", " Estimate. Std. Error t value Pr(>|t|) \n", "p401 12802 1941 6.597 4.2e-11 ***\n", "---\n", "Signif. codes: 0 '***' 0.001 '**' 0.01 '*' 0.05 '.' 0.1 ' ' 1\n", "\n", "\n" ] } ], "source": [ "set.seed(123)\n", "dml_iivm_lasso = DoubleMLIIVM$new(data_dml_flex_iv,\n", " ml_g = lasso, \n", " ml_m = lasso_class,\n", " ml_r = lasso_class,\n", " n_folds = 3,\n", " trimming_threshold = 0.01,\n", " subgroups = list(always_takers = FALSE,\n", " never_takers = TRUE))\n", "dml_iivm_lasso$fit()\n", "dml_iivm_lasso$summary()" ] }, { "attachments": {}, "cell_type": "markdown", "id": "a80e742e", "metadata": { "papermill": { "duration": 0.071304, "end_time": "2021-03-23T13:50:48.725067", "exception": false, "start_time": "2021-03-23T13:50:48.653763", "status": "completed" }, "tags": [] }, "source": [ "Again, we repeat the procedure for the other machine learning methods:" ] }, { "cell_type": "code", "execution_count": 24, "id": "e9cee42d", "metadata": { "execution": { "iopub.execute_input": "2021-03-23T13:50:48.876339Z", "iopub.status.busy": "2021-03-23T13:50:48.874661Z", "iopub.status.idle": "2021-03-23T13:52:00.072131Z", "shell.execute_reply": "2021-03-23T13:52:00.069435Z" }, "papermill": { "duration": 71.274438, "end_time": "2021-03-23T13:52:00.072442", "exception": false, "start_time": "2021-03-23T13:50:48.798004", "status": "completed" }, "tags": [], "vscode": { "languageId": "r" } }, "outputs": [ { "name": "stdout", "output_type": "stream", "text": [ "Estimates and significance testing of the effect of target variables\n", " Estimate. Std. Error t value Pr(>|t|) \n", "p401 11792 1604 7.352 1.95e-13 ***\n", "---\n", "Signif. codes: 0 '***' 0.001 '**' 0.01 '*' 0.05 '.' 0.1 ' ' 1\n", "\n", "\n" ] } ], "source": [ "# Initialize Learner\n", "randomForest = lrn(\"regr.ranger\")\n", "randomForest_class = lrn(\"classif.ranger\")\n", "\n", "# Random Forest\n", "set.seed(123)\n", "dml_iivm_forest = DoubleMLIIVM$new(data_dml_base_iv,\n", " ml_g = randomForest, \n", " ml_m = randomForest_class,\n", " ml_r = randomForest_class,\n", " n_folds = 3,\n", " trimming_threshold = 0.01,\n", " subgroups = list(always_takers = FALSE,\n", " never_takers = TRUE))\n", "\n", "# Set nuisance-part specific parameters\n", "dml_iivm_forest$set_ml_nuisance_params(\n", " \"ml_g0\", \"p401\",\n", " list(max.depth = 6, mtry = 4, min.node.size = 7))\n", "dml_iivm_forest$set_ml_nuisance_params(\n", " \"ml_g1\", \"p401\", \n", " list(max.depth = 6, mtry = 3, min.node.size = 5))\n", "dml_iivm_forest$set_ml_nuisance_params(\n", " \"ml_m\", \"p401\",\n", " list(max.depth = 6, mtry = 3, min.node.size = 6))\n", "dml_iivm_forest$set_ml_nuisance_params(\n", " \"ml_r1\", \"p401\",\n", " list(max.depth = 4, mtry = 7, min.node.size = 6))\n", "\n", "dml_iivm_forest$fit()\n", "dml_iivm_forest$summary()" ] }, { "cell_type": "code", "execution_count": 25, "id": "e2595fb1", "metadata": { "vscode": { "languageId": "r" } }, "outputs": [ { "name": "stdout", "output_type": "stream", "text": [ "Estimates and significance testing of the effect of target variables\n", " Estimate. Std. Error t value Pr(>|t|) \n", "p401 12214 1714 7.128 1.02e-12 ***\n", "---\n", "Signif. codes: 0 '***' 0.001 '**' 0.01 '*' 0.05 '.' 0.1 ' ' 1\n", "\n", "\n" ] } ], "source": [ "# Initialize Learner\n", "trees = lrn(\"regr.rpart\")\n", "trees_class = lrn(\"classif.rpart\")\n", "\n", "# Trees\n", "set.seed(123)\n", "dml_iivm_tree = DoubleMLIIVM$new(data_dml_base_iv,\n", " ml_g = trees, \n", " ml_m = trees_class,\n", " ml_r = trees_class,\n", " n_folds = 3,\n", " trimming_threshold = 0.01,\n", " subgroups = list(always_takers = FALSE,\n", " never_takers = TRUE))\n", "\n", "# Set nuisance-part specific parameters\n", "dml_iivm_tree$set_ml_nuisance_params(\n", " \"ml_g0\", \"p401\",\n", " list(cp = 0.0016, minsplit = 74))\n", "dml_iivm_tree$set_ml_nuisance_params(\n", " \"ml_g1\", \"p401\",\n", " list(cp = 0.0018, minsplit = 70))\n", "dml_iivm_tree$set_ml_nuisance_params(\n", " \"ml_m\", \"p401\",\n", " list(cp = 0.0028, minsplit = 167))\n", "dml_iivm_tree$set_ml_nuisance_params(\n", " \"ml_r1\", \"p401\",\n", " list(cp = 0.0576, minsplit = 55))\n", "\n", "dml_iivm_tree$fit()\n", "dml_iivm_tree$summary()" ] }, { "cell_type": "code", "execution_count": 26, "id": "19d1ef93", "metadata": { "vscode": { "languageId": "r" } }, "outputs": [ { "name": "stdout", "output_type": "stream", "text": [ "Estimates and significance testing of the effect of target variables\n", " Estimate. Std. Error t value Pr(>|t|) \n", "p401 11861 1619 7.324 2.4e-13 ***\n", "---\n", "Signif. codes: 0 '***' 0.001 '**' 0.01 '*' 0.05 '.' 0.1 ' ' 1\n", "\n", "\n" ] } ], "source": [ "# Initialize Learner\n", "boost = lrn(\"regr.xgboost\", objective = \"reg:squarederror\")\n", "boost_class = lrn(\"classif.xgboost\", objective = \"binary:logistic\", eval_metric = \"logloss\")\n", "\n", "# Boosted Trees\n", "set.seed(123)\n", "dml_iivm_boost = DoubleMLIIVM$new(data_dml_base_iv,\n", " ml_g = boost, \n", " ml_m = boost_class,\n", " ml_r = boost_class,\n", " n_folds = 3,\n", " trimming_threshold = 0.01,\n", " subgroups = list(always_takers = FALSE,\n", " never_takers = TRUE))\n", "\n", "# Set nuisance-part specific parameters\n", "if (compareVersion(as.character(packageVersion(\"DoubleML\")), \"0.2.1\") > 0) {\n", " dml_iivm_boost$set_ml_nuisance_params(\n", " \"ml_g0\", \"p401\",\n", " list(nrounds = 9, eta = 0.1))\n", " dml_iivm_boost$set_ml_nuisance_params(\n", " \"ml_g1\", \"p401\",\n", " list(nrounds = 33, eta = 0.1))\n", " dml_iivm_boost$set_ml_nuisance_params(\n", " \"ml_m\", \"p401\",\n", " list(nrounds = 12, eta = 0.1))\n", " dml_iivm_boost$set_ml_nuisance_params(\n", " \"ml_r1\", \"p401\",\n", " list(nrounds = 25, eta = 0.1))\n", "} else {\n", " # behavior of set_ml_nuisance_params() changed in https://github.com/DoubleML/doubleml-for-r/pull/89\n", " dml_iivm_boost$set_ml_nuisance_params(\n", " \"ml_g0\", \"p401\",\n", " list(nrounds = 9, eta = 0.1, objective = \"reg:squarederror\", verbose=0))\n", " dml_iivm_boost$set_ml_nuisance_params(\n", " \"ml_g1\", \"p401\",\n", " list(nrounds = 33, eta = 0.1, objective = \"reg:squarederror\", verbose=0))\n", " dml_iivm_boost$set_ml_nuisance_params(\n", " \"ml_m\", \"p401\",\n", " list(nrounds = 12, eta = 0.1, objective = \"binary:logistic\", eval_metric = \"logloss\", verbose=0))\n", " dml_iivm_boost$set_ml_nuisance_params(\n", " \"ml_r1\", \"p401\",\n", " list(nrounds = 25, eta = 0.1, objective = \"binary:logistic\", eval_metric = \"logloss\", verbose=0))\n", "}\n", "\n", "dml_iivm_boost$fit()\n", "dml_iivm_boost$summary()" ] }, { "cell_type": "code", "execution_count": 27, "id": "08267d88", "metadata": { "vscode": { "languageId": "r" } }, "outputs": [ { "data": { "text/html": [ "\n", "\n", "\n", "\t\n", "\t\n", "\n", "\n", "\t\n", "\t\n", "\t\n", "\t\n", "\n", "
A data.table: 4 × 5
modelMLEstimatelowerupper
<chr><chr><dbl><dbl><dbl>
IIVMglmnet 12802.268998.63916605.88
IIVMranger 11792.228648.57914935.87
IIVMrpart 12214.458855.86415573.04
IIVMxgboost11861.198687.15815035.22
\n" ], "text/latex": [ "A data.table: 4 × 5\n", "\\begin{tabular}{lllll}\n", " model & ML & Estimate & lower & upper\\\\\n", " & & & & \\\\\n", "\\hline\n", "\t IIVM & glmnet & 12802.26 & 8998.639 & 16605.88\\\\\n", "\t IIVM & ranger & 11792.22 & 8648.579 & 14935.87\\\\\n", "\t IIVM & rpart & 12214.45 & 8855.864 & 15573.04\\\\\n", "\t IIVM & xgboost & 11861.19 & 8687.158 & 15035.22\\\\\n", "\\end{tabular}\n" ], "text/markdown": [ "\n", "A data.table: 4 × 5\n", "\n", "| model <chr> | ML <chr> | Estimate <dbl> | lower <dbl> | upper <dbl> |\n", "|---|---|---|---|---|\n", "| IIVM | glmnet | 12802.26 | 8998.639 | 16605.88 |\n", "| IIVM | ranger | 11792.22 | 8648.579 | 14935.87 |\n", "| IIVM | rpart | 12214.45 | 8855.864 | 15573.04 |\n", "| IIVM | xgboost | 11861.19 | 8687.158 | 15035.22 |\n", "\n" ], "text/plain": [ " model ML Estimate lower upper \n", "1 IIVM glmnet 12802.26 8998.639 16605.88\n", "2 IIVM ranger 11792.22 8648.579 14935.87\n", "3 IIVM rpart 12214.45 8855.864 15573.04\n", "4 IIVM xgboost 11861.19 8687.158 15035.22" ] }, "metadata": {}, "output_type": "display_data" }, { "data": { "image/png": "iVBORw0KGgoAAAANSUhEUgAAA0gAAANICAMAAADKOT/pAAAAbFBMVEUAAAAAv8RNTU1oaGh8fHx8rgCDg4OMjIyVlZWampqjo6Onp6evr6+ysrK5ubm9vb2+vr7BwcHHfP/Hx8fJycnQ0NDR0dHY2NjZ2dne3t7h4eHk5OTp6enq6urr6+vv7+/w8PD19fX4dm3////4p3w3AAAACXBIWXMAABJ0AAASdAHeZh94AAAgAElEQVR4nO2d6WLbNhpFWciVLG8a15bjtZar93/HEbVS1vWS+COIS53zYxwm7eQ08IkkEACrOQD8mKprAYA+QEgAARASQACEBBAAIQEEQEgAARASQACEBBDAUYT0b9cC38TEE00BIRWEiSeaAkIqCBNPNAWEVBAmnmgKCKkgTDzRFBBSQZh4oikgpIIw8URTQEgFYeKJpoCQCsLEE00BIRWEiSeaAkIqCBNPNAWEVBAmnmgKCKkgTDzRFBBSQZh4oikgpIIw8URTQEgFYeKJpoCQCsLEE00BIRWEiSeaAkIqCBNPNAWEVBAmnmgKCKkgTDzRFBBSQZh4oikgpIIw8URTQEgFYeKJpoCQCsLEE00BIRWEiSeaAkIqCBNPNAWEVBAmnmgKCKkgTDzRFBBSQZh4oikgpIIw8URTUG5I/5VJm//JfIdGQkjhtPrdHwjfoZEQUjiEFAqaAkIqCL5DIyGkcAgpFDQFhFQQfIdGQkjhEFIoaAoIqSD4Do2EkMIhpFDQFBBSQfAdGgkhhUNIoaApIKSC4Ds0EkIKh5BCQVPQDKn6gKxCbUBIoaApIKSC4Ds0EkIKh5BCQVPgX8k3IKRQ0BQQUkHwHRoJIYVDSKGgKSCkguA7NJKyQpo93pxkEWkTQgoFTYEKaXrCrF0n8B0aSechnTP93RF8h0bSdUiP7zIa3GcVagNCCgVNwWFIixekwWyeqmo6n02q+os9hBQKmoLDkAbLdkZVNZkvX55SVp9WIKRQ0BQchrT6ULR4LRrVV5dVdZNVqA0IKRQ0BR+FtHgpGtRXz+ugrCGkUNAUfBTS5sucWbt88B0aSdchnVTVbPXlefkPEFI2+A6NpOuQzlcTdeer2YZ7QsoH36GRdB3StKrS4+bL44DPSPngOzSSrkOq39RV6y9L/G8kEVIoaApESM9pGVL9peY8q08rEFIoaArkB6DJYP1l8b7O//WIkIJBU+A/k/ANCCkUNAWEVBB8h0ZCSOEQUihoCsQN2ZtZVoMMENJ8/leRtPgf3HlIVVX1rSVCCqXVb/84Cgipby0RUiiEJDgM6fnmpG8tEVIohCSQkw2zm9GmpawybUFIoRCS4MNZu+l41dK4B3dkCSkUQhJ8Nv19f544RSgnhBRJOSHNlzvOCSkbhBRJOSFNx7wiZYWQIikjpBmfkfJDSJEUENJsMwPeh4rmhBQMIQnEfaTLdUW92EGxhJBCISQBKxsKgpAiKSCkLyu63Ry/+pa2rH7i4WLxw4uHzT/4+WUuCCkUQhL80ervh00286d3IZ2tL87m37jMBiGFQkgCcRzXlzMMD9vXn/ldumv+ym0a3r3N3+6G6frry3wQUiiEJJCH6H/+r9zt3sjNL9Jr85fS+vIlDb++zAchhUJIgg+PLP6Il8X7s7NtSMO9JB62LzVX6emry4wQUiiEJPjtV6TFq9HtfBPSW7pq/tp12swjPCz+oS8uM0JIoRCS4DCa+8+PsksXr/NtSE/p16+LlIZXL8vLi/Sy/qde0sVXlxkhpFAISaAOiBxVo5vHj/6Ft/p/NiHdbefslu/ahruHktUfgz6/zAghhdJmSP8UyTfEP7oh+/nDmDchLV6N7uqyni6WJaW09098fpkRQgrF5BXpO9/+cfwspOH2Y89Z/b7tT0L6NwP/5fhNjoe/uhb4Hv9k+D2iQtrxUE878Ir0M3hFiqTrV6TvIEKoP/YcY0h/l0mL/8WEpIgLafFTp/vTCZ9fZsTlFanVb/84CEkQGhLT3z+DkCLxDOk1ndaL6Xa3XK+/uswIIYVCSAId0vPlaPW0sfG9/PXdrN3b+md+1W3sAlmuYvj8MiOEFAohCWRIo+1s3QerHDYhXW/X+pwul89tlqW+rv6Bzy/zQUihEJLgg0df7kKSJW3X2g3TbR3H09lqj9H1dqPE7deX+SCkUAhJoB/GPJiuF4FPF1/EaqHtS8rrcL1EaL14lY19P4GQIuk6pEU7g/l2N8W5fElqvDe7O9stWl3wa38z+eeXuSCkUAhJcBjS4gNSvUV2HdLzKitvCCkUQhJ8uLFvszSIk1bzQUiREFI4hBQKIQm+CmlWVZnnqluAkEIhJMFhSOO9z0iTqhpnFWoDQgqFkARy1i7N5o3pb/+TiwkpFEIS6PtIaXUf6fG8PnY1q08rEFIohCQQIc0GzV196TmrTysQUiiEJJBTcuNdR6M+nKVPSKEQkuCz1d/V6PLDw4SsIKRQCEngf5PoGxBSKIQkIKSCIKRIug7p3VKG1INpO0IKhZAEX4bEEqF8EFIkZYU0JaR8EFIk3YU0riSDrEJtQEihEJKgGdJMhzTJKtQGhBQKIQn23rdNREYn/h0RUiyEJPjtJ/Y5QkihEJKAkAqCkCLpOqQeQkihEJKAkAqCkCLpPqTH872NFP6tEVIohCQQlUzfT9xlFWoDQgqFkASHlTwezIBnFWoDQgqFkASHldTbyyc92BbbgJBCISTBYUipD2sZ9iGkUAhJIO8j9WF7eRNCCoWQBNyQLQhCiqTrkE4IqSsIKZKuQ5r04UjIfQgpFEISiFef1Iuz7JoQUiiEJNCPvkz9mv8mpFAISSAnG7gh2w2EFAkhhUNIoRCSgJAKgpAi6TqkHkJIoRCSgJAKgpAiIaRwCCkUQhIQUkEQUiTdhbSaV2CyoTMIKRJCCoeQQiEkASEVBCFFwmekcAgpFEISEFJBEFIkhBQOIYVCSAJCKghCioSQwiGkUAhJQEgFQUiREFI4hBQKIQkIqSAIKRJCCoeQQiEkASEVBCFFQkjhEFIohCQgpIIgpEgIKRxCCoWQBDqk58tRWq76Ht9ntWkJQgqFkAQypNF2+0RVnWfVaQdCCoWQBPqk1UZIfSiJkEIhJIEI6aSqBtP1Lr/6ebKPWYXagJBCISTBYUiLdgbz7WOSzvvwkkRIoRCS4DCk0eqxLuuQnldZeUNIoRCS4MMn9m3OauDMhnwQUiSEFA4hhUJIgq9CmlVVyirUBoQUCiEJDkMa731GmlTVOKtQGxBSKIQkkLN2aTZvTH/7P1GWkEIhJIG+j5RW95Eezxf/e5LVpxUIKRRCEoiQZoPmMas9eDDzf/+ZlERIkXQe0vJj0obRLKtOG/z3n0tJhBRJASGtV39Xo0v/5UGrjjxKIqRISgipTxBSNIQkIKSCIKRISghpOlh+SWP/qW8+I4VDSAIV0uxktzxo4D9px6xdMIQk+Ghj3+oX+zH9zX2kWAhJIEJadDReT3pPB31YakdIsRCS4DCkSVWNdleLkiYZddqBkEIhJMFhSIsPSI13c9M+rBEipFAISfDhNooPLi0hpFAISUBIBUFIkXQd0mDvrR0b+zJCSJF0HdL+uUGtb+z7NwP/5fhNAvi7a4Hv8VfXAt/jnwy/x2chPTb38j2ysS8jvCJF0vUr0vLA4vPlmd/3bOzLCiFF0nlIs7S3sc9/QxIhxUJIgs8O0V9u7Mtq0xKEFAohCfq/sW9OSMEQksD/JtE3IKRQCElASAVBSJEQUjiEFMlff3mU1H1Ij+d7B3L5t0ZIgfz1l0lJnYc0rd6RVagNCCmOv/5yKanrkB7fd0RIuSCkSLoOqV7NMOnB/vIGhBQHIWkOQ0p92BO7DyEF4tJR5yEtXpB6sCpoD0KKxKSjEkLKapABQgrFo6POQzohpK4gpEi6DmnShx1I+xBSKIQkkOfa9eFQyCaEFAohCfRJq6lf89+EFAohCeRkAzdku4GQIiGkcAgpFI+Q/vkna0mEVBCEFMc//+Qtyb+Sb0BIoTiE9M8/mUsipHL4+2+PkghJQEjF8PffJiURkoCQSuHvv11KcgipkM9Iq1OEFj8Y32d0aQ1CCsUipO5n7ebbc+3qX26eA24LIYXiEVLn95FWz5DdhtSHkixC4jNSMJ2HdFJVg+l6N0V9foP/IZEeITFrF0vXIS3aGcy325LO+/CSZBIS95FC6Tqk0WobxTqk51VW3hBSKIQk+HCH7GZpEEuE8kFIkRBSOIQUCiEJvgppxjNk80FIkXQd0njvM1Lrz5DNASGFQkgCOWtXP6VvN/3tf4IDIYVCSAJ9Hymt7iM98gzZrBBSJJ2HNNt7FkUfDkIhpFAISSCn5Ma7jkZ9OHWVkEIhJAHPkC0IQoqkhJB6BiGFQkgCQioIQoqEkMIhpFAISdAM6fAgLo7jygohRUJI4RBSKIQkIKSCIKRIuv+MNKqq8X19/6he2XCZVacdCCkUQhLoJULb5XXPLBHKCCFF0nVIk72HMU/68GhmQgqFkASHIQ32HsY8Y6t5Pggpkq5Deje7wGRDPggpkrJCmhFSPggpkq5DGr3/jDTKqNMOhBQKIQkOQ7pp7kFih2xOCCmSrkOqZxuq8+X+ifsxO2RzQkiRdB7S9uzvJT3oiJBiISSBmkmYjXYd+Z9XPCekYAhJ8OEO2fq4htFlHzaaE1IwhCTwn9v+BoQUCiEJCKkgCCkSQgqHkEIhJMG7/UhztSspq1AbEFIohCQgpIIgpEgIKRxCCoWQBM1KptN+nAd5ACGFQkiCZkiDHrz4SAgpFEISyLd2WQ0yQEihEJKAkAqCkCLpLqS03IlESJ1BSJF0F9L4cMKOWbucEFIk3YX0TEjdQkiRdLhE6PGEkLqEkCLpeq1dH8p5ByGFQkgCQioIQoqk65B6CCGFQkgCQioIQoqEkMIhpFAISaBCejwfMGvXBYQUSechTZn+7ghCiqTrkB65j9QVhBRJ1yGdL9KZPIt/1BdCCoWQBIchpT48WmwfQgqFkATyhmw/joXcQUihEJLgz1Y23KbtDx8uUkoXD390mQtCCoWQBIfRnHwd0kPahnSWVpz9wWU2CCkUQhLIhzF/8USkRUebkG7T8O5t/nY3TNe/fZkPQgqFkATi1Sc1HjSmuEu7kFJ6XX59ScPfvswHIYVCSAL9fKT08fz3y+L92dkmpIfta8tVevrNy4wQUiiEJJCTDZ/dkF28Gt3ONyFdp83EwcPiZ3/vMiOEFAohCX4/pIvX+Taki/Sy/umXdPGblxkhpFAISfDbIb3V/7MJabibBq8/9/zWZUYIKRRCEvzZQrpNSCnt/dRvXWaEkEIhJEHnIf2bgf9y/CYB/N21wPf4q2uB7/FPht+joJBywCtSKLwiCQipIAgpkhJCejxfnnA3Ov/gOS+bEE735w9+6zIjhBQKIQlUSLPRbsZuJO/MMv3dCoQUSechPafm3LdcLrQJ6bZxj/X6Ny8zQkihEJJAhFS/qzu/r390X++WPRH/1uESoeWyhd+6zAghhUJIgsOQ6rNPpvJix8Gi1dfVz/zWZT4IKRRCEhyGNNrbaj5ZfEw6/Le2JVxvd0bc/vZlPggpFEISfLXVfCY3zLKxrxUIKZICQvrkckXjvdmv/d3jv3WZC0IKhZAEf/aKZAYhhUJIgj/7jGQGIYVCSIJvzNrdZBVqA0IKhZAE4n3bYP8+0iCrTysQUiiEJPizlQ1mEFIohCRQMwnPzbV2fTh1lZBCISTBB6u/L5ctjS4/WP1tBiGFQkgC/7ntb0BIoRCSgJAKgpAiKSGk6WD5JY2/OLzYBEIKhZAEcmPf5hz9evK7B5N2hBQLIQk+mv5e/SLT31khpEg6D2nR0Xg96T0dLErK6tMKhBQKIQnkY10aq+sGfXgQJiGFQkgC+aCxxru5qd5r7gUhhUJIgj/bj2QGIYVCSAJCKghCiqTrkAZ7b+1mfZhtIKRQCElwGNJ5VZ3vriZVNc6o0w6EFAohCQ5Demxu7Nu7sIWQQiEkgfgANPr6gEgzCCkUQhKIkGb7G/t6sCGJkEIhJIGckmtu7Mtq0xKEFAohCfTc9vPlKLGxLzuEFEkJIfUMQgqFkASEVBCEFAkhrfivTNr8TyakSAgpnH+//keKgJAiIaRwCCkUQhIQUkEQUiSEFA4hhUJIAkIqCEKKhJDCIaTFd3+RtPgfTEjxEFIoJn+chBSOycgTUiilhNSHPeZrTEaekEIhpHBMRp6QQiGkcExGnpBCIaRwTEaekBbf/UXyDXFC+iF/l0mL/8Uufy/l1SSkgjDxRFNASAVh4ommgJAKwsQTTQEhFYSJJ5qC3sTyGSYj7+KJpoCQCsLEE03BRyFN67PtTi57cDrk3GbkXTzRFDRD2n0qej7ZHBB5mdWmJUxG3sUTTYEMqXlocQ+O/nYZeRdPNAUypMXr0WC6eFf3PEm9eE0yGXkXTzQFKqTpoqP1Tz0vSvI/tthk5F080RSokMaNZyJN9h47ZorJyLt4oilQIS1ehbazdc99+JRkMvIunmgKVEh7axr6sMDBZORdPNEUEFJBmHiiKVAhneyH5P9Uc5ORd/FEU6BCmjQ+I0378NA+k5F38URT8C6km+Vcd6ruNz836MNTzU1G3sUTTcG7kBYMxpPx+lVoNk27W0rGmIy8iyeagsOQVmx/Ij1n9WkFk5F38URTsDcl93izeghzI6RBDzpyGXkXTzQFYm57mdPqF0f+n49qTEbexRNNgf9Nom9gMvIunmgKCKkgTDzRFByEdH85Gqw2Io0m9+rfMMRk5F080RS8C+lx0Jy6q9Ikq0xbmIy8iyeagv2QLqv3MGuXERNPNAV7IU3r16DnemdsvTF2Nh1zHykrJp5oCvZCSttsnqtq+a7uvA/bkVxG3sUTTUEzpMn+ztjlwtURa+3yYeKJpqAZ0sn+ztjlFvPFu71xVqE2MBl5F080Bfpcu+XFoP4y68OqVZORd/FEU/BJSGK7rCkmI+/iiaagWUmqqu0U3YyQ8mPiiaagWcm4cRrkzXq67pm3dvkw8URT0Axpunj1Wa8Kqg+GXM5/jznXLh8mnmgK9t631Ufnnz8u3tfdpNWZJ7P6liwnrebCxBNNwV5Iz43D85cfl+qv/i9ILiPv4ommYH8mYfc4l9Uau3505DLyLp5oCt5PyU3Hg8W7us3O2DT2f183txl5F080Bf5z29/AZORdPNEUEFJBmHiiKeg8pH8BbNl9H3ceUg5M/gp18URT8GVILBHKh4knmgJCKggTTzQFhFQQJp5oCvwr+QYmI+/iiaaAkArCxBNNASEVhIknmgJCKggTTzQFByE934xHq3Wro/GkFyvtbEbexRNNwbuQpvtHFleDXpxZbDLyLp5oCvb3I73LaHmWfladdjAZeRdPNAUHO2QHl/frN3SPj5NBP0oyGXkXTzQF789seLePrz5Vn5NWc2Hiiabg3SlCB6eqqp+zw2TkXTzRFDRDGjTOtdvwvDoFxRuTkXfxRFPw4Umrn/6kGSYj7+KJpoCQCsLEE03BR0cWb+CtXUZMPNEUMNlQECaeaAreT3/f7P/yhOnvjJh4oin4+obsIKtPK5iMvIsnmoL9mYSTwyVCfXisucnIu3iiKXg3JTdh0WqHmHiiKTiY236crLdRpNF40oNXoxqTkXfxRFPgf5PoG5iMvIsnmgJCKggTTzQFhFQQJp5oCgipIEw80RQQUkGYeKIpIKSCMPFEU0BIBWHiiaaAkArCxBNNASEVhIknmgJCKggTTzQFhFQQJp5oCgipIEw80RS8O7NBklWoDUxG3sUTTQEhFYSJJ5oCQioIE080BaqSUVWN72eLHzyeV9VlVp12MBl5F080BSKkk8Z5J88cop8RE080BYchTapq8tGVKSYj7+KJpuAwpEFVzXZXsz4cI2Qy8i6eaAoOQ3o3u8BkQz5MPNEUfBXSjJDyYeKJpuCwktH7z0ijjDrtYDLyLp5oCg5DuqmqtD2Ga8qRxRkx8URTIN631WdEni+PLb4f8wzZnJh4oikQIT0nnmreDSaeaArUTMJstOvoXPy6HSYj7+KJpkBPyT1fjpaHFl/O5C+7YTLyLp5oCvzntr+Byci7eKIpIKSCMPFEU0BIBWHiiaZAhfR4vv+UpKxCbWAy8i6eaApEJVM29nWEiSeagsNKHtkh2xUmnmgKDis5X6TTl0f1rTEZeRdPNAWHIaU+bOXbx2TkXTzRFMhtFP24DbvDZORdPNEUfLmxrw+YjLyLJ5qCw2hOCKkrTDzRFMjDT/x3IO1jMvIunmgKxKtPamzs6wcmI+/iiaZA70dK/Zr/Nhl5F080BXKygRuy3WDiiaaAkArCxBNNASEVhIknmgL/Sr6Byci7eKIpIKSCMPFEU0BIBWHiiaaAkArCxBNNwVchzR5v/E+2Mxl5F080BSqk6Qmzdp1g4ommQFRyzvR3R5h4oin4eqv54D6rUBuYjLyLJ5oCudV8MKv3yU7ns0nVi6XgJiPv4ommQD76crp9StLi5Sll9WkFk5F38URT8NEO2c0Dxi6r6iarUBuYjLyLJ5qCj0J6XD+E+Zkn9uXDxBNNwYdnNmxm65i1y4eJJ5oCeWbDbPVlubmPkPJh4ommQM7aTVdf6tmGe0LKh4knmoLDSqZVlR43Xx4HfEbKh4knmgLxcnOyehHarhP67EbSW9qy+omHi8UPLx42v/75ZS5MRt7FE03BBw9jnu+eyfzpU2Sf3oV0tr44m3/jMhsmI+/iiaZAfgCaDNZfFu/rPl/YcJfumpe3aXj3Nn+7G6brry/zYTLyLp5oCn44k3CRXpuXaX35koZfX+bDZORdPNEU/DCk4V4SD9uXmqv09NVlRkxG3sUTTcHPQnpLV83L67SZR3hIt19dZsRk5F080RT8LKSn9OvXRUrDq5fl5UV6Wf/CS7r46jIjJiPv4omm4Gch3W3n7Jbv2oa7leL1x6DPLzNiMvIunmgKfhbS4tXo7m3x9eliWVJqpJK+usyIyci7eKIp+FlIw+3HnrP6fdufhPQvgC1RIe14qKcdeEX6GSaeaArCVqTWH3sI6WeYeKIpiAtp0cbp/nTC55cZMRl5F080BaEhMf39M0w80RR8FtJ5qgaTb/7/vKbTejHd7pbr9VeXGTEZeRdPNAUqpNn5oP6y2kfx6YHFw/S2/tGvuo1dIMtVDJ9fZsRk5F080RR8vI1ist6O9Nm+vuvtWp/T5fK5zbLU19VswueX+TAZeRdPNAXyqebLkFL9avS4+N/Hj//tt2G6reN4OlvtMbrebpS4/foyHyYj7+KJpkBvNX9ensNV3S9flz7b2fc6XC8RWi9eZWPfTzDxRFNwGNJ4tbn8cvW6NFufb/chd2e7RasLfu1vJv/8MhcmI+/iiaZAHlm8Po5rOc/AKUL5MPFEU/DZAZHnjUtrTEbexRNNwSdHFq+ODyKkfJh4oin45BD95Tu8+y/uJFlgMvIunmgKDkMaLWfrNh+RFleXWYXawGTkXTzRFByGdFMnVJ9UPJnPZ+NqfQS4NSYj7+KJpuCDG7LV6p1d/cX/Bcll5F080RSIkO53JxX3oyOXkXfxRFOgpuSeF2/oTpaPYB6cf7I+yAeTkXfxRFPgP7f9DUxG3sUTTQEhFYSJJ5qCD1c2bEjcR8qGiSeagi9DYmVDPkw80RR8FdKUkPJh4ommoFnJuJIMsgq1gcnIu3iiKWiGNNMhfff8k3IxGXkXTzQFe+/bJiKjE/+OXEbexRNNwZeTDX3AZORdPNEUEFJBmHiiKehdNAqTkXfxRFNASAVh4ommQIX0eD7Ym2/IKtQGJiPv4ommQFQyfT9xl1WoDUxG3sUTTcFhJY8HM+BZhdrAZORdPNEUHFZyXt+D9d9e3sRk5F080RQchpT6sJZhH5ORd/FEUyDvI82yKrSPyci7eKIp4IZsQZh4oik4jOaEkLrCxBNNwWE0k/VZxT3CZORdPNEUyHPtUr8m7VxG3sUTTYF+9GXq1/y3yci7eKIpkJMN3JDtBhNPNAWEVBAmnmgKCKkgTDzRFPhX8g1MRt7FE00BIRWEiSeaAkIqCBNPNAWEVBAmnmgKdEjPl6O0nGQY32e1aQmTkXfxRFMgQxptZ+uq6jyrTjuYjLyLJ5oCvbKhEVIfSjIZeRdPNAUipJOqGkzXuynq8xv8H9pnMvIunmgKDkOaro7NX9+IPe/DS5LJyLt4oik4DGm0fQ5zffXM0yjyYeKJpuDDHbKbpUEsEcqHiSeaAkIqCBNPNAVfhTSrqpRVqA1MRt7FE03BYUjjvc9Ik6oaZxVqA5ORd/FEUyBn7dJs3pj+9j/BwWTkXTzRFOj7SGl1H+mxPnX1JKtPK5iMvIsnmgIR0mzvWRR9OAjFZORdPNEUyCm5xuPNR304ddVk5F080RR8tvq7Gl36Lw+qMRl5F080Bf43ib6Byci7eKIpIKSCMPFEU0BIBWHiiaagGdJ2CxLHcXWDiSeaAkIqCBNPNAWEVBAmnmgK/Cv5BiYj7+KJpoCQCsLEE00BIRWEiSeaAh3SdLD8ksb+K79rTEbexRNNgQppdrLbHTvowZpVl5F38URT8NG5dqtfZPV3Vkw80RTIZ8hW4/Wa7+mgDzvNXUbexRNNgXyq+Wh3tShpklGnHUxG3sUTTcFhSIsPSI13c9M+bJE1GXkXTzQFH54i9MGlJSYj7+KJpoCQCsLEE03BYSWDvbd2nGuXERNPNAWHIe0fm8+5dhkx8URTcBjSY/Mou70LW0xG3sUTTYH4AFQ/r+98+cjLe861y4qJJ5oCda5dam5GSj04j8tk5F080RR89gzZ5bl2WW1awmTkXTzRFHCuXUGYeKIp8L9J9A1MRt7FE00BIRWEiSeaAkIqCBNPNAWcIlQQJp5oCjoP6V8AWwoKKQcmf4W6eKIpaFYynfZjtvsAk5F38URT0Axp0IMXH4nJyLt4oimQb+2yGmTAZORdPNEUEFJBmHiiKWhGk5YHnRBSZ5h4oiloRjM+nLBj1i4nJp5oCpqVPBNSt5h4oinYq+TxhJC6xMQTTcGXpwj1AZORd/FEU3B4Q5aQOsPEE00BN2QLwsQTTQH3kQrCxBNNASEVhIknmgJuyBaEiSeaAm7IFoSJJ5oCbsgWhIknmgJuyBaEiSeaAm7IFoSJJ5oCQioIE080Bb2LRmEy8i6eaAoIqSBMPNEUEFJBmHiiKfjsEP3FD8b3WW1awgAInR4AAAyuSURBVGTkXTzRFHz2WJf6l5uPwbTFZORdPNEUiJCeU9UIqQ8lmYy8iyeaAhHSSVUNputZ8Onii/+pkSYj7+KJpuAwpEU7g/n2dtJ5H16STEbexRNNwWFIo9VzzNchPa+y8sZk5F080RR8uLJhs8ChDwsdTEbexRNNASEVhIknmoKvQppVVcoq1AYmI+/iiabgMKTx3mekSVWNswq1gcnIu3iiKZCzdmk2b0x/T7MKtYHJyLt4oinQ95HS6j7S4/nif0+y+rSCyci7eKIpECHNBs3tsek5q08rmIy8iyeaAjkl1zgFZTTLqtMOJiPv4omm4LPV39Xo0n95UI3JyLt4oinwv0n0DUxG3sUTTQEhFYSJJ5oCQioIE080BTqkx/PlCXejcz4j5cTEE02BCmk2asza9WD222XkXTzRFHy2Q5b7SJkx8URToFc2VOfLQ0/uWdmQFRNPNAVyrd1ueR1r7XJi4ommQO6QneyuJouPSRl12sFk5F080RTI/UiNZUEzNvblw8QTTcGXh+gTUj5MPNEU8IpUECaeaAr4jFQQJp5oCr4xa3eTVagNTEbexRNNgXjfNti/jzTI6tMKJiPv4ommgJUNBWHiiaZAzSQ8N9fasUM2HyaeaAo+WP19uWyJHbJ5MfFEU+A/t/0NTEbexRNNASEVhIknmgJCKggTTzQFn4U0uOzBjF2Nyci7eKIpOAhpMtj86L7qw7nfNSYj7+KJpuBdSDepqjYPMr/sy20kl5F38URTsB9SvZKhutxc3ddz4KkHN5JMRt7FE03BXkg31fL8/B31Ujv/NasuI+/iiaagGdIsHXwo4qnmOTHxRFPQDOlGrFAd8VTzfJh4oilohjQSB53c9+EYIZORd/FEU9AMaX9v7O4n89m0hMnIu3iiKXgXkvgHCCkbJp5oCgipIEw80RQ0KzmpqoPbr498RsqHiSeagmZI5+J8hss+LBMyGXkXTzQFzZDuxfR34sjifJh4oinY+wA0OLhptHi3l3LqtIPJyLt4oinYC+nx3SENz/WDKe7f/yt+mIy8iyeagv0puUm9aHU8XS4KerwZ7S1hNcZk5F080RS8m9ueVvsk/w9Ic5uRd/FEU/D+JtFs3MzovAd7KOY2I+/iiabg8G7r7Ga8fhJzL16NakxG3sUTTYH/soVvYDLyLp5oCgipIEw80RQQUkGYeKIpIKSCMPFEU0BIBWHiiaYgb0gPFymli4esv+fcZuRdPNEUZA3pLK04y/mbzm1G3sUTTUHOkG7T8O5t/nY3TNcZf9e5zci7eKIpyBlSSq/Lry9pmPF3nduMvIsnmoKMIT1sX4iu0lO+33ZuM/IunmgKMoZ0nTazDA/pNt9vO7cZeRdPNAUZQ7pIL+sfvaSLfL/t3GbkXTzRFGQMabjba5v5Q5LJyLt4oinIGFJK6oc5MBl5F080BZ2H9D8AWwoKKQcmf4W6eKIpIKSCMPFEU5AxpFMmG77AxBNNAdPfBWHiiaYgY0i3jRuyeRfbmYy8iyeagk6WCO3WOOTBZORdPNEUdLFo9TXzXIPLyLt4oinIGdL1dhtF3qV2LiPv4ommgI19BWHiiaYg71bzX2w1/wwTTzQFHH5SECaeaAoIqSBMPNEUEFJBmHiiKSCkgjDxRFNASAVh4ommgJAKwsQTTQEhFYSJJ5oCQioIE080BYRUECaeaAoIqSBMPNEUEFJBmHiiKSCkgjDxRFNASAVh4ommgJAKwsQTTQEhFYSJJ5oCQioIE080BYRUECaeaAoIqSBMPNEUEFJBmHiiKSCkgjDxRFNwFCEBtA0hAQRASAABEBJAAIQEEAAhAQRASAABEBJAAIQE/ab5MK639h5eTEjQb/aeatfeI+4ICfpNs50HQgL4be7SAVdt/V6EBP3l7F1Gw9Y6IiToOZme/E1I0G8IKZxME6FwjBxrSLn+ovoTrq9fu1boIXen6fRXe//3xxpSixOhPyalt64VesXd6Xwzg3fd2m9yJCHlnAj9MQU37sjZ8s9zmM7mL8P00tbvciQh5ZwI/THXqcW3IEfHQzpdvFN+Sump/jH3kQKw+Zv+Kt229jfn0XFRF7T4y2k5+Ky1i8AlpHcvnl3rmLP6AzxNF7uLNjimkFwgpEiWf4BvKd1tL1rhyEJ6uT5d/lle8NbpWFiO90NaTjO8ptO2fpvjCulq81d8Sg9du0AeLuqE1h+Rrtqb/z6qkC7S6cP2byju1RwHv9LZ4pWonq57W+TU2q3uYwrpYfnCvvq76TrddmzzKbwFjWN552P4uvzs2d77kGMK6WL557gK6XXx91S58BY0kuthuqhfiYbXLb4LOaaQ1rcSUuOiTDzegrIGuAkhlYfJW1CXNcB5OKaQho2QSv4r1OQtqMsa4PUnznR23eYHzmMK6arxDXpX/KLVkl85rdYA13Pf7UseU0gvafi2u0H31LXOh5Qfktka4NXSxZfbNks6ppAWf6TD5Yf4l+s2d6b8GJO3oGUmfsDDcup7yeuwvb8+jyqk+e32r9CCO3J5C2oS0tVqld2SFv80jyuk+dvtWesfO3+MyVtQkx3xzRsIHFl8XHi8BS34FlcTjiw+Yizegpq8tdt/RSKk48LhLajJjng+I7XA7Skb5uLw2BG/P2vX2tLFowppyM7TQFw28l5xHymY2/ozPEThEtJ6LT0rG8I4NdmT8G7RwMWtxexYwbxct/+J85hCKvlvzSYH69iGFjdsjhtCKpBf6eyhfhV6e7pevIi+3Za7TAg2HFNIV+0dWBvKW+O9/F3dUHMGt0xeny66VvgEtlHE8pRKHu0d140jOt7qhl7K3JT0etV8/9m1zcewjSKa63RR7Mq1Bqd7i1p2u2UL4615N+G03D9YtlFE4zJfe7g6rEjZ23T6ttzy8farxRudP4ZtFOG4hHTWGO6n+l1di0vEfsBZXc/F6gTT4bDYKXqWCB0tv9Lu2/K0XtB2V+SHu2Xdd6tv07uSz2hhG8WxcpaGd/Xbkde7+vFYi7CKfOe0DOll1fhbmdMhNWyjOGJ202G3y3ekJb4gORwtUcM2imPm9faivvNxW78updMy7yKdLb9Dh6u5+nJD4jMSlM318g3n6vv0qb3npfwUtlHE83rNfqQ4XtLwZfl9+jJ/OS14Ky/bKKJ5ZT9SKKuHDq3WDZS8GpBtFMFcpbNy77/vYbKT92H5fu7uNA3LfT2qYRtFLMlk0So7eQ05rpC6Nvge7OQ15JhCOvM4iM1mJ2+e90wBrDTb3Wl8TCE9FDy11MTllXN+sX37WeQd4w07zRbXMR1TSPPrdFX4351LXELarGR6uRuWXFJTs72/SY8qpJdTiw/xJjt573Zra9u80/lTfu3uFXNDNoYXk20UJjt5m7s9Hsp9aMa+Zmt/sscU0pnLfSSPnbwmz5Bl9Xc4Jo9PcNmAaBLSKSFFU+5g72MS0lnjo9xTuW/tmkfJPLW3beqYQnK5j2RC82yj03K3mjcPNzvlzIYIXsr9a9OSh3T6a7mRt+jDT+qN+svjNmvN9m4kHVNI9aSNwYd4F5Kga6d9choeU0hFD/oHFHyEKSE1IaQCcTnCFHYcU0guuBxhCg0IqTxMjjC9vuZpMzsIqTxMjjB1ub+dB0IqD5sjTLs2+Bb7H4xbe/7hkYRU/gxTA5MjTK/r05TL52Dc23n+ISGVh8kRpvOr1TFXpZPn+YdHEpIVJkeYmvy9lOn5h4RUHiZHmJqElOn5h4RUHi5HmHqQ6fmHhFQgLkeYWpDp+YfHFFKmidAAXI4wdSDT8w+PN6TWJkKhKDI9//CYQso0EfpjWHsTSp7nHx5TSJkmQn8Ma29iyfL8w2MKKdNE6I8pdyrZlBzPPzymkDJNhP4Yk7U30OSYQso0EfpzTNbeQINjCinTROiPMVkyAE2OKaRME6E/hpAiOd17ceep5hHkmQiFomg+zOVhyOEnIWSZCIWieBqm9YvS20WLT0g6rpCyTIRCYVyt+vm1LaoNjiwkOEbqF6Wnxd+gbf7FSUhwBNTv6c9aXXhFSNB/XuuHMbf4vm5OSHAE3KV08XrV6rOYCQn6zstpGtZrrp6YbAD4Y24XL0fr+/BtvigREvSbNNzddX/ihizAn3H1yVUghAQQACEBBEBIAAEQEkAAhAQQACEBBEBIAAEQEkAAhAQQACEBBEBIAAEQEkAAhAQQACEBBEBIAAEQEkAAhAQQACEBBEBIAAEQEkAAhAQQACEBBEBIAAEQEkAAhAQQACEBBEBIAAEQEkAAhAQQACEBBEBIAAEQEkAAhAQQACEBBEBIAAEQEkAAhAQQACEBBEBIAAEQEkAAhAQQACEBBEBIAAEQEkAAhAQQwP8BgCScUIrvx3oAAAAASUVORK5CYII=", "text/plain": [ "plot without title" ] }, "metadata": { "image/png": { "height": 420, "width": 420 } }, "output_type": "display_data" } ], "source": [ "confints = rbind(dml_iivm_lasso$confint(), dml_iivm_forest$confint(),\n", " dml_iivm_tree$confint(), dml_iivm_boost$confint())\n", "estimates = c(dml_iivm_lasso$coef, dml_iivm_forest$coef,\n", " dml_iivm_tree$coef, dml_iivm_boost$coef)\n", "result_iivm = data.table(\"model\" = \"IIVM\", \n", " \"ML\" = c(\"glmnet\", \"ranger\", \"rpart\", \"xgboost\"), \n", " \"Estimate\" = estimates,\n", " \"lower\" = confints[,1],\n", " \"upper\" = confints[,2])\n", "result_iivm\n", "\n", "g_ci = ggplot(result_iivm, aes(x = ML, y = Estimate, color = ML)) + \n", " geom_point() +\n", " geom_errorbar(aes(ymin = lower, ymax = upper, color = ML)) +\n", " geom_hline(yintercept = 0, color = \"grey\") +\n", " theme_minimal() + ylab(\"Coefficients and 0.95- confidence interval\") + \n", " xlab(\"\") + \n", " theme(axis.text.x = element_text(angle = 90), legend.position = \"none\",\n", " text = element_text(size = 20))\n", "\n", "g_ci" ] }, { "attachments": {}, "cell_type": "markdown", "id": "907c3978", "metadata": {}, "source": [ "## Summary of Results" ] }, { "attachments": {}, "cell_type": "markdown", "id": "bdeed622", "metadata": {}, "source": [ "To sum up, let's merge all our results so far and illustrate them in a plot. " ] }, { "cell_type": "code", "execution_count": 28, "id": "f7c5acef", "metadata": { "vscode": { "languageId": "r" } }, "outputs": [], "source": [ "summary_result = rbindlist(list(result_plr, result_irm, result_iivm))\n", "summary_result[, model := factor(model, levels = c(\"PLR\", \"IRM\", \"IIVM\"))]" ] }, { "cell_type": "code", "execution_count": 29, "id": "d0658530", "metadata": { "tags": [ "nbsphinx-thumbnail" ], "vscode": { "languageId": "r" } }, "outputs": [ { "data": { "image/png": "iVBORw0KGgoAAAANSUhEUgAAA0gAAANICAMAAADKOT/pAAAAk1BMVEUAAAAAv8QaGhpNTU1fX19oaGh3d3d8fHx8rgCDg4OJiYmMjIyVlZWYmJiampqjo6OlpaWnp6evr6+wsLCysrK5ubm6urq9vb2+vr7BwcHExMTHfP/Hx8fJycnMzMzQ0NDR0dHV1dXY2NjZ2dnc3Nze3t7h4eHk5OTp6enq6urr6+vv7+/w8PDy8vL19fX4dm3///9UK/E/AAAACXBIWXMAABJ0AAASdAHeZh94AAAgAElEQVR4nO2df0PaSBeFs4zUZbFYqa4vXVmoFhGlLN//071JyE9ySUDuTOaE8/xRHaD12OEhyWTmTrAlhJxN0HYAQroARSJEAYpEiAIUiRAFKBIhClAkQhSgSIQoQJEIUYAiEaIARSJEAYpEiAIUiRAFKBIhClAkQhSgSIQoQJEIUYAiEaIARSJEAYpEiAIUiRAFKFI36KX89f09eUB+vtf7+vfvNhJ2HIrUDQqi9H7uHjj4fPIKoglF6gaZN+/fer3fW0Gk9LvfP//avYJoQpG6QcGbb73v2xqRslcQTShSNyiI8t77sq0X6Xfvyk2qC4IidYOiKPH3dSLtP0nOhyJ1g5NEWvKIpA5F6galU7u/tvUi/cVrJHUoUjcoiPK192NbI9Lyx1XviqN22lCkbpCJ8uvr7ryt7j7Sl3fX8boPReoGBU2u3ncPHHj+6itvx1qAInWDXJN/0wf2no+//HvFyyM7UKRuUBnQPnCN9H7V+9dNoguDInWDY0Xa/kpO/YguFKkbHC3S9gdvItmAInWD40Xa/tX7Zj/PxUGRusEJIr33eksHiS4MitQNThApPLn7Yj/QpUGRuoEgUs7+87u5D0QTitQNThLp9xVX9mlDkQhRgCIRogBFIkQBikSIAhSJEAUoEiEKUCRCFKBIhChAkQhR4CJEQlmAA5KTMQUokkeA5GRMAYrkESA5GVOAInkESE7GFKBIHgGSkzEFKJJHgORkTAGK5BEgORlTgCJ5BEhOxhSgSB4BkpMxBSiSR4DkZEwBiuQRIDkZU4AieQRITsYUoEgeAZKTMQUokkeA5GRMAYrkESA5GVOAInkESE7GFPicSFOTfLMxGbsHFuPw2/EifWF90xUgPY+SkzEFPiXSItVmu9wT6TZp3G6PaDoDpOdRcjKmwGdEWmTHn+3czIvPTM1gvtlu5gMzaW66A6TnUXIypsAnRJrnJ3LbsfkoPmWS5soMmpvuAOl5lJyMKXCySKvw/Ow2E2lQUmKRHWoezLKp6RCQnkfJyZgCJ4sUHo2m21SkjXkoPjcx6TjCInxRQ9MhID2PkpMxBU4XafyxzURamufnsTGDh1XcHJtV8qqVGTc1HQLS8yg5GVPgZJE20R+pSPNszC4+axuY7GXRZVB90yEgPY+SkzEFPncfKRUpPBrNI7OW49gkY0qvqG8mvBMCiyxScIDDIg2yy57b6LztMyK5AOQjFCUnYwqcJ1LOIhp2oEjnAZKTMQW0RIoveyjSeYDkZEyB866R9h66KQ8n1DcdAtLzKDkZU0BVJA5/nwdITsYU0BLpw9xEk+nyW66TpqZDQHoeJSdjCpwn0sBskkeeIzdyQeJZDPVNh4D0PEpOxhRoEmn9+nRdfTQVaZLN9bmJp8+l01I/di+ob7oDpOdRcjKmgCTS7PrYUbvNwEwjOZa3uzVGk2yhxLS56Q6QnkfJyZgCgiV3Jwx/fwySKULJ5FUu7DsHkJyMKVC15HVPo/5L9W8Vzs3mt/mk1ZDn8mLy+qYrQHoeJSdjClRFCg9I/fXWBMFsu34Moi/wgPQ8Sk7GFKiK1I/dGQbB4zY+PDkeGLABSM+j5GRMgapIu4ui8Fg0jFr3QfDkNJANQHoeJSdjChwSKTwU9aPWWyIUNCA9j5KTMQUOiZR+2YqjdmCA9DxKTsYUqFpyHQTr3Ze3+AUUyRkgORlTQBy1m+2+RKMNLxTJHSA5GVOgasksCMxr+uW1z2skd4DkZEwB4XBzvTsIZfOE8G8kgfQ8Sk7GFBBEejOxSNGXiDuneawA0vMoORlTQLwAeuwnX8LzOvzjEUzPo+RkTAH8kYQjAOl5lJyMKUCRPAIkJ2MKtC5S2yX+CPk8+ftYuCH7tHbokRNAPkJRcjKmgDhFqGsugfQ8Sk7GFBBF8sKl//zE5q/Md6gmbYv09nTti0taWH33K8J3qCZtixSyfhqmLjkNYwuKpApjChwctZuNdi6NOnBHliKpwpgCdcPfL3fmQBUhMCiSKowp0GDJI0Vq4E8/sfgbUySJOktmIx6RnGL17a8HRRI4ZMma10juoUiaeCDSOh0B74JFW4qkjM136D9eckRw4T7SfWJRJ1ZQxFAkVUCOSMe8/fXwd2aDIhRJFYokcBFz7SiSKhRJ4CJmf1MkVSiSgFCO64gRhmleEHxR3mDipKYrKJIqFElALKLf9JcW+bYuJ22I1Nb+SBRpu/3DSyz+wq2L1HwHNvQoFWmabcI3ObnpDoqkitW3vx5ti9R4RJqbXKR0W9iVGZzcdAdFUoUiCVSleakvZbcKz89uU5Hyjcofos2YT2o6hCKpQpEEpAKRw2D49HroL4RHo2lhV/N04GARbbB8UtMhFEkViiRw6Ibswc2Yzfgj30N2bNLNY1dmfGLTIRRJFYokcLJIm+iPVKRBPgweXfec1HQIRVKFIgmcLFJMKlJhd/Po25OaDqFIqlAkgc8tNlIUyUUZv/9c/BAF/mw7wHH80XaA4/jHwc/wSCQHWC6ipQePSJpc2BHJPtbL0alBkTRBEummPH5wUtMVDgo7akGRNPFBpLf74W63sdGL+DzS8DdF0oYiCYgiDbPRugOzHFKRpoV7rJMTm67AEcly8R81KJLAga0vc5FEk6pThOJpCyc1nQHkEYZJFElA3oy5P0smgc/CL8Jsocqk1Y/dIyc1nQHkEYRJFEmgKlLoTrSFbHIj9k48JGUmTLKVEdOTm+5A8IgiadO2SOEFUrRENhHpbafVHlzYpw9FUqZtkRKD0qlBtVOEQp7Lq8dParoCQiReI+lyXDk6NT4nEhgYInHUTpNjCztq0STSOgjcjgzYAEQk3kfS4/gSqUpURRqVrpEeg2DkLo0lKJIqFElAHLUz621h+Bu/cjFF0sRy8R8dPBApuo9kdveRXu+isqvuwtiCIilivYyWDu1fI23X/eKqPvPmMI0lKJIeDgrS6dD6qN02vkxKGXahfjFF0gNGpNbvI0XsZn8Hw/uDxYSgoEh6UCQZ/JtER0CRFEHxiCLpQ5E0AfGodZH2pjKYDgzbUSRVMDzyTSROEXIHRdLEL5FmFMkdFEmT9kQaBSLCMgpFHBQfY107XVjXLkMWaS2L9GhVJBfwiKQKj0gCpfO2R0Gja3yPKJIuFEngMzv2wUGRVKFIAhTJIyiSJm2L1EEokioUScBfkf7zE5u/MkXSpH2RXu9KCyn8de1Y3ptf4gUUSZPWRZrtD9w5DWQDiqQKRRKoWvJaGQF3GsgGFEkViiRQtSRaXv7YgWWxBWyK9KefWPyNKZJEVSTThbkMZVCOSCA5KZKAeB+pC8vLi4C8QVFyUiSBi7ghC/IGRclJkQSq0lxTpLYAyUmRBKrSPHahJGQZkDcoSk6bIv3jJUcEl7aa6EQtuyIgb1CUnCBHJLf/m/LWl+bY8e+Nydg9sChv3FLfdAXIGxQlJ0USEAcbjr8hu9wTyc+NxkDeoCg5KZLAmSLNzbzYnGabW06am+4AeYNazfmHl1j8hbFEGif7Kyek2y2vzKC56Q6KpApjCpw51D0oKbHIDjUPZtnUdAhIz6PkZEyB80TamIdic2LScYRFtHF5fdMhID2PkpMxBc4TaWmeo/2VBw+ruDk2q+SJlRk3NR0C0vMoORlT4DyR5tmYXXzWNsi3m40ug+qbDgHpeZScjClQFGk3rnDKYEN4NJpvwq/LcWySKahimpoJDqr4EWIJLZEG2WXPbXTe9hmRXADyEYqSkzEFzhMpZxENO1Ck8wDJyZgCajO9o8seinQeIDkZU0BPpNCNm/JwQn3TISA9j5KTMQVUReLw93mA5GRMAS2RPsxNNJkuv+U6aWo6BKTnUXIypsB5Ig3MJvnuOXIjFySexVDfdAhIz6PkZEyB80SaZHN9buLpc+m01I/daEJ90x0gPY+SkzEFzpxrNzDTSI7l7W6N0SRbKDFtbroDpOdRcjKmwJnXSB+DZIpQMnmVC/vOASQnYwqcPdgwv80nrYY8lxeT1zddAdLzKDkZU6BzpbckQHoeJSdjClAkjwDJyZgCFMkjQHIypgBF8giQnIwpIIv0dj808azv0YvTNJYA6XmUnIwpIIo0zJZPBMGd0zh2AOl5lJyMKSBXWi2I1AWTQHoeJSdjCggiXQdBf5as8ov2k311GsgGID2PkpMxBaoihe70t9k2SXddOCSB9DxKTsYUqIo03G3rkoj0ttMKG5CeR8nJmAIHd+xLazV0YQM/kJ5HycmYAhTJI0ByMqZAk0jrIHC8eMgCID2PkpMxBaoijUrXSI9BMLIaoO0Sf4R8njqRZuExaL0tDH/j7ygL8hGKkpMxBeT7SGZ3H+n1Lvzz2mkeK4D0PEpOxhQQRFr3i2VWu7AxM0jPo+RkTAFxSG6UezRcO41jB5CeR8nJmAJ1s7+D4T3+9KAIkJ5HycmYAvg3iY4ApOdRcjKmAEXyCJCcjCkgizTbza8zI/yh7wiQnkfJyZgCkkjr63x6UL8Dg3YoPY+SkzEFDi3s2z3J4W+ngORkTAFBpNCjUTLoPet3YaodSs+j5GRMgapIj0EwzFuhSY8O49gBpOdRcjKmQFWk8AKpcDY368IcIZCeR8nJmAIHl1EcaEIC0vMoORlTgCJ5BEhOxhSoWtIvndrpLuxbcDeKOkByMqZAVaRy3SDVhX3cH6kekJyMKVAV6bW4lu9Vc2HfNNuxz+1ezCg9j5KTMQWEC6CoYPFdXPP7RXdhX7qH7MoM1P7NowDpeZScjCkgLewzpYV9aguS8l3NH+Kdm90B0vMoORlToK6IfrywT+9HTUw6yrBwvBszSM+j5GRMAYcL+8Ym3Wl2ZcaK/24zID2PkpMxBRzeJBrk4+iOL5JAeh4lJ2MKOBTJGOlbF4D0PEpOxhRoXaT/EQJLvUivd6WCXFqu8YjUBEhOxhQQLJkFeyj9KIrUBEhOxhSQZzZYEemGgw0NgORkTAFxrl3waGN9OYe/mwDJyZgCVZGMrTWx08INWbeT7UB6HiUnYwqI65HslCnO9cnnOLgBpOdRcjKmQOPCPkXSSasfjscaUHoeJSdjCog1Gyz9rEm2jMLtVDtCrCNWEbJVX7WthX2E2Easa2etKORzO0vNCbGNXGnVWBn/JqSziIMNdm7IEtJdKBIhClAkQhSgJYQoQJEIUYAiEaIARSJEgboqQuE3oxe3cQjBpK6uXfR0sQ44IeQAh/aQzUSiSYQ0I4h0HQT9WbKaIqrfoFkkkpBuUhUpdKe/zZYl3fGQREgzVZGGu2UUiUhvO60IIXUcXCGbTg3iFCFCmqFIhCjQJJLuHrKEdJSqSKPSNZLqHrKEdBVx1C7apS8f/rZVwYFo0uslXzK+/v07ee7q64+8pM77j69XLeTrOvJ9JLO7j/Squ4cssUhVpJCf6XO9b9kLv/WSlxJNpD1kS3tRWCuEQlTJREof+P3zr17vd/Lgl9yd8HuKpI84JDfKPRraqbpKtKmItI0OPt+TB/9OD07bn+H3FEkfh3vIEptIIv3uXSUPLrNzu2+9XxTJArxJ1BEkkQoPXqUDDOE3FMkCFKkjSCItsyPS9nvvV/ztr/BsjyJZgCJ1BEmkv7JrpFigiO/hxRJFskBRpGohLpbjgqEi0vLHVe/qd/ZgcnC66u2f/hEVKFJHkO4jfXnPn/sWn9v96n2lSFagSB2hItLV15/F5/6Nz+2+9/6lSFaQLBkGweglun8UzWy4d52IfIryqd2/V8k1Ufbgbij8KrpFS5EsIE8RyqbXvXGKEAh710jvV9Gxp/jcX+G53a/ely1FsoK40djjoRbxlv3Bhl+9q/fScz96f4dndn9vKZIVqiL1S5sxr7nUHIPKqN2P3lXpuffwaPSl976lSFZo3IyZgw0YVO8j/ZVNC9o9+KX3c+cWRbJAk0hrioRBVaT3Xm9ZfO57r7cbgaBIFhCrCJWvkYYO45DPIsxs+BEPLWQP/gpF+rX/GqJEVaSn4hokrpBFQZoi9LX3o/jglTixlaggnLdF6/ru4vUTLyOukEVBXEZxtVvZlzz4LblookgWqKv9HUOPCGlGGklYD3OPWK+YkCM4uEI2KtcwvOdCc0KOgWPbhChAkQhRgCIRosDeeqSttCqprWyEwECRCFGAIhGiQNGS2ayj9SDfm1/iBSA5GVOgKFK/qwcfkJ5HycmYAuKpndMEDgDpeZScjClAkTwCJCdjChSlMfFKJIrUGiA5GVOgKM2oOmDXjVE7kJ5HycmYAkVL3ihSu4DkZEyBkiWv1xSpTUByMqZAYxWhLgDS8yg5GVOAInkESE7GFOicNBIgPY+SkzEFKJJHgORkTAGK5BEgORlTQBLp9a7PUbs2AMnJmAKCJTMOf7cESE7GFKha8sr7SG0BkpMxBaqW3IXqPL4JLy0wNck3G5Oxe2AxDr8dL9IX1jddAdLzKDkZU6AqkmneWmyRarNd7ol0mzRut0c0nQHS8yg5GVNAvCHbUBZykR1/tnMzLz4zNYP5ZruZD8ykuekOkJ5HycmYAp+Y2TDPT+S2Y/NRfMokzZUZNDfdAdLzKDkZU6AqzXW9SKvw/Ow2E2lQUmKRHWoezLKp6RCQnkfJyZgC4mbMdTsihUej6TYVaWMeis9NTDqOsAhf1NB0CEjPo+RkTAHh6GMKG40Jz44/tplIS/P8PDZm8LCKm2OzSl61MuOmpkNAeh4lJ2MKyPsjmcPj35voj1SkeTZmF5+1DUz2sugyqL7pEJCeR8nJmALiYEPjDdlUpPBoNI/MWo5jk4wpvaK+6RCQnkfJyZgC54k0yC57bqPzts+I9E4ILFoi5SyiYQcekc4DJCdjCnxuIp0gQnTZQ5HOAyQnYwroiRQ+dFMeTqhvOgSk51FyMqaAqkgc/j4PkJyMKaAl0oe5iSbT5bdcJ01Nh4D0PEpOxhSQRXq9iyvcDe8O7POSj9ptkkeeIzdyQeJZDPVNh4D0PEpOxhSQRFoP8xG7oXhnNhVpks31uYmnz6XTUj92L6hvugOk51FyMqaAPLOhgDhdKJtrNzDTSI7l7W6N0SRbKDFtbroDpOdRcjKmgCBSdFZ39xJ99xKtlr0W/lZ2SPkYJFOEksmrXNh3DiA5GVOgKlJU+2QmNnIK52bz23zSashzeTF5fdMVID2PkpMxBaoiDUtLzR/DyySHcewA0vMoORlToGmp+ZpVhNwBkpMxBRqXmlMkd4DkZEwBHpE8AiQnYwrwGskjQHIypsARo3ZPTgPZAKTnUXIypoBw3tYv30fqO81jBZCeR8nJmAKfm9kABkjPo+RkTAFpJOGtONeuoeoqBCA9j5KTMQUOzP6+j10a3h+Y/Q0GSM+j5GRMAfyx7SMA6XmUnIwpQJE8AiQnYwrIIs12Q3VmVFe8GAeQnkfJyZgC4sK+tI5+NPhte9Cu7cpkhHyeWpHi4e9UJA5/OwQkJ2MKiEX0g1Ey6D3rhyY5zWMFkJ5HycmYAuK2LoXZdf3mjTD9B6TnUXIypoC40VjhbG4mrzXHAqTnUXIypgDXI3kESE7GFKBIHgGSkzEFqpb0S6d26y6MNoD0PEpOxhSoinQXBHd56zEIRg7j2AGk51FyMqZAVaTX4sK+UgMWkJ5HycmYAsIF0LC5QCQYID2PkpMxBQSR1uWFfW0tSPrPT2z+ynyHatK6SLtjUrqwz2kaS1h99yvCd6gmHoi0fbsfmg4t7KNIqjCmAP5NoiOgSKowpgBF8gi+QzWhSOpQJFUYU+BzIk3z2Q6L8k4tJzVdQZFUYUyBT4m0yPdHOmlnsbY2GqNI2+0fXmLxFwYQKfQoFWma7WY5ObnpDoqkitW3vx7eizQ3uUjp/sorMzi56Q6KpApFEjhZpFV4fnabirTIji0P0a7mJzUdYlOkP/3E4m9MkSROFik8Gk2zPWQnJh04WEQ7lZ/UdAjKEcnq218PmyL94yVHBD9dpPFHvhnz2KS7MK/M+MSmQyiSKiBHpGPe/nqcLNIm+iMVaZAPg0fXPSc1HUKRVKFIAodFqltjnopkTOmhk5oJLsr4/efihyjwZ9sBjuOPtgMcxz8OfoZHIrmARyRVeEQSoEgeQZE0oUjqUCRVKJLAeSLdlMcPTmo6hCKpQpEEzhOJw9+qUCRNkESaFu6xTk5sOoQiqUKRBM4TKTcinrZwUtMhFEkViiRwnkjZPNSP3SMnNd1BkVShSAKfW9iXmTDJVkZMT246w3IRLT0okiZQIiEs7LNejk4NiqSJHyLNotp21/cHqkMWzs2ey6vHT2o6wUFhRy0okibtiZRfFb1dpwUi752msQFF0oYiCYgiFYsWw5f+pkjKWC61oIYHIoXHo/4sPKt7ezQdOCbBeIQhkvWiJVq0L9Is9Ch56C00Cb5sMYxHdleI6+Cg/I8S7Ys0KuyJ9FjadgwUGI8ATKJIMpJI4VEoG61768BVEoZIDqqWaAAj0nGlFtSQRCrNaejCZswUSREcj5yaRJF8AUUkjFG748v/KCGJdF0WCX9XcwiRQK6Rthj3kbwQ6bFwjTTrwqZ9GCJhjNptKZLInkhP8Vi3CV7Sx/pd2NUcRCSM+0gYIrV9jRTSHz2OkqPQembyW0rAUCRVIERqedQuI3vAvLmMYweKpAqGSG3O/n592m3CXBCpb9sjB1X8WCBSFxaIzDggUkGn3ZND/OujCB6RVOERSQD/JtERUCRVKJIARfIIiqRJyyK93A/7u4VIw8cX6W8AQpFUoUgCeyK99otDd4F5dBrGFhRJFYokUBbpPtjH+qidCyiSKhRJoCTSLDoGvUUrY6OFsevZiPeRnEKRNGlRJJNp8xYE8VndXReWI1EkXSiSQFGkx/LK2Hji6pBz7dxBkTRpT6Tr8srYeIl5eLY3chrIBhRJFYokINe1ixvxbNV1F2atUiRVKJJAjUjCcllQKJIqFEmgaIkJgmyIbt26SP/5ic1fmSJp0p5Io0I1yKdkuO6tC6d2780v8QKbIv3hJRZ/4RZFmoVHn2RWUFQYMh7/HnWhrp1Nkf70E4u/McrnUov3kaLS+Xev4Xndk9nVPFlHt2TrKq1uTMbugUV5v4n6pitAeh4lJ0jMFkV6KxTPjy+Xoq+1B6Tlnkg+7o8E0/MoOUFitjnXLt/OZTfHrsmj7dzMi81ptiffpLnpDpCeR8lpM+Y/XnJE8P0hudmoH57VpStjzaihgv442RY2Id0ldmUGzU13gLxBUXIypsCZY9uDkhL5vuUPZtnUdAhIz6PkZEyB80TamIdic2LScYRFtN9yfdMhID2PkpMxBc4TaWmeo21hBw+ruDk2q+SJlRk3NR0C0vMoORlT4DyR5tmYXXzWNsjLhEeXQfVNh4D0PEpOxhRoFKl2ilB4NJpvwq/LcWxSYbPz6Nv6pkNAeh4lJ2MKnCfSILvsuY3O2z4jkoMqfoRYQkuknEU07MAj0nmA5GRMAbWp3dFlD0U6D5CcjCmgJ1Loxk15OKG+6RCQnkfJyZgCqiJx+Ps8QHIypoCWSB/mJppMl99ynTQ1HQLS8yg5GVOgItLb02i4m7c6HD02zLTbDswm+e45ciMXJJ7FUN90CEjPo+RkTIE9kWblksVBv75m8SSb63MTT59Lp6V+7EYT6pvuAOl5lJyMKVBej7SnUVxLv+5vbwZmGsmxvN2tMZpkCyWmzU13gPQ8Sk7GFKiskO3fvyQndK+vj/0mkz4GyRShZPIqF/adA0hOxhTYr9mwt44vqqpfX2l1fptPWg15Li8mr2+6AqTnUXIypsBeFaFKVVXpMThAeh4lJ2MKFEXqF+rapbztqqBgA9LzKDkZU+BgpdXaB8EA6XmUnIwpQJE8AiQnYwocKlmcwlM7h4DkZEwBDjZ4BEhOxhTYH/5+Kj/92Dj8jQBIz6PkZEyB5huy+DX0UXoeJSdjCpRHEq6rU4S6sK05SM+j5GRMgb0hucfTJq2CANLzKDkZU6Aytv36mCyjMMPRYweORhEgPY+SkzEF8G8SHQFIz6PkZEwBiuQRIDkZU4AieQRITsYUaF2ktkv8EfJ58vdx6yK5AOQjFCUnYwpQJI8AycmYAhTJI0ByMqYARfIIkJyMKUCRPAIkJ2MKUCSPAMnJmAIUySNAcjKmAEXyCJCcjCmwV7NBxGkgG4D0PEpOxhSgSB4BkpMxBSiSR4DkZEwByZJhEIxe1uE3r3dBcO80jh1Aeh4lJ2MKCCJdF+qdvDUU0ccApOdRcjKmQFWkxyB4PNQCBaTnUXIypkBVpH4QrPPWugtlhEB6HiUnYwpURdobXVAdbFhwW5c6QHIypkCTSGtNkbjRWD0gORlToGrJcP8aaaj1s6bZ1pduNzVH6XmUnIwpUBXpKQhMVoZrplmyON2MeWUGWv/kcYD0PEpOxhQQztuiGpF3cdnil1HTHrKnsMgORA/xFujuAOl5lJyMKSCI9GaO3tX8JCYmHWVYON7WHKTnUXIypoA0krAe5h7dCc9/krFJt2xembHeP3sEID2PkpMxBeQhubf7YVy0+H4tPv05BvmOZY4vkkB6HiUnYwo4nJFqjPTt/wiBxSORXADyEYqSkzEFKJJHgORkTAFJpNe78i5JSj+KIjUBkpMxBQRLZpYW9t1wsKEBkJyMKVC15NXWClkOfzcBkpMxBaqW3IXqWNmqb1q4Iet2sh1Iz6PkZEyBqkjG1lK+XJ98joMbQHoeJSdjCojLKDRvwxZIJ61+OB5rQOl5lJyMKdC4sE+RSbaMwu1UO0KsU5Xm2l79rbYW9hFiG7H4idoKpH2e21lqTohthKOPKSzsI4Qcg7weyVgZ/yaks4iDDV0rWUyIbSgSIQpQJEIUoCWEKECRCFGAIhGiAEUiRIEmkdavTx3YIIkQy0giza45akfISQiW3HH4m5ATaV5q3n9pIRYhWIhLzfvraJ3sbLt+DCxOBSekO4hbX86yXZLCw5Pj1ayEIHJohWy6wdh9EDy5zkQIHIdEek02YX5T3LGPkM5ysGZDOlrHUTtCmhFrNo1Ys3AAABcMSURBVKx3X+LFfRQJgV4v//q71ysW0Hnv9X5vt1dff+QPvv/4euU2X/cRR+1muy/RaMMLRUKgKNL2r97fhaf+7v0VP9H7lj30rZe+nmhRtWQWBOY1/fLa5zUSAiWRfvaKx5ur3s/4iS+5O+H3FEkZ4XBzvTsIZfOEeCPJf0oihe78yp5JrOr1/o6F2j30N0XS5sBmzNt8T2bFXWSJLcoifS+dxn3fPbHMHvzW+0WRtBEvgB77yZfwvI7HIwTKIi3j8YWY36FByRNX6Qlf+A1F0oYjCZ2gLNL2S+/f5Il/e1/SJ74nJ3y/wmMURdKGInWCPZFSfXKlwid+7U7yQqF+UiR1KFIn2BMpu5X0XnwiGcy76m0pkjoUqRPsiZQOMeTDDtET3+Jzu1+9rxRJH4rUCfZF+pUdfX7lT/wb2/U9OtmjSNpQpE6wL1J4bRTdNfqZXSvt5g5Fdl1FI3oUSRuK1AkqIv0dn9N9zSYLJXOHfoXHqi9biqQPReoEFZF+R7eSfuf3k+InfoRefY/dokjaUKROUBEpPBj9CMX5WnrBe3g0+hKP51EkbShSJ6iK9DOW5mf5BeED6dQ7t/m6D0XqBFWRolnfhVng6SS8Xjr1zmW6S6BOpDsT9B+dJSFnIIiUSVN44lf42K+91xEVJJHWd/Gk1d06ChYsRkAQ6b1XXCmbPHGVfKVI2hxeRvGYLEfiuj4ABJHCC6K/Ki/4lsx0oEjaiLuaxyKZ6Gj0Gv756jwUIWjIS83f4jpcwUt8XOLKPkKaqIo02i0uv98dl9ZJfTtCSA1iyeKkHFc8zsAqQoQ0U1cg8q7QJITUUFOyeFc+iCIR0kxNEf34DO+Fd5IIaaYq0jAerUsvkcLWvfNQhKBRFekpUiiqVPy43a5HQVICnBBSw4EbssHuzC760oED0nvzS7wAJCdjCggiveSVirvhEUrPo+RkTAFpSO4tPKG7jrdg7t91Yn4QSM+j5GRMgYsY2wbpeZScjClAkTwCJCdjChyc2ZBiOnAfCaTnUXIypkCjSF2Y2QDS8yg5GVOgSaQZRXIHSE7GFChaMgpEWlpG8Z+f2PyV+Q7VpD2R1rJI+PVPrL77FeE7VJMWT+0eBY2u8T2iSLowpkDjYEMXoEiqMKYARfIIvkM1aVukDkKRVGFMAYrkEXyHatK+SK93/dJ4g9NANqBIqjCmgGDJbH/grvqSqUm+2ZiM3QOLcfjteJG+sL7pCoqkCmMKVC15rYyAV16ySLXZLvdEuk0at9sjms6gSKowpkDVkrvoHmzt8vJFdvzZzs28+MzUDOab7WY+MJPmpjsokiqMKVAVyTTNZZjnJ3Lbsfko/d2kuTKD5qY7KNJ2+4eXWPyFWxcpqcN1iFV4fnabiTQoKbHIDjUPZtnUdAhFUsXq218PD0Sq+wvh0Wi6TUXamIficxOTjiMswhc1NB1CkVShSAJVaa4bRBp/bDORlub5eWzM4GEVN8dmlbxqZcZNTYdQJFUokkBVmsekVvEBNtEfqUjzbMwuPmsbmOxl0WVQfdMhFEkViiQg1rUzTTUhU5HCo9E8Mms5jk0ypvSK+mbCuwP+c/FDLoc/LP7b/3jJwbi1Ir2FJtWPf2ciDLLLntvovO0zIrnA5hHpTz+x+BujHJH+cfrTxMGGxpkNVREW0bDDJYqkidW3vx4USUBLpPiyhyKdB0XSBFWk8KGb8nBCfdMhFEkViiTwuandB0Ti8Pd5UCRNMEX6MDfRZLr8luukqekQiqQKRRI4T6SB2SSPPEdu5ILEsxjqmw6hSKpQJIHzRJpkc31u4ulz6bTUj90L6pvuoEiqUCQBWaS3+6GJBxlGL+Lz2Vy7gZlGcixvd2uMJtlCiWlz0x0USRWKJCCKNMxG64LgTnpBdkj5GCRThJLJq1zYdw4USZPWRXozQUEk0aTCudn8Np+0GvJcXkxe33QFRVKFIgkIIl0HQX+WrKaI6jfgb9pHkVShSAJVkWa7svnJjdi7Ayd3UFAkVSiSQFWkYbYPc9R6a203CkUokioUSeDgCtl0ahDr2rmDImlCkdShSKpQJIEmkdZB4PjuqQUokioUSaAq0qh0jfQYBCOngWxAkVShSALiqJ1ZbwvD33UVHCCwvGGlHhRJk7ZFiu4jmd19pNeo6uq10zwWsL71qxoUSZPWRVqX9qJoLITiOw42UdaCImnSukil7c2HdVVXIaBI2lAkgbrZ38HwHn96EEVShyIJ4N8kagTGI4qkyoWJZLHYYErokYOfosCfbQc4ij/+sFkhUo/DZR31yN/HrYvkAozjEcgRyfp2LFq0d0TKliA1luMCgyLp4WBjIyUokjoUSQ8Ykf75x6lJFMkfLJfs1gFFpLj4vcOfh2/JEWCIZL34vQ5AHrk0iSL5goNtJHRA8IgiWYEiqQLgkS8izXbLy80IfuZ3DEVSBUEkL66R1tf56tg++pzVCAiRUK6RQERqc9QuIa5rt3uyC7O/tygiYYzabVFEan+KUOjRKJnzPet3YaU5ikgQ95G2FElE3NV8mLdCkx4dxrEDRVKFIglURQovkApnc7MOLJGlSLpQJIGDVYQONCGhSKpQJAGK5BEUSZO2ReqXTu3kunbT/LFFeYOJk5quoEiqUCSBqkjlsvliXbtFvq3LSRsicX+keiiSJm2L9FosZVdqpIQepSJNs034Jic33UGRVKFIAsIFULRf31285eWLWNdubnKR0m1hV2ZwctMdFEkViiQg1bUzxcVIZq8e1yo8P7tNRco3Kn+INmM+qekQiqQKRRKo20M2rmu3/1x4NJoWdjVPBw4W0QbLJzUdQpFUoUgCJ9e1M+OPfA/ZsUk3j12Z8YlNh1AkVSiSwMk3iTbRH6lIg3wYPLruOanpEIqkCkUS+Nzd1lSkwu7m0bcnNR1CkVShSAKti+Sgit87SH1IkAKR7xj1IVssEHl8FSEekazAI5ImCHXtKJIVKJImSCLdlMcPTmo6hCKpQpEEipbMZsfu4sLhbytQJE3aE6l/9IqJVKRp4R7r5MSmQyiSKhRJQDy1a/xb1SlC8bSFk5oOoUiqUCSB80TK5qF+7B45qekOiqQKRRIoSmPiQicniTTJVkZMT266gyKpQpEEitKMqgN2DaN2XNinCkXSpD2R3j4h0va5vHr8pGYt//nJSf+7J0KRNGlxitDr9ZEigfHe/BIvsCnSH15i8Rdufa5dF8zZgyKpAvLf2fYNWYp0En/6icXf2Op/5z9eckTwz92QBQPkIxQlJ2MKfO4+EhggPY+SkzEFKJJHgORkTIHP3ZAFA6TnUXIypsDnbsiCAdLzKDkZU+BzN2TBAOl5lJyMKcAbsh4BkpMxBXhD1iNAcjKmAEXyCJCcjCnQOWkkQHoeJSdjClAkjwDJyZgCrYvkoIofIZbI38d1RfTDb0YvbnSyC8hHKEpOxhSo29Ylerq4DSYsID2PkpMxBQSR3kxQEKkLJoH0PEpOxhQQRLoOgv4sGQWfhV+OrRrpLyA9j5KTMQWqIoXu9LfZ7aS7LhySQHoeJSdjClRFGu72MU9EettphQ1Iz6PkZEyBgzMb0gkOXZjoANLzKDkZU4AieQRITsYUaBJpHQSO6wtbAKTnUXIypkBVpFHpGukxCEZOA9kApOdRcjKmgDhqZ9bbwvD3zGkgG4D0PEpOxhSQ7yOZ3X2k17vwz2uneawA0vMoORlTQBBp3S8ujzVvTvNYAaTnUXIypoA4JFeogjJcO41jB5CeR8nJmAJ1s7+D4T3+9KAIkJ5HycmYAmfeJNqYjN0Di/LGLfVNV4D0PEpOxhQ4U6Tlnkh+bjQG0vMoORlT4EyR5mZebE6zzS0nzU13gPQ8Sk7GFJBFer2LK9wN75qukcbJ/soJ6XbLKzNobroDpOdRcjKmgCTSelgYtasf/R6UlFhkh5oHs2xqOgSk51FyMqZA3QrZ5vtIG/NQbE5MOo6wiDYur286BKTnUXIypoA8syG4i4uevDTNbFia52h/5cHDKm6OzSp5YmXGTU2HgPQ8Sk7GFBDn2uXT6xrm2s2zMbv4rG2QTxSPLoPqmw4B6XmUnIwpIK6Qfcxbj+Fl0uG/HR6N5pvw63Icm2QKqpimZkLblckI+Tx1IoXHoMK0oHXtwr5BdtlzG523fUYkF4B8hKLkZEyBxiL6R66QXUTDDhTpPEByMqbAeUekAtFlD0U6D5CcjClw3jVSgciNm/JwQn3TISA9j5KTMQWOGLV7OuofikTi8Pd5gORkTAHhvK1fvo90XFm7D3MTTabLb7lOmpoOAel5lJyMKXDezIaB2STfPUdu5ILEsxjqmw4B6XmUnIwpII0kvBXn2tWukJ1kc31u4ulz6bTUj91oQn3THSA9j5KTMQUOzP6+j11qXCG7GZhpJMfydrfGaJItlJg2N90B0vMoORlT4Mz1SB+DZIpQMnmVC/vOASQnYwqcXY94fptPWg15Li8mr2+6AqTnUXIypgB+Ye8jAOl5lJyMKVAnUv++AzXtIkB6HiUnYwpURHrM7hu9BF2o+x0B0vMoORlTYE+kJxME6Ubm910ptIrS8yg5GVOgLFI0kyG4T1sv0Ri46UCpVZCeR8nJmAIlkZ6CuH5+TjTV7rg5q14D0vMoORlToCjS2lQuiriruUtAcjKmQFGkJ2GG6pC7mrsDJCdjChRFGgqFTl66sEESSM+j5GRMgaJI5bWx+YPu0lgCpOdRcjKmwJ5IwgsokjNAcjKmAEXyCJCcjClQtOQ6CCq3X195jeQOkJyMKVAU6U6oz3Bve5pQ2yX+CPk8skgvwvC3qS1ZDALIRyhKTsYUKF0A9Ss3jcKzPcfLwm0A0vMoORlToCTS616RhrdoY4qX/b+CB0jPo+RkTIHykNxjNGl1NIsnBb0+DUtTWIEB6XmUnIwpsDe2PQvKGPwLpC1Mz6PkZEyB/ZtE61FRo7sOrKHYwvQ8Sk7GFKjebV0/jZKdmDtxNIoA6XmUnIwpgD9t4QhAeh4lJ2MKUCSPAMnJmAIUySNAcjKmAEXyCJCcjClAkTwCJCdjClAkjwDJyZgCFMkjQHIypgBF8giQnIwpQJE8AiQnYwq4FWnBbV3qAMnJmAJOReJGY/WA5GRMAZciTbOtL91uao7S8yg5GVPApUjpZswrM3D4U7cwPY+SkzEFHIq0yA5ED/EW6O4A6XmUnIwp4FCkiUlHGRaOtzUH6XmUnIwp4FCksUm3bF6Zsbsfu4XpeZScjCngUKRBXo/I8UUSSM+j5GRMAYciGSN9+z9CYPFIJBeAfISi5GRMAYrkESA5GVOAInkESE7GFHAo0g0HGxoAycmYAhz+9giQnIwp4FCkaeGGrNvJdiA9j5KTMQVamSKUz3FwA0jPo+RkTIE2Jq1+OB5rIMQ6LkWaZMso3E61I8Q6F7GwjxDbuF1q/tzOUnNCbHMRxU8IsQ1FIkQBikSIAhSJEAUoEuk2xZuWG3uTPCkS6Talu//2pgJQJNJtiu4sKBIhJzM3FR5s/SyKRLrL7Z5GA2seUSTScRzNkKZIpNtQJHUcDYSSS+RSRXJdf+UUJpOPtiN0kPmNuXm2989fqkgWB0LPxphN2xE6xfxmm47g2StxcCEiuRwIPRuPHUfkNv7/HJjb7WqQ1d9R50JEcjkQejYTY/EU5OJYmJvwTHlpoq2EFryPpAHMJ/2DmVr75Lw4xvFmXJNd53OunQYoIu0dPNuOA87uP/BmV0qRU4QuCYqkSfwfuDFmnjWscGEirSY38f/lmKdOl0Lc3wsTDzN8mBtbP+ayRHpIP+KN4xKVpDXiStnJJdKDvfHvixJpbG4W2ScU79VcBs/mNipJ+hCe303SEqUWuCSRFvGBfffZNPG7SCVPQfWI73wMPuJrT3vnIZck0jj+f9yJ9OF1lUqegmoyGZhxdCQaTCyehVySSMmtBFNo+AnGKSjnABehSP4BcgqKMgfYDZck0qAgks8foSCnoChzgJMrTnM7sXnBeUkiPRTeoHPvJ636fOSEmgMcjX3bD3lJIq3MYJPfoFu2Hecg/osENgd4N3VxNbVp0iWJFG3QFF/EryY2V6acDcgpqJ+KV1jEQ98xHwN7H58XJdJ2mn2EeuwRyikoiEgPu1l2MRb/Ny9LpO1memv9svNsQE5BQVbEF28gsGTxZYFxCurxLa4iLFl8wUCcgoKc2pWPSBTpskA4BQVZEc9rJAtMb7hgTg+MFfHlUTtrUxcvSqQBV54qgrKQ94H3kZSZRtfwRAsUkZK59JzZoMYNyJqEvUkD4ynE6JjHrCb2rzgvSSSfPzWLVOaxDSBu2Fw2FMlDns3tIjoKbZaT8CC6mfo7TYikXJJID/YK1qqyKZzLzyOHiiO4fvKxHLcdoQYuo9BlaXzu7ZxJoUTHJnJo5eeipI+H4vln22kOw2UU2kzM2NuZawVuSpNa8tWynrEp3k248fc/lssotEEZr63ODvMy7NTcbOIlH5tnizc6z4bLKNRBEem20N3L6KzO4hSxM7iN7BnvKpgOBt4O0XOK0MXybPK35U00oW3u5cVdbPd89zad+1yjhcsoLpVbM5hHpyMf82h7rFAsL8+cYpFWO8c3fg6HRHAZxQWTD4dN4zNSHw9ICKUlIriM4pL5mI6jOx/T6Lhkbvy8i3Qbv0MHu7F6f0XiNRLxm0l8wrl7ny7t7ZdyLlxGoc/HhOuR9FiZwSp+n662qxuPl/JyGYU2H1yPpMpu06HdvAGfZwNyGYUyD+bW3/vvJUBW8i7i87n5jRn4ezyK4DIKXQzIpFWu5AXkskRqO8FxcCUvIJck0i1GITaYlbxuzpkU2MW0u9L4kkRaeDy0VATlyLkdZ6efXt4xTsljWpzHdEkibSfmwfPPzhgUkdKZTKv5wGeTijHtfZJelEirG4iLeJCVvPN8bq3NO53n8pzfK+YNWR1WIMsoQFbyFld7LPzdNKMc09r/7CWJdItyHwljJS/IHrKc/a0OyPYJKAsQQUS6oUja+NvZZUBEui1cyi39PbUrlpJZ2ls2dUkiodxHAqFY2+jG36XmxeJmN6zZoMHK349NSBbm5jleyOt18ZNooX5cbjOKae9G0iWJFA3aAFzEo2AE2s5UxmXCSxLJ604/gMclTClSEYrkISglTEnOJYmEAkoJU1KAIvkHSAnTyYS7zeRQJP8AKWGKcn/bDRTJP2BKmLad4CjKF8bW9j+8EJH8H2EqAFLCdBJVU/afSr/b2f+QIvkHSAnT7cOuzJXvuNn/8EJEggKkhCnI55Kj/Q8pkn+AlDAFEcnR/ocUyT9QSphi4Gj/Q4rkISglTCFwtP/hJYnkaCBUAZQSpgg42v/wckWyNhBKvMLR/oeXJJKjgdCz4dwbVdzsf3hJIjkaCD0bzr3Rxcn+h5ckkqOB0LPxdygZFBf7H16SSI4GQs8GZO4NKXJJIjkaCD0fkLk3pMAlieRoIPRsQKYMkCKXJJKjgdCzoUia3JQO7tzVXAM3A6HEK4qbuSwGLH6igpOBUOIVy4FJDkqbscUdki5LJCcDocQzHnb+PGdG2eDCRCKXSHRQWoafoDY/OCkSuQCic/pbqxOvKBLpPh/RZswWz+u2FIlcAHNjxh8PVvdipkik66xuzCCac7XkYAMhn2YaHo6S+/A2D0oUiXQbM8jvui95Q5aQz/FQ01KEIhGiAEUiRAGKRIgCFIkQBSgSIQpQJEIUoEiEKECRCFGAIhGiAEUiRAGKRIgCFIkQBSgSIQpQJEIUoEiEKECRCFGAIhGiAEUiRAGKRIgCFIkQBSgSIQpQJEIUoEiEKECRCFGAIhGiAEUiRAGKRIgCFIkQBSgSIQpQJEIUoEiEKECRCFGAIhGiAEUiRAGKRIgCFIkQBSgSIQpQJEIUoEiEKECRCFGAIhGiAEUiRAGKRIgC/wd3pj2yhVFF4wAAAABJRU5ErkJggg==", "text/plain": [ "plot without title" ] }, "metadata": { "image/png": { "height": 420, "width": 420 } }, "output_type": "display_data" } ], "source": [ "g_all = ggplot(summary_result, aes(x = ML, y = Estimate, color = ML)) + \n", " geom_point() +\n", " geom_errorbar(aes(ymin = lower, ymax = upper, color = ML)) +\n", " geom_hline(yintercept = 0, color = \"grey\") +\n", " theme_minimal() + ylab(\"Coefficients and 0.95- confidence interval\") + \n", " xlab(\"\") + \n", " theme(axis.text.x = element_text(angle = 90), legend.position = \"none\",\n", " text = element_text(size = 20)) + \n", " facet_wrap(model ~., ncol = 1)\n", "\n", "g_all" ] }, { "attachments": {}, "cell_type": "markdown", "id": "62b2d3d5", "metadata": { "papermill": { "duration": 0.079866, "end_time": "2021-03-23T13:52:00.452482", "exception": false, "start_time": "2021-03-23T13:52:00.372616", "status": "completed" }, "tags": [] }, "source": [ "We report results based on four ML methods for estimating the nuisance functions used in\n", "forming the orthogonal estimating equations. We find again that the estimates of the treatment effect are stable across ML methods. The estimates are highly significant, hence we would reject the hypothesis\n", "that 401(k) participation has no effect on financial wealth." ] }, { "attachments": {}, "cell_type": "markdown", "id": "c29ea3cb", "metadata": {}, "source": [ "______\n", "\n", "**Acknowledgement**\n", "\n", "We would like to thank [Jannis Kueck](https://www.dice.hhu.de/en/dice/people/professors-1/kueck) for sharing [the kaggle notebook](https://www.kaggle.com/janniskueck/pm5-401k). The pension data set has been analyzed in several studies, among others [Chernozhukov et al. (2018)](https://arxiv.org/abs/1608.00060).\n", "\n" ] } ], "metadata": { "kernelspec": { "display_name": "R", "language": "R", "name": "ir" }, "language_info": { "codemirror_mode": "r", "file_extension": ".r", "mimetype": "text/x-r-source", "name": "R", "pygments_lexer": "r", "version": "4.2.2" }, "papermill": { "default_parameters": {}, "duration": 503.595131, "end_time": "2021-03-23T13:52:50.313398", "environment_variables": {}, "exception": null, "input_path": "__notebook__.ipynb", "output_path": "__notebook__.ipynb", "parameters": {}, "start_time": "2021-03-23T13:44:26.718267", "version": "2.3.2" } }, "nbformat": 4, "nbformat_minor": 5 }