1. The basics of double/debiased machine learning#
In the following we provide a brief summary of and motivation to the double machine learning (DML) framework and show how the corresponding methods provided by the DoubleML package can be applied. For details we refer to Chernozhukov et al. (2018).
1.1. Data generating process#
We consider the following partially linear model
\[ \begin{align}\begin{aligned}y_i = \theta_0 d_i + g_0(x_i) + \zeta_i, & &\zeta_i \sim \mathcal{N}(0,1),\\d_i = m_0(x_i) + v_i, & &v_i \sim \mathcal{N}(0,1),\end{aligned}\end{align} \]
with covariates \(x_i \sim \mathcal{N}(0, \Sigma)\), where \(\Sigma\) is a matrix with entries \(\Sigma_{kj} = 0.7^{|j-k|}\). We are interested in performing valid inference on the causal parameter \(\theta_0\). The true parameter \(\theta_0\) is set to \(0.5\) in our simulation experiment.
The nuisance functions are given by
\[ \begin{align}\begin{aligned}m_0(x_i) &= x_{i,1} + \frac{1}{4} \frac{\exp(x_{i,3})}{1+\exp(x_{i,3})},\\g_0(x_i) &= \frac{\exp(x_{i,1})}{1+\exp(x_{i,1})} + \frac{1}{4} x_{i,3}.\end{aligned}\end{align} \]
Note
In Python the data can be generated with
doubleml.datasets.make_plr_CCDDHNR2018().In R the data can be generated with DoubleML::make_plr_CCDDHNR2018().
In [1]: import numpy as np
In [2]: from doubleml.datasets import make_plr_CCDDHNR2018
In [3]: np.random.seed(1234)
In [4]: n_rep = 1000
In [5]: n_obs = 500
In [6]: n_vars = 20
In [7]: alpha = 0.5
In [8]: data = list()
In [9]: for i_rep in range(n_rep):
...: (x, y, d) = make_plr_CCDDHNR2018(alpha=alpha, n_obs=n_obs, dim_x=n_vars, return_type='array')
...: data.append((x, y, d))
...:
library(DoubleML)
set.seed(1234)
n_rep = 1000
n_obs = 500
n_vars = 20
alpha = 0.5
data = list()
for (i_rep in seq_len(n_rep)) {
data[[i_rep]] = make_plr_CCDDHNR2018(alpha=alpha, n_obs=n_obs, dim_x=n_vars,
return_type="data.frame")
}
1.2. Regularization bias in simple ML-approaches#
Naive inference that is based on a direct application of machine learning methods to estimate the causal parameter, \(\theta_0\), is generally invalid. The use of machine learning methods introduces a bias that arises due to regularization. A simple ML approach is given by randomly splitting the sample into two parts. On the auxiliary sample indexed by \(i \in I^C\) the nuisance function \(g_0(X)\) is estimated with an ML method, for example a random forest learner. Given the estimate \(\hat{g}_0(X)\), the final estimate of \(\theta_0\) is obtained as (\(n=N/2\)) using the other half of observations indexed with \(i \in I\)
\[\hat{\theta}_0 = \left(\frac{1}{n} \sum_{i\in I} D_i^2\right)^{-1} \frac{1}{n} \sum_{i\in I} D_i (Y_i - \hat{g}_0(X_i)).\]
In [10]: def non_orth_score(y, d, l_hat, m_hat, g_hat, smpls):
....: u_hat = y - g_hat
....: psi_a = -np.multiply(d, d)
....: psi_b = np.multiply(d, u_hat)
....: return psi_a, psi_b
....:
In [11]: from doubleml import DoubleMLData
In [12]: from doubleml import DoubleMLPLR
In [13]: from sklearn.ensemble import RandomForestRegressor
In [14]: from sklearn.base import clone
In [15]: import numpy as np
In [16]: from scipy import stats
In [17]: import matplotlib.pyplot as plt
In [18]: import seaborn as sns
In [19]: face_colors = sns.color_palette('pastel')
In [20]: edge_colors = sns.color_palette('dark')
In [21]: np.random.seed(1111)
In [22]: ml_l = RandomForestRegressor(n_estimators=132, max_features=12, max_depth=5, min_samples_leaf=1)
In [23]: ml_m = RandomForestRegressor(n_estimators=378, max_features=20, max_depth=3, min_samples_leaf=6)
In [24]: ml_g = clone(ml_l)
# to speed up the illustration we hard-code the simulation results
In [25]: theta_nonorth = np.array([0.52328641, 0.57356639, 0.55810707, 0.5977535 , 0.4813668 , 0.5327912 , 0.61704155, 0.54681571, 0.52857031, 0.6247725 , 0.56574599, 0.42765337, 0.46968109, 0.37845713, 0.48736779, 0.54648086, 0.61232815, 0.45526944, 0.58970519, 0.49146413, 0.61463843, 0.60010516, 0.58994949, 0.47546871, 0.53585469, 0.53000956, 0.51277996, 0.54949396, 0.58818701, 0.48921812, 0.49379525, 0.61029898, 0.56305835, 0.58602554, 0.57746665, 0.47510051, 0.42459541, 0.55138846, 0.51448972, 0.49390206, 0.46970825, 0.48389275, 0.5122035 , 0.35942868, 0.54754709, 0.48853634, 0.53462013, 0.57257894, 0.50344216, 0.7003655 , 0.55803537, 0.5603601 , 0.55943751, 0.53039375, 0.57273209, 0.588119 , 0.60273543, 0.57463947, 0.58192052, 0.39363028, 0.5904183 , 0.70081522, 0.5915527 , 0.68069836, 0.60378249, 0.47595378, 0.57105257, 0.57417753, 0.59693217, 0.49271695, 0.54860756, 0.51489703, 0.44138932, 0.50798106, 0.59197585, 0.46038645, 0.61738642, 0.49256698, 0.46462596, 0.5088013 , 0.48879602, 0.40785025, 0.49317757, 0.52101581, 0.36721687, 0.55132674, 0.6145169 , 0.61903097, 0.60885654, 0.60973684, 0.40244356, 0.46518551, 0.6791729 , 0.56333526, 0.55315272, 0.50280275, 0.48186332, 0.45825446, 0.53875985, 0.54707233, 0.55333261, 0.63615035, 0.51991688, 0.55070896, 0.44300732, 0.69778352, 0.64837045, 0.60167985, 0.50493321, 0.55693924, 0.5686461 , 0.6367919 , 0.43807751, 0.42727251, 0.55899025, 0.512766 , 0.42414525, 0.5123373 , 0.54132912, 0.50466702, 0.50510984, 0.63393837, 0.52571856, 0.6110036 , 0.35394133, 0.64759649, 0.426197 , 0.52952409, 0.5371253 , 0.44604935, 0.49752703, 0.57413701, 0.48082096, 0.48286192, 0.48941886, 0.49669565, 0.61011785, 0.5499223 , 0.38338078, 0.5684348 , 0.62734346, 0.51551337, 0.58548162, 0.39843792, 0.51066774, 0.51682297, 0.41662497, 0.51405945, 0.47680314, 0.57564587, 0.55047709, 0.40895692, 0.48507321, 0.48150446, 0.64835334, 0.55744201, 0.4697698 , 0.4778423 , 0.46274545, 0.72789211, 0.47809226, 0.48196451, 0.551625 , 0.54276761, 0.60078249, 0.55327398, 0.62120112, 0.53889818, 0.57168766, 0.59412325, 0.55290987, 0.61479866, 0.45270291, 0.55090591, 0.49514174, 0.52680104, 0.56749322, 0.56505284, 0.4992059 , 0.55127505, 0.57524686, 0.52585235, 0.53229655, 0.47622374, 0.44488789, 0.50545333, 0.56539543, 0.54838977, 0.57060796, 0.62361319, 0.44563308, 0.5003698 , 0.45594423, 0.46606284, 0.45355246, 0.52816031, 0.57419905, 0.5410551 , 0.53759333, 0.56140027, 0.60997517, 0.63110121, 0.43629724, 0.47555077, 0.49690284, 0.54023968, 0.54261783, 0.49963926, 0.65625668, 0.66449324, 0.43884447, 0.57129263, 0.33584012, 0.47511805, 0.34028965, 0.57814497, 0.53450725, 0.61166678, 0.54127341, 0.54147 , 0.55800002, 0.51160151, 0.58028185, 0.59212372, 0.46545939, 0.55286645, 0.49703378, 0.49730541, 0.58857442, 0.56035477, 0.52518858, 0.66457484, 0.73078869, 0.48474359, 0.52586322, 0.60917992, 0.61026355, 0.41290474, 0.51979286, 0.66831908, 0.42185021, 0.48317411, 0.43697095, 0.64413848, 0.55197041, 0.47930367, 0.56469123, 0.44890795, 0.37844697, 0.35164074, 0.50716352, 0.59215554, 0.60447157, 0.60752355, 0.61793807, 0.44814341, 0.64305071, 0.59159908, 0.65125645, 0.58544308, 0.3600717 , 0.45363031, 0.61097706, 0.52458445, 0.42697636, 0.48711005, 0.55300448, 0.37965628, 0.61771679, 0.5089207 , 0.53578385, 0.43862644, 0.56336916, 0.61680458, 0.62323204, 0.47641849, 0.56843699, 0.54752781, 0.59708379, 0.61883269, 0.50909783, 0.43850491, 0.58974292, 0.49420969, 0.58473545, 0.42414872, 0.69756484, 0.55453778, 0.63226348, 0.55137947, 0.54003995, 0.62681415, 0.51932606, 0.50475418, 0.44882915, 0.5472079 , 0.61434172, 0.57540472, 0.61154333, 0.39089563, 0.59718904, 0.57288303, 0.53569031, 0.47663109, 0.4728607 , 0.60943294, 0.67021391, 0.55689497, 0.59862612, 0.5217708 , 0.4728515 , 0.61083456, 0.4769726 , 0.5025934 , 0.59237259, 0.57291053, 0.55393486, 0.47016436, 0.46321853, 0.57593761, 0.59413597, 0.40312949, 0.69317039, 0.58008741, 0.54355454, 0.46655708, 0.53293769, 0.57314345, 0.58849483, 0.46739483, 0.46984072, 0.51811592, 0.34895571, 0.34717479, 0.58424658, 0.34806726, 0.49737285, 0.41588973, 0.49969555, 0.56463711, 0.58363827, 0.44714116, 0.62826166, 0.47811024, 0.43887814, 0.53916079, 0.59560763, 0.5491493 , 0.48654848, 0.46210772, 0.46522759, 0.57971661, 0.38666085, 0.52764583, 0.6287972 , 0.65112072, 0.50850476, 0.46352321, 0.58135214, 0.57978432, 0.58933261, 0.47563279, 0.56898215, 0.4296174 , 0.68463977, 0.64897991, 0.57323481, 0.66167288, 0.553377 , 0.55324918, 0.40643082, 0.49186567, 0.51094983, 0.43758268, 0.45570856, 0.61405943, 0.4608622 , 0.50980897, 0.34744005, 0.46259612, 0.57727937, 0.42545334, 0.58734005, 0.43360918, 0.73543086, 0.53872757, 0.48093209, 0.61192705, 0.61599023, 0.55635875, 0.48847344, 0.50631136, 0.55652377, 0.46278946, 0.54833289, 0.40865905, 0.58969522, 0.50159954, 0.54574829, 0.51228323, 0.47260273, 0.53563636, 0.49616576, 0.5914049 , 0.47119647, 0.64848706, 0.56442955, 0.53782256, 0.59228539, 0.45564465, 0.55603929, 0.40188956, 0.47684626, 0.56215903, 0.45795109, 0.48627697, 0.63027951, 0.48734554, 0.51792059, 0.41021068, 0.49752463, 0.66428508, 0.40135928, 0.48423971, 0.61611091, 0.5681857 , 0.55489614, 0.57716277, 0.50822594, 0.61446625, 0.63694847, 0.47051379, 0.61684335, 0.50454219, 0.46705443, 0.57543441, 0.62874235, 0.61278484, 0.55511877, 0.49467431, 0.57886118, 0.62086825, 0.52985768, 0.57074016, 0.51147937, 0.43235764, 0.54675407, 0.53570403, 0.47359363, 0.54927324, 0.60401485, 0.44723584, 0.52033308, 0.5283196 , 0.56765117, 0.58193342, 0.50924812, 0.53098169, 0.44647117, 0.5860353 , 0.50867477, 0.47155282, 0.48603775, 0.46297341, 0.48213296, 0.42373343, 0.51347689, 0.537851 , 0.47948647, 0.55109115, 0.41575012, 0.57976377, 0.43576637, 0.49168309, 0.48862993, 0.55380779, 0.53885336, 0.60138872, 0.39765241, 0.60137404, 0.53692952, 0.44201017, 0.60829564, 0.52805651, 0.42568115, 0.43395763, 0.51049196, 0.59671629, 0.53932093, 0.61288662, 0.48532588, 0.44755878, 0.39897815, 0.48546131, 0.52340699, 0.62242501, 0.61332726, 0.47966986, 0.39922943, 0.56655363, 0.47240258, 0.70180277, 0.54329484, 0.57983972, 0.48174702, 0.40806803, 0.38012296, 0.53096472, 0.63231761, 0.49852238, 0.58988759, 0.53035235, 0.57124057, 0.54561548, 0.58162089, 0.48705896, 0.41132108, 0.59653803, 0.50096089, 0.51099718, 0.55140545, 0.55798434, 0.53856297, 0.64601571, 0.61537674, 0.61274748, 0.58409855, 0.48583318, 0.53703224, 0.49489112, 0.64557629, 0.42487364, 0.48796559, 0.47811467, 0.48299745, 0.57026384, 0.62516635, 0.50589203, 0.55423402, 0.52439251, 0.58289281, 0.40879611, 0.50295334, 0.44628286, 0.52906527, 0.50026268, 0.54580796, 0.67054523, 0.59118762, 0.5557291 , 0.37692782, 0.61776855, 0.65028869, 0.51684233, 0.44060372, 0.55010748, 0.46544193, 0.57606724, 0.62861932, 0.50409861, 0.49855097, 0.47795148, 0.63604432, 0.4462969 , 0.56485265, 0.47978807, 0.58513272, 0.55038105, 0.52389705, 0.46400566, 0.39643151, 0.40669508, 0.37131903, 0.54409735, 0.52997155, 0.55972177, 0.4513946 , 0.49035362, 0.53726292, 0.70896823, 0.43129426, 0.64302928, 0.53381332, 0.51589647, 0.61336828, 0.49011946, 0.32064252, 0.46011238, 0.48114897, 0.41438599, 0.6162567 , 0.59125858, 0.53468529, 0.62485751, 0.5643462 , 0.43432088, 0.55723546, 0.55677339, 0.51964121, 0.59753128, 0.44999261, 0.54992742, 0.51795846, 0.47951257, 0.57153128, 0.57908496, 0.4626688 , 0.45531525, 0.44313973, 0.54603293, 0.57576854, 0.46361253, 0.527298 , 0.5620756 , 0.46704174, 0.44964006, 0.44489873, 0.59324251, 0.38614873, 0.7052925 , 0.54426546, 0.57456808, 0.44467559, 0.53451817, 0.46977995, 0.62371312, 0.44698664, 0.60374147, 0.47923972, 0.59966241, 0.45159563, 0.59206654, 0.54386941, 0.42658765, 0.53008371, 0.55862966, 0.4920328 , 0.56761111, 0.52064894, 0.39681885, 0.44574245, 0.65423631, 0.49700968, 0.52949852, 0.69074303, 0.47083257, 0.56112144, 0.61171242, 0.57201577, 0.55023754, 0.50197115, 0.54440172, 0.41029551, 0.55578038, 0.59709186, 0.50556125, 0.60242079, 0.54916367, 0.51934554, 0.65992837, 0.37152973, 0.61838874, 0.55014278, 0.5717791 , 0.48307382, 0.53725693, 0.56510068, 0.56899223, 0.47274061, 0.38206639, 0.57567921, 0.58460404, 0.69554675, 0.44126231, 0.5453808 , 0.55469192, 0.51558473, 0.45490972, 0.32829398, 0.57063905, 0.61816149, 0.4851176 , 0.62280177, 0.55124556, 0.54876465, 0.62502545, 0.53016613, 0.55607373, 0.42591779, 0.47576693, 0.54436353, 0.61658262, 0.72634035, 0.41672421, 0.52000937, 0.55465022, 0.52901531, 0.55834794, 0.58534343, 0.51694802, 0.43374908, 0.66757089, 0.47552899, 0.50901742, 0.61552002, 0.58569846, 0.49247126, 0.62729266, 0.45454611, 0.57492484, 0.36653027, 0.57255697, 0.57824612, 0.53146156, 0.53596233, 0.53638362, 0.46116763, 0.62344651, 0.52368548, 0.43118748, 0.39730289, 0.58060133, 0.51892256, 0.50985834, 0.52621487, 0.45318262, 0.42636468, 0.55077814, 0.51755157, 0.50558189, 0.48513592, 0.60846418, 0.56548573, 0.63888894, 0.64851154, 0.54867037, 0.46621769, 0.51043987, 0.72938861, 0.56584276, 0.56401497, 0.45178682, 0.73033294, 0.46287789, 0.45124177, 0.50136769, 0.3687888 , 0.55620501, 0.56414095, 0.50050497, 0.51400812, 0.51195696, 0.49870403, 0.54124953, 0.51817863, 0.58274345, 0.36739039, 0.6223105 , 0.49839431, 0.48262867, 0.60799173, 0.48497077, 0.50533284, 0.52068533, 0.51819816, 0.53736352, 0.43353299, 0.49234493, 0.6137744 , 0.50568935, 0.40132373, 0.5764364 , 0.48163201, 0.41512117, 0.47245353, 0.49395644, 0.58230514, 0.58230577, 0.50858737, 0.54478675, 0.49901037, 0.55797797, 0.51010655, 0.53948516, 0.55935642, 0.39606487, 0.59820544, 0.47212327, 0.62012946, 0.66593186, 0.58616071, 0.54776001, 0.4611425 , 0.62522599, 0.3766581 , 0.51283615, 0.44134182, 0.5182332 , 0.60188039, 0.4490443 , 0.46919351, 0.47447443, 0.49600183, 0.57066395, 0.52439289, 0.56334611, 0.56060475, 0.39914239, 0.3675798 , 0.51564721, 0.49865563, 0.50048977, 0.51907243, 0.48807419, 0.4837173 , 0.54264044, 0.56838568, 0.54979317, 0.51717151, 0.47623781, 0.45639949, 0.49258588, 0.59268146, 0.58045385, 0.573669 , 0.63253715, 0.42660048, 0.51028837, 0.52477234, 0.60977474, 0.48255809, 0.50034024, 0.58582749, 0.56229419, 0.55128857, 0.67506051, 0.50764811, 0.59438154, 0.42687047, 0.45088934, 0.56769317, 0.69933473, 0.64133968, 0.58965277, 0.52414937, 0.47795837, 0.51327079, 0.47880271, 0.51766668, 0.56479121, 0.42850267, 0.70736401, 0.36848061, 0.56501167, 0.4173219 , 0.59616084, 0.63402178, 0.59863361, 0.40275257, 0.61610026, 0.55072114, 0.54363882, 0.55507125, 0.40734622, 0.54603062, 0.48360112, 0.37964225, 0.49283026, 0.59018895, 0.59009867, 0.4508418 , 0.56922331, 0.61952373, 0.48797082, 0.53357994, 0.463265 , 0.51038265, 0.59259957, 0.40175384, 0.54355008, 0.48951247, 0.51671369, 0.48761615, 0.45277739, 0.60093516, 0.42003811, 0.51235 , 0.59074681, 0.50383085, 0.30816788, 0.58677263, 0.52795737, 0.55613922, 0.53170224, 0.30069634, 0.57035186, 0.57304229, 0.50054292, 0.47112149, 0.5553012 , 0.45457155, 0.59640452, 0.50753294, 0.60960469, 0.54302708, 0.45963418, 0.64439113, 0.40809175, 0.50408166, 0.57626158, 0.54048658, 0.5797436 , 0.58846209, 0.48535104, 0.47475958, 0.62056536, 0.55518476, 0.5876704 , 0.60219568, 0.61776265, 0.65848052, 0.5299857 , 0.51399049, 0.56802533, 0.509565 , 0.48821165, 0.4238927 , 0.57421096, 0.51616527, 0.65465229, 0.46116782, 0.33355847, 0.58515526, 0.55622069, 0.63010995, 0.49775216, 0.54856983, 0.57677015, 0.52546157, 0.53642151, 0.57683734, 0.51020746, 0.45510774, 0.42469847, 0.42367087, 0.38633576, 0.56000824, 0.51460849, 0.57506362, 0.49294988, 0.64173013, 0.45438165, 0.40507967, 0.58082999, 0.4938575 , 0.53151652, 0.61474763, 0.35312902, 0.53849575, 0.48628075, 0.59266678, 0.52805816, 0.52097269, 0.53184868, 0.63211204, 0.66178091, 0.58451712, 0.49163581, 0.61183509, 0.57385018, 0.56755006, 0.58717042, 0.47939109, 0.46031912, 0.41769586, 0.5317388 , 0.57788374, 0.57080485, 0.49382667, 0.63281403, 0.50398231, 0.60290742, 0.43699714, 0.49636521, 0.5053502 , 0.37261477, 0.48081627, 0.50270047, 0.66441706, 0.42627842, 0.53421282, 0.57808942, 0.44486317, 0.58220927, 0.57082553, 0.52144876, 0.5852739 , 0.60581384, 0.47197106, 0.58540418])
In [26]: se_nonorth = np.array([0.04831872, 0.04832813, 0.0440032 , 0.04114123, 0.04031513, 0.04924838, 0.04834419, 0.04662809, 0.04292674, 0.04538775, 0.04924342, 0.04409527, 0.0417032 , 0.05194798, 0.05030047, 0.0367301 , 0.05075158, 0.05003675, 0.04019231, 0.0408118 , 0.03840654, 0.05299583, 0.04520576, 0.04644108, 0.04959359, 0.03872031, 0.04183655, 0.05109119, 0.05050073, 0.04835535, 0.04488573, 0.0381961 , 0.04422992, 0.04346715, 0.05144883, 0.04486199, 0.04513496, 0.04031142, 0.04932358, 0.04871112, 0.0440838 , 0.03665062, 0.04599354, 0.04412655, 0.04248635, 0.04130969, 0.04234649, 0.05137093, 0.04609058, 0.05068716, 0.0416494 , 0.03860875, 0.04452859, 0.04575008, 0.03959878, 0.04713601, 0.04416249, 0.04282636, 0.04742508, 0.04341061, 0.04468596, 0.04133775, 0.05035144, 0.0429771 , 0.04869872, 0.04490033, 0.04602814, 0.04482479, 0.04350321, 0.0507471 , 0.04018135, 0.04230126, 0.03655288, 0.0468958 , 0.04503343, 0.04182587, 0.04241048, 0.04675773, 0.04176107, 0.04655241, 0.049157 , 0.04956828, 0.04646219, 0.04328973, 0.04411666, 0.04837818, 0.04206927, 0.04889302, 0.05010237, 0.04739132, 0.04636621, 0.04739799, 0.04309182, 0.04921978, 0.04310753, 0.04744372, 0.04151646, 0.04221348, 0.04493263, 0.03779441, 0.04688649, 0.03919651, 0.04590451, 0.05016068, 0.03819017, 0.04131799, 0.04337898, 0.0392395 , 0.04348177, 0.04323946, 0.05393595, 0.04038024, 0.04830559, 0.04124088, 0.05180402, 0.05567998, 0.04244997, 0.03911898, 0.04337131, 0.04624824, 0.04949055, 0.04985597, 0.04326196, 0.04212764, 0.04842716, 0.04561389, 0.04082098, 0.04448624, 0.0448946 , 0.05293593, 0.04388315, 0.04321804, 0.04614838, 0.04602541, 0.04429507, 0.04405809, 0.05071437, 0.05146057, 0.04623581, 0.048504 , 0.04321356, 0.04723778, 0.03439781, 0.04850456, 0.04551087, 0.04798238, 0.03911667, 0.03747808, 0.04200668, 0.04340506, 0.04620754, 0.05201033, 0.05035776, 0.0456115 , 0.04366328, 0.04285165, 0.04355332, 0.04149404, 0.04189363, 0.04271938, 0.043017 , 0.04975933, 0.04724617, 0.0478123 , 0.04583418, 0.05493124, 0.04423306, 0.04228908, 0.04539415, 0.04335721, 0.03785391, 0.04805826, 0.04607728, 0.04239644, 0.04415323, 0.04616612, 0.04793052, 0.05040471, 0.04690316, 0.04168859, 0.04188572, 0.04151215, 0.04243537, 0.04459925, 0.05220799, 0.04072488, 0.03941964, 0.05134895, 0.0484997 , 0.04823143, 0.05272294, 0.0518343 , 0.04815165, 0.04984051, 0.0464933 , 0.03990746, 0.05754139, 0.04443048, 0.03948512, 0.04694665, 0.04627509, 0.04513908, 0.05028515, 0.03946741, 0.04535004, 0.04954032, 0.04290176, 0.04387563, 0.05135577, 0.0504392 , 0.04766801, 0.05580917, 0.04468912, 0.04505744, 0.04120763, 0.04365832, 0.05030036, 0.04299801, 0.05215436, 0.04222343, 0.04385739, 0.04116473, 0.04945468, 0.04284718, 0.04603566, 0.0459052 , 0.04613431, 0.03829818, 0.04267634, 0.04598015, 0.04513768, 0.04448734, 0.05130906, 0.04129426, 0.04086385, 0.04564178, 0.04209469, 0.04482055, 0.04366908, 0.04894291, 0.05089028, 0.0456697 , 0.04560245, 0.04475189, 0.04815378, 0.04657151, 0.04587175, 0.03997987, 0.04141135, 0.04238376, 0.04364734, 0.04414603, 0.05422814, 0.05288109, 0.04474988, 0.04839057, 0.03951434, 0.04106973, 0.04353691, 0.04514385, 0.04276905, 0.04152265, 0.04575897, 0.05536061, 0.04523277, 0.04283036, 0.04593579, 0.04794558, 0.04065449, 0.04222009, 0.04894703, 0.04746394, 0.04702815, 0.04692336, 0.03422089, 0.04594671, 0.05291489, 0.04307596, 0.04955153, 0.04553489, 0.04901159, 0.04473089, 0.04589466, 0.0441002 , 0.046049 , 0.04348205, 0.05187483, 0.04383983, 0.04346503, 0.04739405, 0.04447819, 0.04102896, 0.0431398 , 0.04656729, 0.03896816, 0.0542891 , 0.04270792, 0.04289141, 0.05348902, 0.0495009 , 0.04643517, 0.04410614, 0.04200152, 0.04493118, 0.04806057, 0.04190507, 0.04127871, 0.04847289, 0.04842111, 0.04833609, 0.04889407, 0.05076527, 0.04744793, 0.05150318, 0.04888322, 0.04992893, 0.04023113, 0.03881747, 0.0386972 , 0.04694449, 0.03855655, 0.0457303 , 0.04371627, 0.04143969, 0.03938361, 0.04876415, 0.04742728, 0.04700492, 0.04762283, 0.0488795 , 0.04395567, 0.04441107, 0.05068248, 0.04547187, 0.04361415, 0.05412799, 0.04036971, 0.04535149, 0.04536353, 0.03678323, 0.04403829, 0.04812895, 0.0457191 , 0.04258916, 0.04280755, 0.05153953, 0.04413582, 0.04770393, 0.05038132, 0.03757377, 0.04973624, 0.04261597, 0.0456351 , 0.04472504, 0.04771097, 0.04373888, 0.04370413, 0.04552507, 0.04293194, 0.04761842, 0.04287779, 0.04733812, 0.04423761, 0.0448224 , 0.04636959, 0.04296324, 0.0477725 , 0.04653299, 0.04201292, 0.05075975, 0.05017969, 0.04138296, 0.04060744, 0.04932747, 0.0400757 , 0.05220853, 0.03915695, 0.04293621, 0.06503597, 0.03595801, 0.03622248, 0.04329143, 0.0445115 , 0.03907058, 0.04102573, 0.04438729, 0.04948462, 0.04521646, 0.04211678, 0.04829002, 0.05505099, 0.03515793, 0.04173115, 0.04445393, 0.05032834, 0.05006664, 0.04629111, 0.04772484, 0.04092481, 0.04156632, 0.04229721, 0.04327583, 0.04579568, 0.04716264, 0.04178555, 0.04428462, 0.04725546, 0.04493506, 0.05485039, 0.0442419 , 0.03824128, 0.048013 , 0.04400902, 0.03878737, 0.04157463, 0.03909924, 0.04262371, 0.04611657, 0.03955994, 0.04055998, 0.04706018, 0.04562143, 0.04247859, 0.03936683, 0.04726424, 0.04247808, 0.04722227, 0.05177304, 0.05076287, 0.04027302, 0.038523 , 0.04700016, 0.04626175, 0.04963526, 0.04924924, 0.04556549, 0.03992306, 0.04264629, 0.04201707, 0.0421803 , 0.04335506, 0.04803246, 0.04784608, 0.04146418, 0.03996568, 0.05063166, 0.04452316, 0.04521715, 0.04285809, 0.04856424, 0.04510337, 0.04772573, 0.04363936, 0.04424811, 0.04980449, 0.04387433, 0.04367579, 0.04510261, 0.043214 , 0.0401914 , 0.05182074, 0.03951072, 0.0477948 , 0.04673033, 0.04689404, 0.04998034, 0.04552733, 0.04831458, 0.04267216, 0.04826833, 0.04793905, 0.05178392, 0.04410214, 0.03943804, 0.04291292, 0.0406942 , 0.04885506, 0.04308423, 0.04549851, 0.04976944, 0.05076628, 0.04412881, 0.04532336, 0.04897802, 0.0469969 , 0.04439076, 0.04942879, 0.04328135, 0.04629906, 0.0459189 , 0.03818804, 0.04568807, 0.04147959, 0.05170615, 0.05463537, 0.04573667, 0.04900106, 0.04359653, 0.03808469, 0.04496064, 0.03948091, 0.04690949, 0.04154617, 0.0474799 , 0.04454476, 0.04736569, 0.05066139, 0.04207634, 0.0479254 , 0.04132016, 0.04558345, 0.044382 , 0.04258322, 0.04245783, 0.04792109, 0.03685759, 0.04711378, 0.04406799, 0.04270924, 0.05258154, 0.04412806, 0.04270278, 0.04686045, 0.04326495, 0.04031116, 0.04062513, 0.04254267, 0.0404767 , 0.04825754, 0.04323409, 0.05539793, 0.04773133, 0.04162422, 0.04661492, 0.04853648, 0.03979454, 0.04288096, 0.04452497, 0.04612258, 0.04615601, 0.04294166, 0.05242546, 0.04947169, 0.05055515, 0.04491648, 0.04057551, 0.04777092, 0.04561228, 0.04684564, 0.04517448, 0.04342776, 0.04343005, 0.04704644, 0.04112438, 0.03617482, 0.04135897, 0.05115549, 0.04477676, 0.04699584, 0.05039405, 0.04516071, 0.04827367, 0.04621755, 0.04747895, 0.0479186 , 0.0460015 , 0.04309225, 0.04366407, 0.0392787 , 0.03586747, 0.04604356, 0.03884698, 0.05004952, 0.04139646, 0.04689492, 0.04140696, 0.04730347, 0.04878515, 0.04813615, 0.04342726, 0.05762307, 0.04489961, 0.04359375, 0.04612299, 0.04854138, 0.04527062, 0.04472304, 0.0395711 , 0.0405797 , 0.04505481, 0.04728124, 0.04570094, 0.04527361, 0.03862806, 0.04442718, 0.04443652, 0.04895133, 0.04588046, 0.04839031, 0.04469987, 0.05000096, 0.03906171, 0.051967 , 0.03978374, 0.04194069, 0.04661182, 0.03479072, 0.04340331, 0.04006326, 0.05954248, 0.04160275, 0.04321778, 0.04882777, 0.04787445, 0.04925952, 0.05323071, 0.04701163, 0.05337292, 0.04709804, 0.04334403, 0.04061232, 0.04478596, 0.04398654, 0.04274207, 0.05090697, 0.04455623, 0.04872436, 0.0421355 , 0.04898239, 0.04857274, 0.05415483, 0.04700655, 0.04014555, 0.0480144 , 0.03973658, 0.05139387, 0.04163909, 0.04686196, 0.03972785, 0.04952517, 0.04355752, 0.04586151, 0.04503243, 0.04574733, 0.04706773, 0.05015332, 0.04530172, 0.04176983, 0.04060019, 0.04704383, 0.04182002, 0.03868631, 0.04489151, 0.04019074, 0.04062664, 0.04333441, 0.04373353, 0.0441841 , 0.04185141, 0.04855307, 0.03711605, 0.04173668, 0.05377995, 0.04700167, 0.04350333, 0.04455046, 0.04259438, 0.04334646, 0.04146749, 0.04634779, 0.04688937, 0.04008222, 0.0391518 , 0.04666003, 0.04264922, 0.04064489, 0.04484932, 0.04550754, 0.05596455, 0.04654242, 0.04698534, 0.05086802, 0.04602974, 0.04597451, 0.04194012, 0.04508152, 0.0533805 , 0.04400257, 0.04563091, 0.04757315, 0.04725896, 0.04570736, 0.0478959 , 0.04467469, 0.04802436, 0.04367885, 0.04876105, 0.04211308, 0.04510159, 0.04371484, 0.04133507, 0.04158446, 0.04393359, 0.04355243, 0.04995303, 0.04931508, 0.0381846 , 0.04688839, 0.03784551, 0.04850098, 0.05309496, 0.04718066, 0.04085384, 0.04463436, 0.04367085, 0.04746603, 0.047315 , 0.03405395, 0.04403485, 0.0502298 , 0.04417355, 0.03596471, 0.04452519, 0.04578993, 0.04632251, 0.03958637, 0.04449887, 0.03931985, 0.04310929, 0.03709216, 0.04151165, 0.04229465, 0.04899224, 0.0489364 , 0.04226488, 0.03964909, 0.04748648, 0.04178246, 0.05107194, 0.04452271, 0.05032763, 0.04413235, 0.04676899, 0.04810656, 0.04573807, 0.04076975, 0.05570845, 0.0443732 , 0.04552938, 0.0496156 , 0.04152792, 0.04781586, 0.04225201, 0.0495326 , 0.0443289 , 0.05493486, 0.04521976, 0.0468836 , 0.04401311, 0.04912803, 0.04230038, 0.04714092, 0.03973136, 0.04395779, 0.04519541, 0.04936377, 0.04713223, 0.04564903, 0.04156458, 0.04433356, 0.0505021 , 0.04753149, 0.04174502, 0.056807 , 0.0496208 , 0.04767351, 0.04489522, 0.04642526, 0.05266117, 0.04438333, 0.04227131, 0.05097436, 0.04070389, 0.04836683, 0.04708687, 0.05225942, 0.03992956, 0.04902233, 0.04267317, 0.04477181, 0.03611383, 0.05027674, 0.03796292, 0.05075041, 0.04374405, 0.04413964, 0.0406099 , 0.04255497, 0.04242483, 0.04303759, 0.04170372, 0.04663662, 0.04296794, 0.05192534, 0.0449956 , 0.04730325, 0.0450723 , 0.04445592, 0.04766065, 0.0490991 , 0.04479711, 0.05232366, 0.04572643, 0.04847861, 0.04174633, 0.04394398, 0.0432659 , 0.04822833, 0.04773257, 0.04472821, 0.05111633, 0.04862911, 0.04401816, 0.04236195, 0.04526448, 0.05078418, 0.03735922, 0.04876512, 0.04758175, 0.04562471, 0.04335687, 0.04624443, 0.04618243, 0.04639856, 0.03822104, 0.0572194 , 0.04308923, 0.0419536 , 0.03811534, 0.03959254, 0.04130681, 0.05258445, 0.04986443, 0.04372495, 0.05031003, 0.05073988, 0.04547629, 0.050313 , 0.05054227, 0.04252951, 0.05065258, 0.04641125, 0.04230864, 0.04063602, 0.04855955, 0.04236878, 0.04650167, 0.03732677, 0.05326182, 0.04636859, 0.04556033, 0.04708447, 0.04356933, 0.04375771, 0.05121428, 0.04205749, 0.0518704 , 0.04518894, 0.04231454, 0.04398824, 0.05386235, 0.04887486, 0.04226957, 0.03547307, 0.04334838, 0.05197605, 0.04584531, 0.04269476, 0.04454851, 0.04730934, 0.05087434, 0.04269729, 0.04609613, 0.04301385, 0.04510682, 0.04449971, 0.04376506, 0.04954276, 0.04427552, 0.03896203, 0.04919564, 0.04771758, 0.04393024, 0.03870266, 0.0406708 , 0.0469429 , 0.04648953, 0.04738561, 0.04893255, 0.04408171, 0.04506601, 0.05193102, 0.04054061, 0.05030557, 0.04590494, 0.04596106, 0.04486667, 0.04068828, 0.0439199 , 0.0445398 , 0.05113532, 0.04578913, 0.0427432 , 0.04100801, 0.04285793, 0.04261937, 0.04108686, 0.04369689, 0.04362722, 0.03982833, 0.04838574, 0.05134018, 0.04221049, 0.04859673, 0.04196777, 0.05194669, 0.05153446, 0.04606078, 0.04232377, 0.0469268 , 0.04235938, 0.04001685, 0.04145275, 0.05063048, 0.04944555, 0.0413224 , 0.04250659, 0.04647227, 0.04430456, 0.04549796, 0.04603226, 0.04760732, 0.03799237, 0.0473626 , 0.04558186, 0.0486614 , 0.04415898, 0.04871813, 0.04439888, 0.044249 , 0.04096636, 0.04800334, 0.04382408, 0.04466881, 0.04820415, 0.04541524, 0.04297626, 0.04546258, 0.04837869, 0.04856685, 0.04739173, 0.04542921, 0.04363037, 0.04663879, 0.03848235, 0.04428369, 0.04391203, 0.04583622, 0.04338766, 0.04707669, 0.04744157, 0.03843356, 0.04494826, 0.05118331, 0.04920233, 0.04636591, 0.04606445, 0.04217975, 0.04145861, 0.04620102, 0.04249437, 0.03885329, 0.04489372, 0.04626471, 0.04990879, 0.04971651, 0.04811536, 0.0440326 , 0.04365409, 0.04810089, 0.04607049, 0.0445267 , 0.04082594, 0.04242013, 0.04945364, 0.04470348, 0.04007413, 0.04396535, 0.05119831, 0.04607697])
# to run the full simulation uncomment the following line to fit the model for every dataset and not just for the first dataset
#for i_rep in range(n_rep):
In [27]: for i_rep in range(1):
....: (x, y, d) = data[i_rep]
....: obj_dml_data = DoubleMLData.from_arrays(x, y, d)
....: obj_dml_plr_nonorth = DoubleMLPLR(obj_dml_data,
....: ml_l, ml_m, ml_g,
....: n_folds=2,
....: apply_cross_fitting=False,
....: score=non_orth_score)
....: obj_dml_plr_nonorth.fit()
....: this_theta = obj_dml_plr_nonorth.coef[0]
....: this_se = obj_dml_plr_nonorth.se[0]
....: print(np.abs(theta_nonorth[i_rep] - this_theta))
....: print(np.abs(se_nonorth[i_rep] - this_se))
....: theta_nonorth[i_rep] = this_theta
....: se_nonorth[i_rep] = this_se
....:
0.00044536980732112763
0.0010940735020170678
In [28]: plt.figure(constrained_layout=True);
In [29]: ax = sns.histplot((theta_nonorth - alpha)/se_nonorth,
....: color=face_colors[0], edgecolor = edge_colors[0],
....: stat='density', bins=30, label='Non-orthogonal ML');
....:
In [30]: ax.axvline(0., color='k');
In [31]: xx = np.arange(-5, +5, 0.001)
In [32]: yy = stats.norm.pdf(xx)
In [33]: ax.plot(xx, yy, color='k', label='$\\mathcal{N}(0, 1)$');
In [34]: ax.legend(loc='upper right', bbox_to_anchor=(1.2, 1.0));
In [35]: ax.set_xlim([-6., 6.]);
In [36]: ax.set_xlabel('$(\hat{\\theta}_0 - \\theta_0)/\hat{\sigma}$');
non_orth_score = function(y, d, l_hat, m_hat, g_hat, smpls) {
u_hat = y - g_hat
psi_a = -1*d*d
psi_b = d*u_hat
psis = list(psi_a = psi_a, psi_b = psi_b)
return(psis)
}
library(ggplot2)
library(mlr3)
library(mlr3learners)
library(data.table)
lgr::get_logger("mlr3")$set_threshold("warn")
options(repr.plot.width=5, repr.plot.height=4)
set.seed(1111)
ml_l = lrn("regr.ranger", num.trees = 132, max.depth = 5, mtry = 12, min.node.size = 1)
ml_m = lrn("regr.ranger", num.trees = 378, max.depth = 3, mtry = 20, min.node.size = 6)
ml_g = ml_l$clone()
# to speed up the illustration we hard-code the simulation results
theta_nonorth = c(0.555490805, 0.626761547, 0.527232714, 0.590215088, 0.379737837, 0.398791295, 0.471529799, 0.447306859, 0.398580212, 0.544042875, 0.580504564, 0.474363062, 0.544946070, 0.492569324, 0.476388613, 0.432095547, 0.463801034, 0.538834518, 0.498974469, 0.541736784, 0.556051749, 0.414388248, 0.465484169, 0.523789310, 0.450045410, 0.462671128, 0.483402627, 0.665449409, 0.444034892, 0.571431921, 0.514748934, 0.504854642, 0.543430321, 0.607804819, 0.425343408, 0.461166556, 0.473782998, 0.626301926, 0.554365067, 0.498898748, 0.539603020, 0.588260718, 0.392087266, 0.563189621, 0.622539334, 0.603381790, 0.537399218, 0.450688767, 0.506590096, 0.375848237, 0.518585287, 0.549772758, 0.621927569, 0.488298153, 0.590428253, 0.696371638, 0.627708633, 0.433970065, 0.425033876, 0.537542890, 0.480869086, 0.458919543, 0.653851249, 0.584819357, 0.587246769, 0.459961246, 0.518142266, 0.511701654, 0.620336868, 0.476887219, 0.481538784, 0.616313409, 0.529185572, 0.618412368, 0.522084966, 0.580750958, 0.477682206, 0.574414735, 0.508342507, 0.371303328, 0.527577838, 0.398635240, 0.494730015, 0.508128512, 0.603642166, 0.655167716, 0.601126516, 0.440808052, 0.544282497, 0.708230201, 0.541813291, 0.378489246, 0.437844992, 0.580074282, 0.426458304, 0.644148033, 0.571626341, 0.600921811, 0.710259470, 0.450440901, 0.540628619, 0.431058282, 0.442400358, 0.561393132, 0.523929900, 0.483505805, 0.519636552, 0.607386052, 0.535609002, 0.510519531, 0.490837474, 0.607186868, 0.475418778, 0.614141891, 0.398599358, 0.474117625, 0.532997363, 0.352785069, 0.599932209, 0.506270174, 0.582559300, 0.573352058, 0.688411509, 0.459070744, 0.633930860, 0.531340117, 0.554952484, 0.476316494, 0.586937919, 0.531143264, 0.531667832, 0.604187114, 0.474454410, 0.634159600, 0.515934881, 0.528853550, 0.561666177, 0.632040507, 0.573736861, 0.634677664, 0.545228666, 0.452539377, 0.602514389, 0.406644548, 0.526514048, 0.484831291, 0.515626324, 0.694950769, 0.558903366, 0.543989664, 0.595270602, 0.544302473, 0.701224740, 0.462491671, 0.721976217, 0.509779134, 0.608551613, 0.511985210, 0.610036501, 0.523291208, 0.571238422, 0.486944282, 0.457603820, 0.605811112, 0.602538755, 0.437573404, 0.572777807, 0.540359885, 0.503639372, 0.643482254, 0.542442515, 0.557072215, 0.548365345, 0.520257386, 0.559477646, 0.589800976, 0.620407048, 0.519814608, 0.515998436, 0.361384876, 0.548933980, 0.501600804, 0.627079396, 0.645484925, 0.495473132, 0.442635400, 0.584397481, 0.585125950, 0.352607330, 0.563273699, 0.503440598, 0.488867579, 0.557098343, 0.612293879, 0.463620892, 0.592306697, 0.546759827, 0.643941943, 0.354424800, 0.598255795, 0.484472850, 0.612693579, 0.410380974, 0.598567337, 0.512124207, 0.466188290, 0.416940820, 0.541944143, 0.586775221, 0.546650192, 0.536563665, 0.576996348, 0.503872606, 0.505680665, 0.555680519, 0.518350769, 0.652987312, 0.627428041, 0.552042874, 0.504864880, 0.611606827, 0.517154197, 0.544605688, 0.406896793, 0.597414935, 0.582586483, 0.590689714, 0.495941043, 0.498246261, 0.543623886, 0.555031880, 0.551186152, 0.478571512, 0.472812207, 0.518706192, 0.563640332, 0.554419638, 0.515959815, 0.458494588, 0.549400619, 0.490260796, 0.597798784, 0.510691719, 0.684161072, 0.506404568, 0.397770673, 0.514802604, 0.655332528, 0.472896035, 0.546293061,
0.450750417, 0.577340695, 0.547787327, 0.599718155, 0.544373382, 0.516858470, 0.511922800, 0.536369798, 0.499136617, 0.378726334, 0.617034292, 0.478520873, 0.521760931, 0.540347935, 0.655351297, 0.571225200, 0.512656762, 0.483168208, 0.605376870, 0.458754981, 0.422778271, 0.559776438, 0.700760982, 0.552098162, 0.582239482, 0.682263179, 0.534457440, 0.424484890, 0.517715608, 0.515762565, 0.498068539, 0.434358715, 0.671502664, 0.485220049, 0.415942514, 0.534758763, 0.619185459, 0.546414463, 0.568457400, 0.457187636, 0.650218256, 0.485625253, 0.407528701, 0.524748207, 0.454240783, 0.515931350, 0.480874561, 0.484779080, 0.550198966, 0.603725101, 0.576954166, 0.565338393, 0.606319473, 0.515972696, 0.618398708, 0.592751447, 0.589232898, 0.529610016, 0.552006633, 0.554400179, 0.696469507, 0.514473164, 0.576544908, 0.550114416, 0.526571101, 0.511727507, 0.643502178, 0.553529481, 0.631429147, 0.513860803, 0.712015201, 0.395955817, 0.537149164, 0.576820745, 0.472036962, 0.510995078, 0.559692897, 0.712613979, 0.548511940, 0.603141209, 0.646451322, 0.578597740, 0.650072597, 0.595784184, 0.444147267, 0.431644371, 0.470301501, 0.534208588, 0.570610331, 0.555056229, 0.507935162, 0.316119055, 0.557401212, 0.527316271, 0.605040348, 0.538434426, 0.576806205, 0.452004053, 0.544753597, 0.512988932, 0.619476411, 0.396076184, 0.566070831, 0.566438419, 0.450220982, 0.484173158, 0.620380771, 0.457030217, 0.436522912, 0.527267470, 0.495845368, 0.530519106, 0.592038170, 0.493137420, 0.590874952, 0.617868220, 0.486070270, 0.557348301, 0.536603623, 0.442690115, 0.498870579, 0.522430964, 0.586517685, 0.570256169, 0.589559448, 0.571328252, 0.492033061, 0.535250779, 0.512178730, 0.460879072, 0.500807069, 0.600888729, 0.517356646, 0.451303618, 0.587678099, 0.529998865, 0.471530533, 0.559646452, 0.593765971, 0.480317206, 0.466479902, 0.529183809, 0.510594353, 0.466759937, 0.437646506, 0.472243995, 0.482003863, 0.597815242, 0.425598907, 0.561641434, 0.508943548, 0.584697847, 0.636510983, 0.660346903, 0.454066017, 0.462487897, 0.505506043, 0.461258540, 0.535885979, 0.534607554, 0.470024303, 0.562676413, 0.586149433, 0.591356781, 0.455978848, 0.416752953, 0.418661640, 0.419590718, 0.571766706, 0.446425494, 0.588554376, 0.550779772, 0.550332231, 0.582390257, 0.462180960, 0.564796119, 0.607846426, 0.534457252, 0.522840491, 0.445373229, 0.549251612, 0.514152907, 0.501535072, 0.511931118, 0.614990004, 0.546573038, 0.498853768, 0.491775889, 0.637549986, 0.620760302, 0.540748899, 0.545831441, 0.489000728, 0.555212756, 0.516338580, 0.654872002, 0.482842368, 0.564765308, 0.503483424, 0.565809729, 0.623620196, 0.606398633, 0.545202760, 0.434323485, 0.592081701, 0.600379316, 0.552709762, 0.599656634, 0.472034047, 0.533803354, 0.531558379, 0.565841531, 0.350490085, 0.397840404, 0.548444844, 0.561229603, 0.563060424, 0.476912213, 0.564984359, 0.364141386, 0.502445480, 0.498539394, 0.513570785, 0.555188996, 0.448156175, 0.512119255, 0.445386379, 0.549415579, 0.341524411, 0.559606604, 0.593945702, 0.712481162, 0.587995572, 0.547201619, 0.466410676, 0.380301797, 0.562187203, 0.627409297, 0.558438269, 0.485853148, 0.587346549, 0.428806963, 0.655803314, 0.555888423, 0.542544980, 0.481775583, 0.480071956, 0.553643949, 0.552254460, 0.480791541,
0.560288765, 0.420843103, 0.361128494, 0.580809347, 0.527009418, 0.624330658, 0.532888624, 0.501396826, 0.618831171, 0.580389499, 0.629836529, 0.626352731, 0.571653376, 0.618028332, 0.357476413, 0.633596384, 0.518716872, 0.485010063, 0.574022133, 0.614519740, 0.602180299, 0.539065547, 0.553195187, 0.566421759, 0.478586567, 0.599254258, 0.543793383, 0.494030057, 0.310873684, 0.617009882, 0.404612458, 0.473790800, 0.534740643, 0.467572927, 0.498699660, 0.595133903, 0.455755429, 0.434058956, 0.594241142, 0.500503867, 0.572898720, 0.503342001, 0.532700074, 0.358150489, 0.412343967, 0.593475196, 0.374554699, 0.572783358, 0.442674990, 0.572843680, 0.575662283, 0.430268257, 0.467648882, 0.485725966, 0.579486503, 0.542010045, 0.400613627, 0.599057054, 0.563136422, 0.601306038, 0.500426996, 0.683184450, 0.470516049, 0.542721091, 0.563799526, 0.479949143, 0.636318482, 0.474724140, 0.482722868, 0.478795197, 0.543045187, 0.508977326, 0.547923904, 0.656525818, 0.604179155, 0.617628939, 0.469231387, 0.611045344, 0.667565827, 0.526248076, 0.606692621, 0.456038037, 0.482437027, 0.550598196, 0.454910374, 0.498118203, 0.555453296, 0.388227585, 0.533219722, 0.465068971, 0.557911420, 0.525542619, 0.443990102, 0.652073439, 0.528233460, 0.484280433, 0.541355253, 0.576684202, 0.523036842, 0.408319970, 0.617762803, 0.475120963, 0.662496201, 0.542095207, 0.571554663, 0.598805910, 0.638052186, 0.411580462, 0.557013065, 0.536381225, 0.511820061, 0.597580866, 0.518459700, 0.575230334, 0.696764491, 0.478841907, 0.551567186, 0.442349951, 0.522234760, 0.467536070, 0.513388134, 0.433268867, 0.383600813, 0.510964693, 0.536118893, 0.440663782, 0.579161992, 0.657013957, 0.501574228, 0.563880082, 0.570798093, 0.490814322, 0.582170037, 0.625847586, 0.549656969, 0.584665763, 0.529731144, 0.585040100, 0.507699586, 0.594906651, 0.542939508, 0.398198811, 0.371421696, 0.424559307, 0.521478929, 0.520841332, 0.557020071, 0.632433927, 0.540125247, 0.715177321, 0.635280103, 0.527653695, 0.462895585, 0.493750242, 0.499309380, 0.646233121, 0.535371808, 0.597260111, 0.484279193, 0.513683043, 0.457277699, 0.467642751, 0.499138169, 0.589887038, 0.534180778, 0.545297904, 0.517087020, 0.428642229, 0.464747534, 0.523041568, 0.518719639, 0.584106122, 0.533317756, 0.612415820, 0.461592490, 0.514983428, 0.518066482, 0.601919812, 0.530134121, 0.570187117, 0.528290332, 0.555818843, 0.453708364, 0.565527574, 0.607981783, 0.561120053, 0.669381588, 0.502013841, 0.564757923, 0.649643755, 0.570801738, 0.518847480, 0.468506826, 0.470142833, 0.499153449, 0.570689436, 0.479974005, 0.584116226, 0.488575346, 0.502941116, 0.473483029, 0.600104460, 0.444049959, 0.518093762, 0.616213737, 0.674839602, 0.535144087, 0.526260998, 0.463163914, 0.482726493, 0.460620798, 0.634715728, 0.676769679, 0.627133255, 0.507020514, 0.610770290, 0.532994049, 0.451198382, 0.542620243, 0.499428399, 0.429161177, 0.566230087, 0.678148969, 0.496447356, 0.558217776, 0.463545066, 0.487483001, 0.490059157, 0.431736489, 0.556338926, 0.455109444, 0.513704397, 0.573809102, 0.612380315, 0.503807651, 0.553517819, 0.576927711, 0.615872495, 0.531366945, 0.656660823, 0.517993343, 0.572913412, 0.569664360, 0.523645805, 0.490471201, 0.387093740, 0.564115440, 0.558273802, 0.598239938, 0.419955486,
0.465388602, 0.507796791, 0.506115556, 0.622283883, 0.596742526, 0.520855930, 0.644617222, 0.448209817, 0.564246989, 0.459447558, 0.715872399, 0.485503115, 0.569842986, 0.553621077, 0.451906806, 0.451427601, 0.540991056, 0.520360361, 0.450443033, 0.581866440, 0.560943492, 0.597019058, 0.494960963, 0.517031056, 0.435453039, 0.534349993, 0.631871756, 0.401181743, 0.597606115, 0.463709582, 0.584207932, 0.578141132, 0.464073722, 0.560488250, 0.542096705, 0.416232294, 0.590416817, 0.515450313, 0.545576702, 0.376863244, 0.480802909, 0.540922995, 0.653557885, 0.669806192, 0.534690873, 0.411004986, 0.578063132, 0.560783860, 0.571479896, 0.512483132, 0.523661488, 0.557963010, 0.445497341, 0.586293496, 0.440482669, 0.417545555, 0.548913084, 0.550121545, 0.368974569, 0.615313747, 0.531380568, 0.501810588, 0.400781748, 0.498819399, 0.543734006, 0.609199732, 0.576704893, 0.604352164, 0.627460052, 0.560995623, 0.603813764, 0.595195256, 0.635549955, 0.500937607, 0.564012122, 0.515636906, 0.635622134, 0.537348045, 0.512691695, 0.511057536, 0.615937193, 0.510478444, 0.518828953, 0.571380267, 0.580241326, 0.537779208, 0.593899836, 0.517439804, 0.589540384, 0.566012500, 0.548740539, 0.502564261, 0.511657997, 0.409964475, 0.589915535, 0.652057794, 0.634579542, 0.496301701, 0.535769343, 0.676293472, 0.607051859, 0.659077347, 0.602118707, 0.550680479, 0.563329950, 0.511840194, 0.577850813, 0.471617247, 0.414955147, 0.563989678, 0.463254931, 0.479894191, 0.623420693, 0.593593868, 0.458635132, 0.613803756, 0.488046496, 0.542319978, 0.350973957, 0.412569564, 0.490260699, 0.700491440, 0.428683103, 0.610299924, 0.524726774, 0.317358994, 0.524104380, 0.490904473, 0.582942177, 0.511242949, 0.589722146, 0.461090940, 0.563778935, 0.509113728, 0.606855462, 0.536871498, 0.535835864, 0.311305684, 0.519460682, 0.617250309, 0.459249686, 0.613208421, 0.689944309, 0.600880392, 0.470012487, 0.473908584, 0.618506078, 0.463854332, 0.424564259, 0.609790770, 0.619358325, 0.566629982, 0.558465168, 0.367649118, 0.590299657, 0.567348639, 0.672643300, 0.537803918, 0.333977141, 0.584732617, 0.648302831, 0.454857335, 0.623665038, 0.462490043, 0.634790342, 0.477715737, 0.547378369, 0.543000665, 0.565847084, 0.661123797, 0.553120335, 0.485868839, 0.502218315, 0.477975844, 0.497536727, 0.627151462, 0.532951470, 0.589626479, 0.543310420, 0.578503587, 0.547612194, 0.670302172, 0.515456325, 0.538013972, 0.454596159, 0.552985860, 0.702189700, 0.574203991, 0.469594424, 0.581663990, 0.440378605, 0.445220423, 0.565606288, 0.469661635, 0.578705416, 0.450488617, 0.667395060, 0.484036907, 0.644102750, 0.519522226, 0.402077245, 0.463575413, 0.552887803, 0.512808224, 0.447670073, 0.591463960, 0.512150833, 0.559739620, 0.578457682, 0.497693480, 0.602815427, 0.541223208, 0.500136941, 0.574870749, 0.522447052, 0.659254938, 0.518345275, 0.581820859, 0.537628623, 0.469688518, 0.490970116, 0.539631057, 0.457655973, 0.570659774, 0.452771742, 0.598276557, 0.473113951, 0.645215405, 0.499805678, 0.538075991, 0.555687688, 0.651869368, 0.499626409, 0.406144900, 0.427681771, 0.457855260, 0.404348940, 0.567678710, 0.598015105, 0.371846343, 0.660219655, 0.496971782, 0.572141968, 0.502629015, 0.463091719, 0.535049004, 0.681186139, 0.492846017, 0.386504347, 0.656911448)
se_nonorth = c(0.0416942713, 0.0407183651, 0.0444523186, 0.0425346420, 0.0444558383, 0.0399894960, 0.0418382112, 0.0512202142, 0.0422534312, 0.0437289855, 0.0496170886, 0.0416615042, 0.0468264920, 0.0461997063, 0.0444825046, 0.0432499219, 0.0437510148, 0.0490838025, 0.0469682703, 0.0446521440, 0.0417557029, 0.0482750833, 0.0450491739, 0.0429404077, 0.0494601912, 0.0484775332, 0.0409040531, 0.0477823535, 0.0448376149, 0.0537685039, 0.0451218632, 0.0421525198, 0.0405617359, 0.0476656050, 0.0443286501, 0.0472993077, 0.0388136573, 0.0470112065, 0.0484876333, 0.0407014787, 0.0415448868, 0.0428025247, 0.0438924214, 0.0478182170, 0.0410469492, 0.0510127368, 0.0474686464, 0.0443057077, 0.0468339705, 0.0473232919, 0.0462538776, 0.0439858438, 0.0465037186, 0.0464214005, 0.0443683801, 0.0427980771, 0.0484656933, 0.0522435665, 0.0460452073, 0.0454639544, 0.0515455624, 0.0412344156, 0.0408156437, 0.0434971046, 0.0376215666, 0.0414815328, 0.0387734453, 0.0466417572, 0.0470888219, 0.0494321834, 0.0404736386, 0.0475966279, 0.0487751816, 0.0486634726, 0.0437035312, 0.0410891602, 0.0452614467, 0.0464142602, 0.0501091573, 0.0412405139, 0.0427498890, 0.0417762646, 0.0423550412, 0.0460234105, 0.0398641824, 0.0421414452, 0.0478292271, 0.0489479360, 0.0461427499, 0.0460249422, 0.0455823647, 0.0452108363, 0.0457211134, 0.0483323986, 0.0527181857, 0.0469950113, 0.0525710347, 0.0396720464, 0.0437818655, 0.0440221527, 0.0419406668, 0.0389232488, 0.0427301013, 0.0390849993, 0.0456962791, 0.0420672659, 0.0449914650, 0.0466148688, 0.0454478670, 0.0466533404, 0.0486936215, 0.0364006041, 0.0426142879, 0.0454636835, 0.0465283379, 0.0459340507, 0.0473761325, 0.0476661830, 0.0459159887, 0.0418073445, 0.0440123549, 0.0442796534, 0.0676434323, 0.0473605677, 0.0354092828, 0.0479795031, 0.0457478011, 0.0432144400, 0.0468389475, 0.0399661007, 0.0427871569, 0.0432939489, 0.0492717801, 0.0428361964, 0.0441044028, 0.0487274650, 0.0410106469, 0.0451210397, 0.0488886283, 0.0475810864, 0.0454722031, 0.0451376826, 0.0454999190, 0.0403121211, 0.0443778939, 0.0426164410, 0.0427421954, 0.0425622191, 0.0488372313, 0.0425927956, 0.0465584086, 0.0467233237, 0.0507913546, 0.0480643340, 0.0535318482, 0.0432534969, 0.0440594081, 0.0462619955, 0.0455910743, 0.0414613637, 0.0476070789, 0.0455846838, 0.0483576070, 0.0444985152, 0.0477921977, 0.0456495523, 0.0508897497, 0.0437550082, 0.0415551073, 0.0477370929, 0.0440603149, 0.0481728048, 0.0475273242, 0.0424167929, 0.0532847025, 0.0512992716, 0.0481748552, 0.0451459126, 0.0510471717, 0.0472198535, 0.0418179134, 0.0443819534, 0.0398429609, 0.0389292671, 0.0432117377, 0.0468339776, 0.0439764387, 0.0457969584, 0.0493033277, 0.0421968064, 0.0520255201, 0.0522945549, 0.0477131749, 0.0440483435, 0.0490376561, 0.0516126310, 0.0442248163, 0.0412546455, 0.0367354793, 0.0438534662, 0.0475180763, 0.0429398205, 0.0463706405, 0.0390414772, 0.0458774038, 0.0474482566, 0.0434896629, 0.0422358214, 0.0497988767, 0.0434132514, 0.0455985023, 0.0486931260, 0.0474450877, 0.0403084870, 0.0458375287, 0.0432554677, 0.0416824887, 0.0505585269, 0.0482389139, 0.0416516380, 0.0448090050, 0.0435162678, 0.0373855753, 0.0404456033, 0.0460920936, 0.0505343302, 0.0441023636, 0.0442842299, 0.0428609066, 0.0400794578, 0.0448480934, 0.0469449477, 0.0417936292, 0.0493436580, 0.0431128129, 0.0454712341, 0.0430130687, 0.0470833089, 0.0422605240, 0.0424348846, 0.0461216695, 0.0489060014, 0.0423069923, 0.0478447765, 0.0404220993, 0.0586981641, 0.0443232220, 0.0388575388, 0.0413215017, 0.0426211671,
0.0471838924, 0.0388989423, 0.0455790426, 0.0438913234, 0.0383028275, 0.0468147265, 0.0431720131, 0.0482008522, 0.0543839804, 0.0422250358, 0.0464682066, 0.0498224018, 0.0432196416, 0.0432677344, 0.0468795800, 0.0438977953, 0.0461659516, 0.0497379706, 0.0476103774, 0.0442774275, 0.0500619746, 0.0458558863, 0.0397456154, 0.0412716802, 0.0497800508, 0.0440394081, 0.0396038858, 0.0564439287, 0.0555597922, 0.0420128405, 0.0472330029, 0.0496680684, 0.0395024756, 0.0432825606, 0.0469085318, 0.0491087369, 0.0442754952, 0.0409932732, 0.0403405130, 0.0426314318, 0.0389934958, 0.0422365158, 0.0522774036, 0.0475572037, 0.0417722370, 0.0444766319, 0.0463969372, 0.0513068203, 0.0493066368, 0.0416801613, 0.0421615069, 0.0436823115, 0.0430911503, 0.0426003370, 0.0411770611, 0.0473561144, 0.0547397784, 0.0464928763, 0.0455073663, 0.0401664048, 0.0436246303, 0.0461393917, 0.0424488331, 0.0439597172, 0.0509171080, 0.0482319344, 0.0483124905, 0.0364286638, 0.0448411210, 0.0447490233, 0.0555590871, 0.0428486167, 0.0417124416, 0.0474719537, 0.0458114554, 0.0479360604, 0.0363660927, 0.0518490611, 0.0485668638, 0.0497163548, 0.0453206328, 0.0455803082, 0.0479854657, 0.0459699134, 0.0468712259, 0.0449448234, 0.0511424957, 0.0471113059, 0.0433347419, 0.0391208352, 0.0408913894, 0.0470659488, 0.0399841820, 0.0406308572, 0.0542143584, 0.0450145498, 0.0467033471, 0.0516124686, 0.0456065530, 0.0435843876, 0.0499941746, 0.0506719959, 0.0508443416, 0.0495156494, 0.0395405138, 0.0410419238, 0.0514470700, 0.0488962113, 0.0487538055, 0.0414726009, 0.0368267934, 0.0473123550, 0.0485298852, 0.0456861398, 0.0437143319, 0.0428824935, 0.0384520731, 0.0445882610, 0.0463727393, 0.0436154414, 0.0455220525, 0.0415109848, 0.0433445991, 0.0405737652, 0.0456195515, 0.0469804624, 0.0402089500, 0.0369510343, 0.0445247896, 0.0441592102, 0.0468461274, 0.0490547903, 0.0432990337, 0.0465668271, 0.0497264371, 0.0441192165, 0.0431313364, 0.0482525086, 0.0439745154, 0.0436839465, 0.0470566984, 0.0488376254, 0.0461766734, 0.0462701814, 0.0533789686, 0.0385214032, 0.0470878503, 0.0458131715, 0.0441274138, 0.0623588736, 0.0558354639, 0.0507552647, 0.0361639478, 0.0466204676, 0.0388749604, 0.0492020032, 0.0404862240, 0.0458497881, 0.0466700119, 0.0420892063, 0.0440183150, 0.0506405688, 0.0491806559, 0.0442083848, 0.0441417694, 0.0456755901, 0.0551192413, 0.0409182169, 0.0436943445, 0.0424569610, 0.0418949597, 0.0450919899, 0.0440379844, 0.0394901321, 0.0372667880, 0.0449608934, 0.0467084684, 0.0459461881, 0.0505850160, 0.0392913702, 0.0454423262, 0.0455367197, 0.0511507111, 0.0445989706, 0.0432381412, 0.0454507054, 0.0474978923, 0.0455950894, 0.0460462368, 0.0444929558, 0.0431117647, 0.0424168741, 0.0470416143, 0.0419458258, 0.0428595772, 0.0462222860, 0.0403097898, 0.0417097778, 0.0404122960, 0.0500143386, 0.0462261921, 0.0427382211, 0.0523602976, 0.0466083519, 0.0429828128, 0.0419430749, 0.0416151695, 0.0520834213, 0.0385143472, 0.0491620897, 0.0441673021, 0.0468161058, 0.0521244105, 0.0432531652, 0.0431773782, 0.0482408085, 0.0460797071, 0.0475796039, 0.0441044884, 0.0390626583, 0.0480827456, 0.0432076181, 0.0411163678, 0.0398631080, 0.0451781089, 0.0506179206, 0.0483444849, 0.0493423400, 0.0449097787, 0.0451427591, 0.0416698630, 0.0523988996, 0.0414144081, 0.0410177862, 0.0421860966, 0.0457489231, 0.0472401395, 0.0461553633, 0.0506701534, 0.0447794311, 0.0479170684, 0.0432160823, 0.0406424202, 0.0432600513, 0.0489419005, 0.0473253697, 0.0416913096, 0.0493886111, 0.0440630101, 0.0401957469,
0.0438400485, 0.0469706961, 0.0379419954, 0.0498916208, 0.0421411392, 0.0404144925, 0.0467110555, 0.0439446736, 0.0463083553, 0.0497131722, 0.0484687136, 0.0505424229, 0.0479257700, 0.0474604174, 0.0435021261, 0.0475139088, 0.0386510772, 0.0407415152, 0.0415758675, 0.0457929174, 0.0445308164, 0.0399679240, 0.0479620843, 0.0436231597, 0.0442219275, 0.0422322599, 0.0494076074, 0.0475472135, 0.0438761612, 0.0422145844, 0.0395191281, 0.0420427008, 0.0508861560, 0.0483396730, 0.0398827945, 0.0501383066, 0.0540309624, 0.0409913626, 0.0412386320, 0.0435887472, 0.0443302167, 0.0443502175, 0.0481808945, 0.0473181803, 0.0488766269, 0.0529367635, 0.0428916178, 0.0447638538, 0.0458472612, 0.0404790119, 0.0399257227, 0.0377888836, 0.0386492614, 0.0446937626, 0.0502840660, 0.0437966863, 0.0465801076, 0.0496721460, 0.0489452659, 0.0464502063, 0.0437991473, 0.0494891160, 0.0503485209, 0.0473798105, 0.0458485017, 0.0413926382, 0.0447822677, 0.0441246584, 0.0484195496, 0.0430158347, 0.0454672105, 0.0437217941, 0.0416578277, 0.0456674215, 0.0430853669, 0.0489013653, 0.0472174986, 0.0446444115, 0.0419116187, 0.0449237332, 0.0436448839, 0.0430536200, 0.0438416684, 0.0496886557, 0.0483889357, 0.0436115925, 0.0473577009, 0.0513557121, 0.0492885243, 0.0487500719, 0.0436059358, 0.0492746868, 0.0388168102, 0.0508673325, 0.0451668028, 0.0412011276, 0.0548782211, 0.0479393631, 0.0442228499, 0.0433162070, 0.0443213114, 0.0443725770, 0.0493525004, 0.0405233001, 0.0442736712, 0.0408752068, 0.0417812872, 0.0520464905, 0.0436899010, 0.0415405681, 0.0490416480, 0.0499029777, 0.0445336837, 0.0416151062, 0.0408112066, 0.0428046438, 0.0450605378, 0.0532768714, 0.0396644557, 0.0444696943, 0.0504156986, 0.0443956988, 0.0528172902, 0.0390398316, 0.0466802712, 0.0462500025, 0.0473485596, 0.0460854911, 0.0412017954, 0.0440077856, 0.0415717449, 0.0487651621, 0.0480823967, 0.0455474796, 0.0474422098, 0.0372996608, 0.0539488638, 0.0472896522, 0.0486949128, 0.0432878967, 0.0467050944, 0.0468515101, 0.0469311873, 0.0494697275, 0.0417226496, 0.0448879556, 0.0449673198, 0.0430951188, 0.0622614825, 0.0425694862, 0.0443975843, 0.0455196655, 0.0465424642, 0.0410751939, 0.0469711908, 0.0450505494, 0.0414393933, 0.0496273343, 0.0429266249, 0.0415579401, 0.0506347047, 0.0441589079, 0.0430085648, 0.0391062482, 0.0493892022, 0.0475849965, 0.0430335663, 0.0388700253, 0.0486225382, 0.0386786601, 0.0521838145, 0.0424971130, 0.0476506097, 0.0444575042, 0.0421267772, 0.0406615732, 0.0501312064, 0.0452361956, 0.0406248912, 0.0408469612, 0.0392444290, 0.0455309395, 0.0476790281, 0.0464622391, 0.0425475893, 0.0445142131, 0.0446443886, 0.0436690511, 0.0467629886, 0.0390306459, 0.0447922159, 0.0420200127, 0.0460367286, 0.0495337585, 0.0443241229, 0.0410023677, 0.0413692134, 0.0423467020, 0.0410082263, 0.0451696943, 0.0493794927, 0.0403973911, 0.0488905545, 0.0472538337, 0.0425203525, 0.0413610982, 0.0365841273, 0.0438075830, 0.0454242611, 0.0414027566, 0.0490544717, 0.0422271786, 0.0460426506, 0.0418820975, 0.0479000468, 0.0407397797, 0.0396700663, 0.0416426682, 0.0439507791, 0.0473532784, 0.0467900357, 0.0436291674, 0.0480063679, 0.0429754876, 0.0449870427, 0.0434355341, 0.0431861184, 0.0464974683, 0.0419407774, 0.0491435513, 0.0470466006, 0.0509939175, 0.0453228328, 0.0494626877, 0.0492348281, 0.0455664378, 0.0436146932, 0.0504675435, 0.0466653769, 0.0418666473, 0.0503067258, 0.0383718571, 0.0522584215, 0.0431252691, 0.0400283783, 0.0450449964, 0.0475258218, 0.0400685390, 0.0357934857, 0.0486042528,
0.0370261877, 0.0393641427, 0.0427189381, 0.0499489653, 0.0400037362, 0.0488202329, 0.0429777500, 0.0448634724, 0.0406805124, 0.0446702644, 0.0481321945, 0.0424083009, 0.0472734733, 0.0475518183, 0.0474519988, 0.0507773212, 0.0448938420, 0.0439749001, 0.0519301415, 0.0462917291, 0.0467769251, 0.0431394745, 0.0511398877, 0.0468973616, 0.0457477577, 0.0453645425, 0.0473912954, 0.0503678343, 0.0431073378, 0.0379354863, 0.0448704311, 0.0390333493, 0.0446870021, 0.0356292568, 0.0519402829, 0.0479014381, 0.0474969074, 0.0485018059, 0.0471529881, 0.0457874882, 0.0514035445, 0.0464573276, 0.0436958739, 0.0471457232, 0.0432730238, 0.0420443922, 0.0425626359, 0.0400757175, 0.0428709057, 0.0523370930, 0.0457095456, 0.0387461239, 0.0454164368, 0.0474067583, 0.0422056453, 0.0423306811, 0.0458742009, 0.0418851150, 0.0375060607, 0.0476444692, 0.0473158918, 0.0437627534, 0.0489777285, 0.0459813410, 0.0485886882, 0.0473717508, 0.0406818520, 0.0462999317, 0.0363856680, 0.0450027066, 0.0492219529, 0.0390316111, 0.0461459314, 0.0509053393, 0.0460205187, 0.0428744223, 0.0421943189, 0.0403611049, 0.0441886496, 0.0496008183, 0.0422805921, 0.0532239884, 0.0416174164, 0.0459029114, 0.0398305637, 0.0467203692, 0.0451460843, 0.0544602119, 0.0420129382, 0.0461996776, 0.0444963448, 0.0491157891, 0.0470482264, 0.0367606777, 0.0444011869, 0.0476917130, 0.0387451532, 0.0484612156, 0.0520540431, 0.0426581207, 0.0530953686, 0.0488003880, 0.0452046344, 0.0456663570, 0.0504463665, 0.0395335748, 0.0418899004, 0.0404035187, 0.0448713245, 0.0476916901, 0.0461400539, 0.0403482903, 0.0420793960, 0.0469157779, 0.0426540095, 0.0411240546, 0.0475006544, 0.0442164510, 0.0427828038, 0.0493846273, 0.0438576577, 0.0406313613, 0.0465298653, 0.0445994443, 0.0463059937, 0.0438992617, 0.0468660278, 0.0411646643, 0.0516189392, 0.0417182243, 0.0447124494, 0.0460886082, 0.0503461423, 0.0495413210, 0.0418845428, 0.0470049760, 0.0398121373, 0.0485289895, 0.0503722920, 0.0435886254, 0.0489487204, 0.0494519494, 0.0438545065, 0.0397148739, 0.0520638856, 0.0406954281, 0.0474046990, 0.0460098363, 0.0407100521, 0.0434347753, 0.0377814424, 0.0450191260, 0.0532696689, 0.0428635977, 0.0462511433, 0.0451252219, 0.0595109881, 0.0415188135, 0.0462933305, 0.0399142707, 0.0457308366, 0.0458449235, 0.0466075330, 0.0459267034, 0.0452804927, 0.0450031724, 0.0405805835, 0.0481327093, 0.0446886594, 0.0407773776, 0.0428498566, 0.0418913126, 0.0416114384, 0.0482691372, 0.0468687012, 0.0475622762, 0.0451409043, 0.0440544641, 0.0466109319, 0.0444994155, 0.0395735845, 0.0489770988, 0.0513474350, 0.0476471824, 0.0474166294, 0.0422779747, 0.0483946630, 0.0460189236, 0.0433891859, 0.0480850303, 0.0505456668, 0.0461110607, 0.0392525139, 0.0440482730, 0.0510736230, 0.0430907766, 0.0513039372, 0.0437664660, 0.0478224785, 0.0387364981, 0.0391934132, 0.0443478705, 0.0468773935, 0.0450421815, 0.0494911337, 0.0413486799, 0.0422647277, 0.0385349835, 0.0561999522, 0.0380104667, 0.0524001385, 0.0497509208, 0.0494165088, 0.0506288203, 0.0499250336, 0.0483102620, 0.0444882253, 0.0432652527, 0.0489046969, 0.0570060453, 0.0359903313, 0.0503936218, 0.0444652977, 0.0449598097, 0.0525180496, 0.0383749194, 0.0522110450, 0.0430210793, 0.0405819589, 0.0393943644, 0.0488834320, 0.0437871919, 0.0419881386, 0.0502366835, 0.0483239506, 0.0454876973, 0.0393104070, 0.0387805425, 0.0507050119, 0.0417782241, 0.0455323571, 0.0439461195, 0.0398420253, 0.0418836739, 0.0478967241, 0.0437425739, 0.0448702453, 0.0396011810, 0.0437846518, 0.0474342375)
# to run the full simulation uncomment the following line to fit the model for every dataset and not just for the first dataset
#for (i_rep in seq_len(n_rep)) {
for (i_rep in seq_len(1)) {
df = data[[i_rep]]
obj_dml_data = double_ml_data_from_data_frame(df, y_col = "y", d_cols = "d")
obj_dml_plr_nonorth = DoubleMLPLR$new(obj_dml_data,
ml_l, ml_m, ml_g,
n_folds=2,
score=non_orth_score,
apply_cross_fitting=FALSE)
obj_dml_plr_nonorth$fit()
this_theta = obj_dml_plr_nonorth$coef
this_se = obj_dml_plr_nonorth$se
print(abs(theta_nonorth[i_rep] - this_theta))
print(abs(se_nonorth[i_rep] - this_se))
theta_nonorth[i_rep] = this_theta
se_nonorth[i_rep] = this_se
}
g_nonorth = ggplot(data.frame(theta_rescaled=(theta_nonorth - alpha)/se_nonorth)) +
geom_histogram(aes(y=after_stat(density), x=theta_rescaled, colour = "Non-orthogonal ML", fill="Non-orthogonal ML"),
bins = 30, alpha = 0.3) +
geom_vline(aes(xintercept = 0), col = "black") +
suppressWarnings(geom_function(fun = dnorm, aes(colour = "N(0, 1)", fill="N(0, 1)"))) +
scale_color_manual(name='',
breaks=c("Non-orthogonal ML", "N(0, 1)"),
values=c("Non-orthogonal ML"="dark blue", "N(0, 1)"='black')) +
scale_fill_manual(name='',
breaks=c("Non-orthogonal ML", "N(0, 1)"),
values=c("Non-orthogonal ML"="dark blue", "N(0, 1)"=NA)) +
xlim(c(-6.0, 6.0)) + xlab("") + ylab("") + theme_minimal()
g_nonorth
d
3.33028e-11
d
5.110218e-12
The regularization bias in the simple ML-approach is caused by the slow convergence of \(\hat{\theta}_0\)
\[|\sqrt{n} (\hat{\theta}_0 - \theta_0) | \rightarrow_{P} \infty\]
i.e., slower than \(1/\sqrt{n}\). The driving factor is the bias that arises by learning \(g\) with a random forest or any other ML technique. A heuristic illustration is given by
\[\sqrt{n}(\hat{\theta}_0 - \theta_0) = \underbrace{\left(\frac{1}{n} \sum_{i\in I} D_i^2\right)^{-1} \frac{1}{n} \sum_{i\in I} D_i \zeta_i}_{=:a}
+ \underbrace{\left(\frac{1}{n} \sum_{i\in I} D_i^2\right)^{-1} \frac{1}{n} \sum_{i\in I} D_i (g_0(X_i) - \hat{g}_0(X_i))}_{=:b}.\]
\(a\) is approximately Gaussian under mild conditions. However, \(b\) (the regularization bias) diverges in general.
1.3. Overcoming regularization bias by orthogonalization#
To overcome the regularization bias we can partial out the effect of \(X\) from \(D\) to obtain the orthogonalized regressor \(V = D - m(X)\). We then use the final estimate
\[\check{\theta}_0 = \left(\frac{1}{n} \sum_{i\in I} \hat{V}_i D_i\right)^{-1} \frac{1}{n} \sum_{i\in I} \hat{V}_i (Y_i - \hat{g}_0(X_i)).\]
In [37]: import numpy as np
In [38]: np.random.seed(2222)
# to speed up the illustration we hard-code the simulation results
In [39]: theta_orth_nosplit = np.array([0.5269861 , 0.48682698, 0.45808479, 0.46591054, 0.47453753, 0.474066 , 0.50998524, 0.45472461, 0.51296185, 0.57402367, 0.49932393, 0.51328342, 0.53728227, 0.38413749, 0.46069765, 0.47095928, 0.49359267, 0.44035397, 0.50447968, 0.47268968, 0.46074218, 0.41151753, 0.434847 , 0.4873961 , 0.46688001, 0.43024809, 0.49051175, 0.45356252, 0.45427558, 0.50022074, 0.48977533, 0.56098269, 0.38961805, 0.41125901, 0.4748894 , 0.42412004, 0.44705749, 0.5231581 , 0.49033896, 0.48503933, 0.43015338, 0.44154682, 0.4902393 , 0.38245209, 0.51450476, 0.47647825, 0.50007773, 0.47903055, 0.49418268, 0.56080375, 0.45953682, 0.46652166, 0.51456846, 0.50829186, 0.54538745, 0.41911111, 0.50936297, 0.53669094, 0.42254389, 0.4544877 , 0.50019279, 0.52443831, 0.52700521, 0.48787306, 0.53024376, 0.44000176, 0.43955207, 0.4733464 , 0.55669921, 0.52731979, 0.43930057, 0.45837929, 0.45972765, 0.4862641 , 0.49900815, 0.51834668, 0.50323945, 0.50786989, 0.47200566, 0.50307507, 0.46898 , 0.47696915, 0.46770453, 0.46545997, 0.4250659 , 0.494894 , 0.52010544, 0.49542341, 0.51601883, 0.49493945, 0.40182449, 0.43205712, 0.53710759, 0.4757337 , 0.5136859 , 0.47012369, 0.47963284, 0.48218851, 0.58030577, 0.51763202, 0.45985742, 0.54198937, 0.4611221 , 0.5098156 , 0.43023138, 0.56368806, 0.48300061, 0.48374618, 0.40530258, 0.47329255, 0.41705756, 0.4606845 , 0.43832176, 0.44102828, 0.44333831, 0.46029671, 0.46405733, 0.47607037, 0.4542386 , 0.44354708, 0.45175568, 0.55117658, 0.47413396, 0.46172446, 0.38677175, 0.54103262, 0.44239398, 0.46169587, 0.45518892, 0.41257724, 0.52436548, 0.43443768, 0.47283858, 0.46127727, 0.44831562, 0.54196493, 0.52168875, 0.5235551 , 0.42356335, 0.47162521, 0.4545478 , 0.44284108, 0.5520522 , 0.49528219, 0.54823811, 0.40642283, 0.39873622, 0.53167736, 0.43939755, 0.5220321 , 0.53124308, 0.49231836, 0.44953381, 0.4603219 , 0.47413766, 0.60070383, 0.45073656, 0.4587013 , 0.42961732, 0.58569857, 0.4786611 , 0.45579044, 0.50914344, 0.5383562 , 0.47894433, 0.45978526, 0.53156866, 0.47936777, 0.4969537 , 0.49535029, 0.50277886, 0.45905374, 0.46699655, 0.44532146, 0.48658664, 0.48709778, 0.49174929, 0.55692821, 0.47684158, 0.46423733, 0.43865106, 0.50698636, 0.48973865, 0.42167159, 0.3957361 , 0.44379821, 0.39898509, 0.45963827, 0.53670471, 0.4993672 , 0.44754017, 0.45666512, 0.46914767, 0.38802663, 0.46365638, 0.46621008, 0.46457694, 0.52799443, 0.45461167, 0.44619208, 0.49933583, 0.52501451, 0.48158253, 0.4880402 , 0.45147875, 0.42344256, 0.46843027, 0.44768437, 0.5329015 , 0.50538285, 0.41877673, 0.58733098, 0.43604415, 0.39121616, 0.40919979, 0.52937176, 0.44574901, 0.52549154, 0.51688899, 0.49038683, 0.524215 , 0.50012394, 0.47773777, 0.51262749, 0.4798687 , 0.52585334, 0.48500928, 0.48475539, 0.56675896, 0.46403703, 0.53489713, 0.56264991, 0.60394943, 0.47180687, 0.50745135, 0.56095857, 0.4994691 , 0.44306861, 0.56434319, 0.50159993, 0.47104477, 0.51991907, 0.45333344, 0.51688131, 0.4593441 , 0.42931374, 0.49392818, 0.4453436 , 0.40615444, 0.40862632, 0.44951026, 0.47870518, 0.51139498, 0.50771963, 0.49280773, 0.39782204, 0.47624009, 0.48449106, 0.56456506, 0.44407961, 0.43908048, 0.42033429, 0.48527659, 0.45935971, 0.42840397, 0.51883242, 0.44761422, 0.52896034, 0.51500774, 0.41552883, 0.41259031, 0.43836491, 0.38030853, 0.55514969, 0.51433095, 0.51124236, 0.49550809, 0.45138244, 0.46332312, 0.49087237, 0.5171917 , 0.45277732, 0.5694115 , 0.46729251, 0.51141906, 0.51404239, 0.54713944, 0.4825544 , 0.55747765, 0.41725985, 0.50233933, 0.509464 , 0.48269443, 0.44506905, 0.44742224, 0.47860412, 0.59143417, 0.41965323, 0.52311822, 0.40899553, 0.50470071, 0.54587067, 0.51466936, 0.45873567, 0.45099424, 0.49444131, 0.50251223, 0.42547873, 0.51704076, 0.4893706 , 0.46876871, 0.42282548, 0.46696793, 0.54961649, 0.41825518, 0.46044238, 0.35542112, 0.50897749, 0.48225231, 0.50376042, 0.57978601, 0.42038623, 0.54424115, 0.50913384, 0.51967352, 0.45558098, 0.47920347, 0.46498909, 0.44171909, 0.44867245, 0.45688495, 0.55573675, 0.43855156, 0.43588202, 0.47434703, 0.4534039 , 0.57217491, 0.37514574, 0.52972952, 0.46987186, 0.48217399, 0.41992847, 0.48096654, 0.43321465, 0.4188434 , 0.53961441, 0.49815367, 0.50685561, 0.47791186, 0.44074012, 0.46634329, 0.45512305, 0.53029002, 0.46238261, 0.56227888, 0.53536169, 0.51079319, 0.41878747, 0.42561402, 0.44543746, 0.50810509, 0.43612899, 0.51354395, 0.49977016, 0.50051541, 0.53551507, 0.49244442, 0.50277678, 0.5160166 , 0.46901205, 0.41402606, 0.47680296, 0.45360224, 0.46301156, 0.43486424, 0.44673988, 0.52310368, 0.4632103 , 0.45214978, 0.40959885, 0.52888551, 0.46164559, 0.51460441, 0.44789535, 0.50312455, 0.44696829, 0.50911277, 0.47995035, 0.43894944, 0.45281734, 0.48133637, 0.48008574, 0.52402988, 0.46908205, 0.43736268, 0.40082123, 0.46024889, 0.47725868, 0.46421821, 0.49376396, 0.46658517, 0.45112352, 0.42877219, 0.51217106, 0.43646888, 0.47200625, 0.46484351, 0.46914844, 0.47738484, 0.45602496, 0.48689799, 0.40210866, 0.4214575 , 0.38249861, 0.46310603, 0.46960364, 0.53381176, 0.45413129, 0.40388998, 0.45762165, 0.46173383, 0.47930384, 0.47416536, 0.46042911, 0.52835253, 0.4808568 , 0.51425763, 0.46488765, 0.51519737, 0.54480274, 0.5512091 , 0.41074881, 0.52537858, 0.44539956, 0.45011978, 0.47728473, 0.52023034, 0.50593401, 0.48986729, 0.45146434, 0.54722046, 0.51931633, 0.51448259, 0.48541474, 0.45924783, 0.42715107, 0.49323281, 0.49868261, 0.49982019, 0.4889621 , 0.46777377, 0.44102073, 0.49832579, 0.45041965, 0.44345536, 0.4837865 , 0.5367405 , 0.46253561, 0.42348893, 0.50155398, 0.52359747, 0.45658634, 0.47784916, 0.39994949, 0.46946692, 0.45305992, 0.49578233, 0.47494601, 0.42032238, 0.51439767, 0.43803864, 0.46185746, 0.50932719, 0.43742352, 0.44015234, 0.51447279, 0.46725811, 0.457112 , 0.46233817, 0.47329737, 0.48512522, 0.45297114, 0.56179915, 0.47435093, 0.4592582 , 0.39308681, 0.51746997, 0.55920064, 0.53658168, 0.49889319, 0.46111205, 0.44999989, 0.40907121, 0.46986319, 0.43990175, 0.55234788, 0.53661098, 0.49330377, 0.4040682 , 0.48534139, 0.42742481, 0.51077351, 0.49533248, 0.42045776, 0.5344603 , 0.40972033, 0.39091224, 0.5215851 , 0.53294953, 0.48338073, 0.47730069, 0.44219701, 0.47943696, 0.56002162, 0.54204664, 0.47793652, 0.44526255, 0.5103111 , 0.46524098, 0.42589576, 0.46350273, 0.43878478, 0.50774426, 0.45034939, 0.47611014, 0.53052248, 0.49346943, 0.51199455, 0.48761231, 0.43072246, 0.55977859, 0.46252589, 0.51027744, 0.49178512, 0.43658063, 0.44140418, 0.54593446, 0.5099492 , 0.48637227, 0.44615891, 0.52222773, 0.49006639, 0.44954301, 0.42483585, 0.45181009, 0.42576419, 0.50461041, 0.51571916, 0.46911646, 0.49450105, 0.49478691, 0.50042925, 0.52024835, 0.55977333, 0.46990362, 0.52257744, 0.46704933, 0.46301227, 0.46681985, 0.40915423, 0.39569977, 0.52492256, 0.44241541, 0.49612922, 0.46483458, 0.47232841, 0.53177213, 0.48038733, 0.55456041, 0.52527035, 0.40861517, 0.47510742, 0.41377477, 0.46857172, 0.5091455 , 0.50210738, 0.4661592 , 0.43931654, 0.45337505, 0.58026336, 0.42008203, 0.57453132, 0.54057293, 0.4936788 , 0.46884728, 0.40872442, 0.3421093 , 0.43545374, 0.47669734, 0.39441519, 0.43645366, 0.47273993, 0.4449061 , 0.52002601, 0.44433336, 0.45906409, 0.50075323, 0.44701585, 0.52082957, 0.480724 , 0.47608199, 0.50947882, 0.51152428, 0.45794717, 0.50034175, 0.50344393, 0.46710402, 0.52115728, 0.47081633, 0.48959728, 0.50418432, 0.517792 , 0.51938963, 0.4133624 , 0.4279831 , 0.44733654, 0.49945796, 0.53871876, 0.46003599, 0.50691651, 0.54157237, 0.51353142, 0.48113239, 0.50680141, 0.41546118, 0.48189775, 0.41209008, 0.50158812, 0.50325101, 0.53988773, 0.48862906, 0.63422704, 0.47889701, 0.4553412 , 0.48044833, 0.56318904, 0.46243581, 0.51149109, 0.46522226, 0.47128073, 0.44339338, 0.57568214, 0.48302812, 0.51718435, 0.54207562, 0.41400792, 0.52320126, 0.45159391, 0.51923563, 0.42497262, 0.52047988, 0.53274828, 0.40228563, 0.45588327, 0.48982012, 0.47352543, 0.47223994, 0.53485516, 0.4118618 , 0.51781447, 0.46538265, 0.52966071, 0.52424753, 0.47217289, 0.46273616, 0.51869698, 0.46317502, 0.48169949, 0.41855489, 0.44349482, 0.54201979, 0.47137472, 0.52945612, 0.44409319, 0.50464629, 0.43547169, 0.47253531, 0.43410852, 0.39608272, 0.51737693, 0.54404677, 0.44838106, 0.47019746, 0.51934514, 0.48196157, 0.5892373 , 0.46130456, 0.48244219, 0.50055673, 0.46595818, 0.49014524, 0.50559026, 0.55906078, 0.43110781, 0.49156007, 0.46765774, 0.52517817, 0.51003126, 0.42910519, 0.4654511 , 0.44930796, 0.52347481, 0.4193886 , 0.48611425, 0.45503422, 0.44758526, 0.51913489, 0.48615427, 0.45915381, 0.48966807, 0.41589988, 0.52822353, 0.49645715, 0.48918748, 0.47859385, 0.48490774, 0.39952061, 0.53175053, 0.50365121, 0.42264625, 0.45147569, 0.50496877, 0.48535063, 0.50744424, 0.51465332, 0.42414122, 0.45978169, 0.46411468, 0.49467868, 0.54409124, 0.47484253, 0.5068238 , 0.49396259, 0.53964916, 0.51172738, 0.47625784, 0.44923915, 0.48536552, 0.53069993, 0.49840198, 0.51187068, 0.46036831, 0.59581793, 0.48621146, 0.51410491, 0.4376168 , 0.52375321, 0.50083837, 0.46521782, 0.43536398, 0.47399854, 0.57235617, 0.48849083, 0.49901722, 0.50293966, 0.57851074, 0.39839549, 0.53103419, 0.54299908, 0.43289476, 0.46317743, 0.50137498, 0.51942289, 0.48557863, 0.4876492 , 0.48626463, 0.43484103, 0.4987074 , 0.56855659, 0.47981524, 0.41360043, 0.48092136, 0.44830716, 0.43753123, 0.45849459, 0.38110897, 0.47937367, 0.49034043, 0.46447731, 0.46973607, 0.49351532, 0.41452509, 0.42686193, 0.42527102, 0.53671829, 0.45457996, 0.46858151, 0.46619171, 0.47110253, 0.52061093, 0.57677477, 0.4699085 , 0.4496484 , 0.51996 , 0.46174003, 0.56715301, 0.42983828, 0.47521886, 0.52808329, 0.44851982, 0.4396226 , 0.43644067, 0.46111939, 0.46671984, 0.47274524, 0.45005976, 0.47461079, 0.44815431, 0.50703117, 0.45689512, 0.55516936, 0.55672735, 0.45293766, 0.48078288, 0.53844884, 0.43451484, 0.4775106 , 0.47762549, 0.50073684, 0.45568321, 0.50863967, 0.46257647, 0.55713696, 0.42439858, 0.53324603, 0.50664103, 0.42667372, 0.48713298, 0.43443033, 0.41249548, 0.43178557, 0.49643472, 0.5357908 , 0.49244275, 0.48560644, 0.57357695, 0.47801147, 0.4656959 , 0.45555179, 0.38060909, 0.44908041, 0.51529283, 0.51116703, 0.51160535, 0.56733604, 0.4637596 , 0.5161148 , 0.49774216, 0.46608273, 0.45297828, 0.35560759, 0.54295982, 0.45633101, 0.47249958, 0.43947559, 0.54984426, 0.43906653, 0.53537798, 0.45456857, 0.48796028, 0.52143596, 0.49293529, 0.53605175, 0.40139111, 0.50600271, 0.51908615, 0.47864096, 0.48689579, 0.50902385, 0.50369694, 0.43217983, 0.50677829, 0.49677785, 0.45289065, 0.41667242, 0.40712091, 0.45834197, 0.5057925 , 0.47651157, 0.54407932, 0.50013731, 0.48122896, 0.51997758, 0.47265008, 0.52992725, 0.40527474, 0.52039882, 0.48131845, 0.4510595 , 0.40413769, 0.52876278, 0.47489569, 0.4596825 , 0.5314986 , 0.42287487, 0.52030867, 0.49836773, 0.42014062, 0.49254797, 0.538642 , 0.42061216, 0.46500315, 0.43892295, 0.43702694, 0.49656952, 0.42301431, 0.51570935, 0.37316705, 0.5071268 , 0.52374101, 0.46331057, 0.48839092, 0.45366578, 0.42403456, 0.55836914, 0.48513398, 0.50529783, 0.45320212, 0.4491557 , 0.53586077, 0.57710138, 0.49492567, 0.47280252, 0.43089228, 0.53389067, 0.48039643, 0.46776854, 0.51406094, 0.42801787, 0.52341148, 0.46519127, 0.43181267, 0.56571248, 0.52404914, 0.49590624, 0.52889378, 0.52266558, 0.49889508, 0.4612402 , 0.47452792, 0.46355367, 0.54064643, 0.5086885 , 0.43543993, 0.47346238, 0.42796893, 0.47182714, 0.4295712 , 0.4643917 , 0.49052014, 0.57363413, 0.48587792, 0.45667329, 0.52436139, 0.39282397, 0.49423503, 0.48071968, 0.44336415, 0.47717448, 0.46242517, 0.51544736, 0.4538936 , 0.48770825, 0.39656014, 0.53140325, 0.52410917, 0.47079746, 0.46092059, 0.5299388 , 0.41879821, 0.49509749, 0.45823221, 0.41639722, 0.48395687, 0.4033108 , 0.47893983, 0.48929843, 0.46444785, 0.43743348, 0.47776494, 0.4557678 , 0.53800864, 0.46523766, 0.48527377, 0.45602462, 0.46100225, 0.57398326, 0.53971796, 0.55100335, 0.49459865, 0.47626581, 0.46692972, 0.36834415, 0.54171753, 0.51124387, 0.46717469, 0.51236106, 0.48676235, 0.45806207, 0.5146505])
In [40]: se_orth_nosplit = np.array([0.03300109, 0.03481441, 0.03023695, 0.03309708, 0.03608485, 0.03213616, 0.03648971, 0.03603703, 0.03801085, 0.03446007, 0.03402841, 0.03384802, 0.02981774, 0.03169046, 0.03356073, 0.03099595, 0.03922716, 0.03448312, 0.03044878, 0.03433093, 0.03455709, 0.04124444, 0.03382217, 0.03354967, 0.03725346, 0.02786431, 0.03397265, 0.03386515, 0.03639662, 0.03304494, 0.03120514, 0.03220385, 0.03133162, 0.03302797, 0.03650748, 0.03161229, 0.03712612, 0.0344661 , 0.03387415, 0.03423439, 0.03472111, 0.03380087, 0.0349508 , 0.03290457, 0.03236628, 0.03372614, 0.03360537, 0.03333013, 0.03117967, 0.03528887, 0.03265259, 0.03079767, 0.03340276, 0.03616061, 0.03364504, 0.0375724 , 0.03255368, 0.03167439, 0.03308361, 0.03185402, 0.03473072, 0.03695189, 0.03641993, 0.0330973 , 0.03363454, 0.0345028 , 0.03639678, 0.03246405, 0.03399911, 0.03587028, 0.03256571, 0.0306337 , 0.031742 , 0.03721978, 0.03176932, 0.03141078, 0.03409519, 0.03513022, 0.03262367, 0.03497397, 0.0360815 , 0.03600124, 0.03243677, 0.0331164 , 0.03149059, 0.03691718, 0.03196211, 0.03608905, 0.03755199, 0.03560633, 0.03318104, 0.03333913, 0.03130253, 0.03838744, 0.03212319, 0.03383321, 0.03205457, 0.03356666, 0.03544808, 0.03182409, 0.03454725, 0.03517473, 0.03614165, 0.03405882, 0.03397554, 0.0320861 , 0.0347675 , 0.03705034, 0.03187931, 0.0335782 , 0.0360765 , 0.03155507, 0.03304993, 0.03317409, 0.03580993, 0.03796723, 0.03467349, 0.03234889, 0.03437548, 0.0342068 , 0.03577822, 0.03535941, 0.03214434, 0.03227508, 0.03595557, 0.0353463 , 0.03429855, 0.03149308, 0.03357296, 0.03478293, 0.03668304, 0.03814905, 0.03689617, 0.03031397, 0.0356352 , 0.03752262, 0.03552895, 0.03582576, 0.03441394, 0.03186472, 0.03521422, 0.03531235, 0.03257761, 0.03639319, 0.03429881, 0.03462154, 0.03072465, 0.03311946, 0.0308567 , 0.03208536, 0.0337658 , 0.03710826, 0.03553074, 0.03607779, 0.03132417, 0.03221319, 0.03311266, 0.03093447, 0.03409707, 0.03604383, 0.0341283 , 0.03440112, 0.03587337, 0.03311389, 0.03730926, 0.03545398, 0.03334088, 0.03262428, 0.03348249, 0.03625748, 0.033844 , 0.03630546, 0.0344714 , 0.03217738, 0.03130945, 0.03101241, 0.03133432, 0.03678077, 0.03668157, 0.03587959, 0.02945016, 0.0339122 , 0.0336982 , 0.03594457, 0.03729026, 0.0333612 , 0.02820126, 0.04043288, 0.03398932, 0.03383912, 0.03688906, 0.03529162, 0.03308525, 0.03710377, 0.03697566, 0.03545765, 0.0393287 , 0.03511189, 0.03734153, 0.0309365 , 0.03599573, 0.03340091, 0.03631366, 0.03222798, 0.03391268, 0.03210976, 0.03517214, 0.03514128, 0.0362618 , 0.03526742, 0.03506047, 0.03744133, 0.03687915, 0.0312038 , 0.03342389, 0.0322474 , 0.03504425, 0.03462889, 0.03503286, 0.03175144, 0.03223341, 0.03574766, 0.03786785, 0.03079034, 0.0384242 , 0.03443336, 0.03509659, 0.0331189 , 0.03253633, 0.03212039, 0.03106236, 0.03329483, 0.03644602, 0.02921328, 0.03308668, 0.03062067, 0.03640539, 0.03360491, 0.03389683, 0.03850131, 0.03817785, 0.03238483, 0.03239908, 0.03544456, 0.03526306, 0.03552583, 0.03512413, 0.03581594, 0.03142119, 0.03688495, 0.03302685, 0.03583831, 0.03554739, 0.03517519, 0.03342819, 0.03240985, 0.0306706 , 0.03516093, 0.03546941, 0.03305869, 0.03444686, 0.03196067, 0.03258359, 0.03225225, 0.03405359, 0.0314135 , 0.03757875, 0.03694918, 0.03637657, 0.03603543, 0.03546733, 0.03470401, 0.03565601, 0.03408186, 0.03189548, 0.03375535, 0.03530298, 0.03328103, 0.03603692, 0.03215662, 0.03455473, 0.03628099, 0.03542867, 0.03459071, 0.03740486, 0.03241454, 0.03699095, 0.03362525, 0.03453697, 0.03560096, 0.03032834, 0.03261881, 0.03519571, 0.03175207, 0.03087034, 0.03661209, 0.0384522 , 0.03349561, 0.03298505, 0.03541992, 0.03483835, 0.0326382 , 0.0331954 , 0.03040188, 0.03141423, 0.03178553, 0.03399286, 0.03670659, 0.03529897, 0.03445289, 0.03910183, 0.03693579, 0.03745123, 0.03544073, 0.03308966, 0.03547556, 0.03707074, 0.03525615, 0.03456536, 0.03722953, 0.03312706, 0.03419826, 0.03419267, 0.03646339, 0.03257063, 0.03177225, 0.03429715, 0.03310622, 0.0358156 , 0.03475818, 0.03237383, 0.03740262, 0.03248381, 0.03041655, 0.03560246, 0.035361 , 0.03596175, 0.03756068, 0.03343584, 0.02914506, 0.03257972, 0.03186928, 0.03444595, 0.03448274, 0.03487147, 0.03307192, 0.03687837, 0.03554501, 0.02969871, 0.02882754, 0.03339066, 0.03207911, 0.03308664, 0.03293925, 0.03573058, 0.02838616, 0.03435363, 0.0370067 , 0.03200265, 0.03037309, 0.03357999, 0.03656119, 0.03843627, 0.03333474, 0.03880846, 0.03059737, 0.03275688, 0.03285129, 0.03220145, 0.03253442, 0.03618327, 0.03462011, 0.03432366, 0.03459524, 0.03157359, 0.03489503, 0.03187288, 0.03052811, 0.03633338, 0.03112468, 0.0328625 , 0.03208614, 0.03383455, 0.03118336, 0.03521309, 0.03375108, 0.03258716, 0.03324911, 0.03317217, 0.03288663, 0.03633928, 0.03324796, 0.02999422, 0.03228744, 0.03726359, 0.03703165, 0.03518771, 0.0349349 , 0.03320673, 0.03414009, 0.03372712, 0.031291 , 0.03871908, 0.0389572 , 0.03480945, 0.03369572, 0.03286408, 0.03432919, 0.03617588, 0.03514486, 0.03164417, 0.03464502, 0.03206587, 0.03142204, 0.03520723, 0.03901041, 0.03472864, 0.03446797, 0.03601697, 0.03129906, 0.03326078, 0.03360178, 0.03040458, 0.03195887, 0.03420034, 0.0362044 , 0.03443136, 0.03663701, 0.0338398 , 0.03190395, 0.03857493, 0.03287289, 0.03439351, 0.03784665, 0.03712634, 0.0350762 , 0.03203719, 0.03249487, 0.03143156, 0.03528639, 0.0366816 , 0.03070296, 0.03364303, 0.03233427, 0.03063859, 0.03310549, 0.03643439, 0.0330483 , 0.03227323, 0.0332104 , 0.03490121, 0.03534769, 0.03526284, 0.0350183 , 0.03454025, 0.03610774, 0.03448665, 0.0351776 , 0.03492046, 0.03587773, 0.03598657, 0.0330358 , 0.03617942, 0.03409603, 0.03405978, 0.03243784, 0.0357111 , 0.03422289, 0.03395089, 0.03419511, 0.03751485, 0.03752663, 0.03583497, 0.03050398, 0.03557723, 0.03388622, 0.03735758, 0.03667404, 0.03525028, 0.03827193, 0.03351546, 0.03367019, 0.03508188, 0.03651116, 0.0323478 , 0.03165301, 0.0352913 , 0.03669054, 0.03384447, 0.03291889, 0.03256246, 0.03364132, 0.03335759, 0.03923671, 0.03611274, 0.03519416, 0.03637248, 0.03100812, 0.02919954, 0.0342526 , 0.03437272, 0.03316797, 0.03233925, 0.03599803, 0.03184775, 0.03229374, 0.03498719, 0.03182368, 0.03486041, 0.03209741, 0.03499238, 0.03355567, 0.03165611, 0.0341816 , 0.03607304, 0.03268691, 0.03477272, 0.03287331, 0.03539434, 0.03271712, 0.03505849, 0.03153159, 0.03890978, 0.0341056 , 0.03538186, 0.03438103, 0.03405756, 0.03129275, 0.03284114, 0.03289583, 0.03842744, 0.03463848, 0.03313342, 0.03575486, 0.03577612, 0.02983565, 0.03523834, 0.03203394, 0.03804164, 0.03472436, 0.03164143, 0.03505926, 0.032858 , 0.03478454, 0.03435101, 0.0328792 , 0.03834945, 0.03813096, 0.03258165, 0.03688323, 0.03380965, 0.03459666, 0.03554661, 0.03254974, 0.02755169, 0.03403197, 0.03686333, 0.03104238, 0.03510361, 0.03461223, 0.03273386, 0.03506437, 0.03576537, 0.03456747, 0.03157396, 0.0356792 , 0.03312858, 0.03533342, 0.03395183, 0.02952398, 0.03133257, 0.0290001 , 0.03676953, 0.03198111, 0.03659722, 0.03297687, 0.03075361, 0.03692363, 0.03547212, 0.03423718, 0.03911962, 0.0337903 , 0.03549648, 0.03114438, 0.03639941, 0.03452909, 0.03333842, 0.03325164, 0.0318147 , 0.03348661, 0.03410641, 0.03684512, 0.03423216, 0.03254072, 0.03454452, 0.0342963 , 0.03554913, 0.03560372, 0.03737662, 0.03501553, 0.03416058, 0.03649484, 0.0353373 , 0.03306659, 0.03383092, 0.03258973, 0.03128851, 0.03535036, 0.0326183 , 0.03313489, 0.03298149, 0.03001876, 0.03292446, 0.0336928 , 0.0353293 , 0.03914796, 0.03681156, 0.03548131, 0.03490516, 0.03623682, 0.03389732, 0.03374424, 0.03564595, 0.0317539 , 0.03420668, 0.03417477, 0.03212741, 0.03260072, 0.03469001, 0.03336305, 0.03773065, 0.03327173, 0.03285666, 0.03506289, 0.03237553, 0.03862934, 0.03316271, 0.03686143, 0.03178891, 0.03681042, 0.03392235, 0.03511499, 0.03274074, 0.03035431, 0.03218739, 0.02936936, 0.03737124, 0.03797012, 0.03170762, 0.03751291, 0.03129583, 0.03430933, 0.03584066, 0.03263332, 0.03347521, 0.0301352 , 0.03149596, 0.03259497, 0.02944742, 0.03743508, 0.02899995, 0.03271523, 0.03931528, 0.03301583, 0.03106761, 0.03268265, 0.0339521 , 0.03190526, 0.03722284, 0.03506228, 0.03461957, 0.03004975, 0.03152583, 0.03188864, 0.03137891, 0.03125911, 0.03443212, 0.03086839, 0.03683474, 0.0362767 , 0.03311872, 0.03572921, 0.03316816, 0.0323083 , 0.03521522, 0.03557253, 0.03365509, 0.03403009, 0.0326287 , 0.03244894, 0.03690506, 0.03537427, 0.03649063, 0.03128687, 0.03721945, 0.03366335, 0.03434353, 0.03885856, 0.03119781, 0.03581426, 0.03252629, 0.03185808, 0.03493191, 0.03395895, 0.03227437, 0.03889401, 0.03115375, 0.03044907, 0.03352168, 0.03725254, 0.04023851, 0.03216466, 0.03268951, 0.03385814, 0.03274238, 0.03196035, 0.0368327 , 0.03088809, 0.03402669, 0.03434839, 0.03245973, 0.03077444, 0.03989451, 0.03465213, 0.03461663, 0.03282729, 0.03283131, 0.02965922, 0.03205531, 0.0327355 , 0.03385212, 0.03641985, 0.03535839, 0.03367922, 0.03383552, 0.03400515, 0.03173495, 0.03555596, 0.03687678, 0.03612996, 0.0335418 , 0.03272114, 0.03485328, 0.03215544, 0.03069584, 0.03557681, 0.03713931, 0.03404416, 0.03707127, 0.03633673, 0.03394292, 0.03453973, 0.03580137, 0.03147122, 0.03635144, 0.03505335, 0.03346194, 0.0352425 , 0.03309578, 0.03361246, 0.03162954, 0.03824726, 0.03128456, 0.03308664, 0.03249983, 0.0362276 , 0.03332539, 0.03361665, 0.03060977, 0.03693433, 0.0366655 , 0.03647699, 0.03065926, 0.03489498, 0.03484303, 0.03671298, 0.03384923, 0.0325788 , 0.04077159, 0.03860608, 0.03624336, 0.03847722, 0.03230998, 0.0378641 , 0.0311665 , 0.03302566, 0.03502139, 0.03389091, 0.03040683, 0.0327089 , 0.02989121, 0.03733698, 0.03252497, 0.0368373 , 0.03775841, 0.03408025, 0.03216691, 0.03484384, 0.03315263, 0.03539599, 0.03259381, 0.03414304, 0.03295092, 0.03846051, 0.03206971, 0.03048838, 0.03200569, 0.03212656, 0.03333666, 0.0348813 , 0.03645685, 0.03917926, 0.03368409, 0.03389568, 0.03244002, 0.03411523, 0.03402238, 0.03634798, 0.03779468, 0.03357757, 0.03237467, 0.03858004, 0.03477383, 0.03394397, 0.03230739, 0.03672567, 0.03409324, 0.03521135, 0.03204122, 0.03561428, 0.03498935, 0.03859978, 0.03507295, 0.03702111, 0.03092078, 0.03511057, 0.02930761, 0.03544484, 0.03255327, 0.03181242, 0.02968715, 0.04256435, 0.03930061, 0.03385781, 0.03310617, 0.03527292, 0.03542688, 0.03528373, 0.03279445, 0.03181911, 0.03577111, 0.03326912, 0.03262289, 0.03629811, 0.0378563 , 0.0312053 , 0.03712152, 0.03346831, 0.03766047, 0.03423268, 0.03298717, 0.03066189, 0.03542913, 0.03547632, 0.03402202, 0.03417251, 0.03692622, 0.036344 , 0.03403163, 0.03608709, 0.03406768, 0.03759235, 0.03547541, 0.03092644, 0.03172698, 0.03815645, 0.0333857 , 0.03187081, 0.03629718, 0.03323217, 0.03266878, 0.03316033, 0.03248826, 0.03524635, 0.03389906, 0.03489943, 0.02924824, 0.03481939, 0.03539659, 0.03357566, 0.03399703, 0.03150613, 0.0364799 , 0.03248547, 0.03047435, 0.03421486, 0.03462341, 0.03470768, 0.03761934, 0.03315931, 0.03509053, 0.03477462, 0.0304219 , 0.03633647, 0.03796296, 0.03736833, 0.03386945, 0.03103569, 0.03055042, 0.03651902, 0.03561151, 0.03592187, 0.03560313, 0.03095704, 0.03515765, 0.03537821, 0.03516444, 0.03394301, 0.03365487, 0.03376338, 0.03625588, 0.03294306, 0.03143146, 0.03241328, 0.03014768, 0.03647779, 0.03977323, 0.03352695, 0.03299449, 0.03759643, 0.03312409, 0.031813 , 0.03387483, 0.03599901, 0.03765133, 0.03177601, 0.03316265, 0.03215408, 0.03199014, 0.03368082, 0.03376967, 0.0316642 , 0.03014013, 0.03350298, 0.03401597, 0.03083467, 0.03712924, 0.03491432, 0.0317063 , 0.03464181, 0.03251904, 0.03534762, 0.03439422, 0.03835411, 0.03623541, 0.0340211 , 0.03545453, 0.03167305, 0.03432224, 0.03212673, 0.03596564, 0.03338769, 0.03552314, 0.03368746, 0.03383131, 0.03705007, 0.03340853, 0.03626988, 0.03358266, 0.03560516, 0.03447281, 0.03138491, 0.03209008, 0.0345985 , 0.03430723, 0.0344173 , 0.03037132, 0.03406071, 0.03662991, 0.03487076, 0.03266249, 0.03427314, 0.03168054, 0.03632044, 0.03633239, 0.03903545, 0.03464841, 0.03560016, 0.03340148, 0.03447032, 0.03320291, 0.02977214, 0.03428828, 0.03499615, 0.03676726, 0.03597865, 0.0333828 , 0.03563908, 0.03401826, 0.03394263])
# to run the full simulation uncomment the following line to fit the model for every dataset and not just for the first dataset
#for i_rep in range(n_rep):
In [41]: for i_rep in range(1):
....: (x, y, d) = data[i_rep]
....: obj_dml_data = DoubleMLData.from_arrays(x, y, d)
....: obj_dml_plr_orth_nosplit = DoubleMLPLR(obj_dml_data,
....: ml_l, ml_m, ml_g,
....: n_folds=1,
....: score='IV-type',
....: apply_cross_fitting=False)
....: obj_dml_plr_orth_nosplit.fit()
....: this_theta = obj_dml_plr_orth_nosplit.coef[0]
....: this_se = obj_dml_plr_orth_nosplit.se[0]
....: print(np.abs(theta_orth_nosplit[i_rep] - this_theta))
....: print(np.abs(se_orth_nosplit[i_rep] - this_se))
....: theta_orth_nosplit[i_rep] = this_theta
....: se_orth_nosplit[i_rep] = this_se
....:
0.01910224353423784
0.0009369453685205489
In [42]: plt.figure(constrained_layout=True);
In [43]: ax = sns.histplot((theta_orth_nosplit - alpha)/se_orth_nosplit,
....: color=face_colors[1], edgecolor = edge_colors[1],
....: stat='density', bins=30, label='Double ML (no sample splitting)');
....:
In [44]: ax.axvline(0., color='k');
In [45]: xx = np.arange(-5, +5, 0.001)
In [46]: yy = stats.norm.pdf(xx)
In [47]: ax.plot(xx, yy, color='k', label='$\\mathcal{N}(0, 1)$');
In [48]: ax.legend(loc='upper right', bbox_to_anchor=(1.2, 1.0));
In [49]: ax.set_xlim([-6., 6.]);
In [50]: ax.set_xlabel('$(\hat{\\theta}_0 - \\theta_0)/\hat{\sigma}$');
library(data.table)
lgr::get_logger("mlr3")$set_threshold("warn")
set.seed(2222)
# to speed up the illustration we hard-code the simulation results
theta_orth_nosplit = c(0.497919857, 0.493917534, 0.416514503, 0.476232175, 0.412929436, 0.448641257, 0.564799593, 0.454269839, 0.420637156, 0.544918810, 0.517223713, 0.435538685, 0.444877006, 0.401903040, 0.461285589, 0.409819054, 0.454794314, 0.528790715, 0.499351284, 0.481874391, 0.430690729, 0.468141883, 0.473995519, 0.463688361, 0.486383887, 0.434219030, 0.488078921, 0.562504934, 0.493648169, 0.462918129, 0.502065895, 0.470101714, 0.579575510, 0.506011813, 0.462908503, 0.452329421, 0.507392534, 0.509547597, 0.460559260, 0.509577147, 0.482517277, 0.534971643, 0.355355696, 0.533763683, 0.569134235, 0.539763423, 0.535231022, 0.462566215, 0.481408931, 0.448715263, 0.491620360, 0.539071425, 0.548421990, 0.424896296, 0.492694630, 0.598550280, 0.488188605, 0.406294717, 0.463690315, 0.437664829, 0.466196729, 0.429505517, 0.495047976, 0.460093925, 0.511632538, 0.449426928, 0.438359907, 0.462342174, 0.553995422, 0.444503574, 0.526797886, 0.524949776, 0.432833900, 0.442813460, 0.480755144, 0.486566148, 0.493205249, 0.493628693, 0.452248443, 0.468385518, 0.442034753, 0.469355823, 0.489001043, 0.465142246, 0.486235968, 0.503548996, 0.465061905, 0.493052173, 0.456772879, 0.582864496, 0.539590475, 0.459290682, 0.434292315, 0.461609052, 0.402391034, 0.540729856, 0.471067738, 0.479175940, 0.529790415, 0.472256945, 0.446532830, 0.423005718, 0.458991254, 0.472673412, 0.464360526, 0.410565221, 0.493283688, 0.476627731, 0.481205501, 0.484919280, 0.462872501, 0.500217244, 0.432746440, 0.513275023, 0.405544128, 0.468472273, 0.508256506, 0.479762021, 0.483677880, 0.459198605, 0.525016367, 0.521710553, 0.512508317, 0.453933743, 0.481405329, 0.487911822, 0.430889503, 0.496192756, 0.484952386, 0.486819106, 0.506353419, 0.465582071, 0.415050715, 0.564972286, 0.461292032, 0.495115703, 0.553161337, 0.446704656, 0.472220429, 0.478994803, 0.463584015, 0.425939267, 0.533053318, 0.448593061, 0.501115032, 0.453835534, 0.452416742, 0.533753794, 0.528535596, 0.492446280, 0.566103956, 0.497161427, 0.570305831, 0.429709123, 0.608087018, 0.479085101, 0.408917109, 0.431701271, 0.517403775, 0.446731746, 0.574939312, 0.493070008, 0.502484831, 0.466572517, 0.562413467, 0.488633751, 0.560111815, 0.483960839, 0.518017051, 0.570124237, 0.490545797, 0.497894062, 0.459843449, 0.454269914, 0.504642066, 0.482850322, 0.444971692, 0.502197372, 0.438373473, 0.437651566, 0.449240020, 0.431742188, 0.520381100, 0.512522492, 0.429333319, 0.425580694, 0.477770341, 0.503151151, 0.419390810, 0.477861521, 0.486361358, 0.535398562, 0.474534557, 0.556408474, 0.492789379, 0.523017560, 0.459284415, 0.478682601, 0.420632794, 0.412326311, 0.449064497, 0.504571054, 0.396834013, 0.551599643, 0.538262919, 0.397096030, 0.446499532, 0.544902623, 0.506097343, 0.508293937, 0.450355311, 0.515324340, 0.500617844, 0.439873355, 0.495198247, 0.525396961, 0.552202345, 0.488730062, 0.402460890, 0.416635327, 0.497267997, 0.520875427, 0.455588919, 0.402049605, 0.476783076, 0.537876118, 0.470556885, 0.457867359, 0.456765160, 0.504479051, 0.489995323, 0.455569306, 0.469483513, 0.390079702, 0.471024748, 0.482894810, 0.513571476, 0.415534294, 0.452503027, 0.434399958, 0.422852488, 0.511785340, 0.506290191, 0.482292080, 0.469634682, 0.444112925, 0.451343094, 0.563579449, 0.471451713, 0.504841289,
0.406902606, 0.531770369, 0.487491033, 0.507892615, 0.448710661, 0.486197505, 0.394209765, 0.437785839, 0.454763800, 0.371616500, 0.520606855, 0.480428767, 0.442876913, 0.459196417, 0.507085852, 0.474554017, 0.462776120, 0.410038223, 0.515197604, 0.451497096, 0.478149025, 0.502887512, 0.547606719, 0.488839629, 0.494233732, 0.506036670, 0.467519109, 0.464524742, 0.508500974, 0.475975637, 0.461159476, 0.457126837, 0.523177636, 0.463959211, 0.487123543, 0.525315944, 0.491118552, 0.431768234, 0.569325492, 0.506794008, 0.527366522, 0.498723377, 0.476015949, 0.547964981, 0.450281890, 0.454583215, 0.425495858, 0.492947219, 0.461566235, 0.466014009, 0.431693899, 0.463059411, 0.487301203, 0.490328960, 0.544269556, 0.464415072, 0.566969528, 0.445252051, 0.462514001, 0.478097843, 0.565904142, 0.487664489, 0.442908689, 0.446474670, 0.483292597, 0.465313413, 0.503799898, 0.438849661, 0.481389333, 0.474853887, 0.522913447, 0.374109506, 0.454236279, 0.486927489, 0.502901127, 0.509538213, 0.433342962, 0.532164120, 0.472878759, 0.524534580, 0.500116551, 0.488895397, 0.489134366, 0.506899394, 0.441709786, 0.451354616, 0.492003313, 0.523118242, 0.509114607, 0.482828539, 0.358140577, 0.489292694, 0.477061463, 0.495402486, 0.535556587, 0.500220764, 0.448570380, 0.450850394, 0.477938931, 0.525512810, 0.493065395, 0.391602699, 0.533114415, 0.536416015, 0.446714275, 0.437596139, 0.467890326, 0.488901105, 0.483402132, 0.490373081, 0.506180100, 0.504457745, 0.539816949, 0.487440588, 0.436065814, 0.491635827, 0.466587478, 0.496002901, 0.502890698, 0.403564886, 0.447846202, 0.451144085, 0.537151792, 0.453237238, 0.492467500, 0.489060635, 0.483867166, 0.478079107, 0.504410439, 0.458613011, 0.518290080, 0.536046545, 0.502619007, 0.464894707, 0.502700404, 0.495946217, 0.414731709, 0.471975282, 0.487328511, 0.413396165, 0.542984304, 0.521382504, 0.492463477, 0.426331611, 0.434812060, 0.458787566, 0.508306876, 0.518786668, 0.474315857, 0.486822128, 0.520971953, 0.574398576, 0.549453493, 0.506843957, 0.426187920, 0.498536554, 0.477876796, 0.482435691, 0.484321573, 0.426083985, 0.470625505, 0.493741241, 0.500207599, 0.536492958, 0.452246607, 0.377825451, 0.473493136, 0.435814098, 0.424855958, 0.501319454, 0.483646558, 0.469367424, 0.486704759, 0.495795203, 0.517041019, 0.478155002, 0.496000721, 0.464572537, 0.459796986, 0.470899664, 0.511479534, 0.485689684, 0.529648017, 0.449999118, 0.420596592, 0.500594356, 0.434085603, 0.490270135, 0.477414613, 0.526618665, 0.536497112, 0.467812620, 0.519881940, 0.449234834, 0.508020928, 0.535812313, 0.442558061, 0.484649535, 0.512863915, 0.541242690, 0.550442852, 0.565686029, 0.504660914, 0.396277618, 0.484955116, 0.515723327, 0.477598622, 0.416861345, 0.502547968, 0.537359503, 0.489845172, 0.469928615, 0.446495108, 0.381923366, 0.431854062, 0.488958967, 0.484907945, 0.433950046, 0.501524827, 0.435876120, 0.442164637, 0.433197212, 0.483391341, 0.483728319, 0.546662847, 0.504339642, 0.423932310, 0.496990723, 0.428061227, 0.419677195, 0.546069140, 0.452573252, 0.496171145, 0.504014929, 0.491759941, 0.497083929, 0.449750318, 0.514053104, 0.526085161, 0.434903119, 0.459551015, 0.489590939, 0.563278794, 0.472936465, 0.498477471, 0.475405241, 0.476529760, 0.508896391, 0.543750617, 0.460913380,
0.431687488, 0.383495477, 0.434627056, 0.488995114, 0.527154098, 0.490159276, 0.550225931, 0.468891897, 0.580101771, 0.516873799, 0.525226380, 0.509673522, 0.527530156, 0.484039748, 0.389759962, 0.557329601, 0.534055641, 0.470421996, 0.513326534, 0.506179662, 0.466411715, 0.427173217, 0.441512652, 0.487748190, 0.469667544, 0.551689030, 0.416847005, 0.483970006, 0.406643898, 0.510567708, 0.468731842, 0.438849360, 0.471268111, 0.489649755, 0.475076521, 0.535650143, 0.436620485, 0.430850300, 0.462973276, 0.496233915, 0.464436052, 0.482813618, 0.424198267, 0.440998957, 0.421224739, 0.553106564, 0.431543119, 0.514836309, 0.362607660, 0.511919674, 0.470734442, 0.436490286, 0.473845405, 0.538398221, 0.434689200, 0.453848478, 0.452607824, 0.490511387, 0.545845137, 0.481027562, 0.534631343, 0.481061454, 0.421952035, 0.501293447, 0.503267908, 0.524129919, 0.468005273, 0.420095523, 0.482321408, 0.435568140, 0.505468457, 0.445403584, 0.394345362, 0.481000182, 0.497226180, 0.511953478, 0.519256134, 0.547721311, 0.545972972, 0.490388860, 0.559827807, 0.424905265, 0.396301574, 0.520752281, 0.437183024, 0.463689867, 0.464779675, 0.426796849, 0.545466021, 0.462249159, 0.451134135, 0.450480408, 0.467373532, 0.520838877, 0.492399606, 0.467859700, 0.504814931, 0.542760844, 0.529152117, 0.464995017, 0.449365195, 0.379441350, 0.513754653, 0.501034146, 0.439605643, 0.484473863, 0.522262813, 0.492058343, 0.473674131, 0.420512721, 0.498284020, 0.467669993, 0.477529434, 0.528584514, 0.572190697, 0.506540793, 0.447599793, 0.433805725, 0.503107069, 0.516442451, 0.485110381, 0.424682777, 0.398027102, 0.482317104, 0.433980477, 0.461538676, 0.498237221, 0.522989967, 0.510300106, 0.435528357, 0.527344115, 0.500848274, 0.479741224, 0.529417686, 0.509268466, 0.486988107, 0.476190014, 0.526693447, 0.490499951, 0.518619236, 0.439907971, 0.405807477, 0.452138227, 0.387661145, 0.482211942, 0.478250228, 0.539888152, 0.567478785, 0.422010076, 0.633773150, 0.523156768, 0.506399586, 0.465029074, 0.483846798, 0.449405185, 0.491074076, 0.502972176, 0.440784191, 0.522960685, 0.498880328, 0.438603839, 0.454750522, 0.445834027, 0.449463661, 0.506585882, 0.494284249, 0.456720783, 0.412222464, 0.416898819, 0.438749372, 0.497099949, 0.528262461, 0.454684460, 0.495619336, 0.476348749, 0.472585690, 0.478588038, 0.539029150, 0.476124122, 0.488084420, 0.468494323, 0.473745218, 0.479520879, 0.435325020, 0.500484853, 0.516165636, 0.531860743, 0.452802681, 0.454032355, 0.500077045, 0.448727192, 0.527624874, 0.387018205, 0.455842884, 0.446188323, 0.491140491, 0.406183665, 0.478779564, 0.439811868, 0.500198597, 0.483131044, 0.474464810, 0.403223509, 0.360137418, 0.480683506, 0.512824257, 0.427721937, 0.514294840, 0.437335093, 0.476997897, 0.430668800, 0.483014635, 0.540177148, 0.483564038, 0.476608503, 0.574151432, 0.514086285, 0.406449496, 0.445330206, 0.466581023, 0.450281046, 0.506629446, 0.533208780, 0.456159214, 0.530903448, 0.447681063, 0.528580441, 0.486323292, 0.430875137, 0.460236801, 0.461124368, 0.464023642, 0.497824208, 0.547315556, 0.475132506, 0.474351242, 0.475417534, 0.468169578, 0.467709981, 0.522400146, 0.459957848, 0.476054806, 0.499018913, 0.443057685, 0.507859707, 0.467337669, 0.511088398, 0.475492235, 0.559957889, 0.392966003,
0.437582776, 0.488870608, 0.504682437, 0.481458344, 0.463886671, 0.544692229, 0.444973521, 0.458163319, 0.496613965, 0.447092139, 0.559698601, 0.448910753, 0.464233475, 0.446752838, 0.506994189, 0.437084268, 0.508235221, 0.510687882, 0.492108348, 0.483867449, 0.509621877, 0.464298702, 0.383227605, 0.531522980, 0.493945852, 0.544596720, 0.516481546, 0.427392454, 0.547600686, 0.476038520, 0.457152549, 0.475628514, 0.463455359, 0.493895299, 0.494257331, 0.402184735, 0.523407084, 0.482738434, 0.528094660, 0.442753847, 0.470158656, 0.528099587, 0.522995738, 0.536049557, 0.468727950, 0.426443099, 0.577860218, 0.449334835, 0.535210544, 0.457928359, 0.488717815, 0.463366187, 0.434782169, 0.482365010, 0.419272021, 0.476663003, 0.484050988, 0.530193801, 0.400530994, 0.528516545, 0.480400951, 0.412261902, 0.447723736, 0.431595929, 0.477774047, 0.496965051, 0.514886223, 0.481970969, 0.533231482, 0.520593999, 0.527400870, 0.473956442, 0.503813567, 0.475057158, 0.365714266, 0.456621426, 0.505654981, 0.449702872, 0.463849338, 0.501395034, 0.488040758, 0.511687766, 0.446462081, 0.530227938, 0.512639575, 0.568302832, 0.486468879, 0.478180501, 0.528639074, 0.505271757, 0.416626066, 0.441627338, 0.445530329, 0.398814867, 0.460657830, 0.539945224, 0.531506161, 0.441873002, 0.474964925, 0.523338395, 0.444374358, 0.501827806, 0.492979781, 0.462656815, 0.484503882, 0.447677988, 0.510457265, 0.449300942, 0.503920646, 0.487079653, 0.480318847, 0.471914031, 0.490198105, 0.500074103, 0.482298891, 0.434724021, 0.513164025, 0.505832639, 0.386535437, 0.436334646, 0.499193197, 0.529868043, 0.441765697, 0.498210710, 0.495532505, 0.405010092, 0.486182160, 0.445503948, 0.440922032, 0.421635887, 0.481897807, 0.440487422, 0.480065084, 0.489150585, 0.531277696, 0.519740256, 0.494447331, 0.489307880, 0.491201360, 0.573117701, 0.506330793, 0.564565611, 0.496265655, 0.505822893, 0.401513863, 0.412639206, 0.522171206, 0.488410297, 0.458498142, 0.558719188, 0.558437543, 0.536569968, 0.443789456, 0.500621109, 0.475052117, 0.429723059, 0.555333689, 0.501533054, 0.383230580, 0.515683818, 0.523511203, 0.442013196, 0.561273430, 0.430105089, 0.506817930, 0.547548435, 0.514616112, 0.460726629, 0.469794405, 0.496923502, 0.467042619, 0.493594450, 0.411298369, 0.440571309, 0.431002730, 0.502676551, 0.417416215, 0.513590941, 0.564697160, 0.469856431, 0.473954756, 0.555199003, 0.516448780, 0.523839290, 0.379553757, 0.468202220, 0.515611181, 0.511841423, 0.510301188, 0.449573024, 0.472320167, 0.495526555, 0.448585315, 0.382244378, 0.478399441, 0.480704674, 0.532706935, 0.487930802, 0.507435761, 0.516170692, 0.442699560, 0.421042231, 0.526773898, 0.520541246, 0.489166652, 0.526574686, 0.500307267, 0.491709586, 0.455747912, 0.531639173, 0.527049979, 0.411682061, 0.423745075, 0.438643649, 0.477633185, 0.448793670, 0.477032846, 0.513398960, 0.488323599, 0.483460427, 0.478246132, 0.485472070, 0.469997191, 0.479596811, 0.465766931, 0.516689292, 0.459517968, 0.539667522, 0.507254357, 0.490934597, 0.496862265, 0.495062143, 0.493874819, 0.424999460, 0.371145367, 0.471599298, 0.437495249, 0.480209734, 0.509153575, 0.441003067, 0.531033347, 0.467096286, 0.522305089, 0.449518335, 0.466920063, 0.576018776, 0.517314814, 0.493501645, 0.395402059, 0.536592951)
se_orth_nosplit = c(0.0346303525, 0.0345186695, 0.0324054836, 0.0315913224, 0.0320982393, 0.0328292864, 0.0356994291, 0.0343032388, 0.0331411217, 0.0305183221, 0.0353208144, 0.0299280661, 0.0315422764, 0.0337302803, 0.0314802450, 0.0316912427, 0.0396537845, 0.0402949951, 0.0353169839, 0.0311644387, 0.0355320543, 0.0361554683, 0.0338439286, 0.0346927486, 0.0301085789, 0.0352784218, 0.0338974935, 0.0348253645, 0.0339281726, 0.0379045744, 0.0360186695, 0.0350992025, 0.0373538879, 0.0362703686, 0.0345240534, 0.0322348171, 0.0309015894, 0.0354513826, 0.0334694952, 0.0389673170, 0.0311000043, 0.0337174020, 0.0362021297, 0.0395214331, 0.0305395565, 0.0361004230, 0.0324450794, 0.0380780074, 0.0344571911, 0.0303129742, 0.0373512808, 0.0334584609, 0.0334547230, 0.0353865100, 0.0352390233, 0.0331133601, 0.0392275466, 0.0364469319, 0.0344314601, 0.0332729712, 0.0358759180, 0.0344242778, 0.0304058808, 0.0360297071, 0.0339669352, 0.0337922567, 0.0318169995, 0.0362293277, 0.0343186277, 0.0343823799, 0.0348466099, 0.0342637609, 0.0342168893, 0.0320795331, 0.0325809091, 0.0328985792, 0.0347311770, 0.0321404075, 0.0344739070, 0.0318137709, 0.0316052845, 0.0338281257, 0.0321817674, 0.0363744347, 0.0331454887, 0.0318688264, 0.0321536015, 0.0358149120, 0.0370362852, 0.0331707892, 0.0318954105, 0.0329878538, 0.0296531163, 0.0373907873, 0.0362126394, 0.0342919559, 0.0350699038, 0.0337703047, 0.0325924870, 0.0333960577, 0.0320443726, 0.0339642596, 0.0302790341, 0.0346340810, 0.0344070786, 0.0331529557, 0.0295188409, 0.0344982917, 0.0340503444, 0.0326257713, 0.0361580167, 0.0321259476, 0.0327708808, 0.0339509995, 0.0326487791, 0.0364927353, 0.0380448308, 0.0355253484, 0.0312146000, 0.0331322147, 0.0321922246, 0.0333917312, 0.0394408285, 0.0350678586, 0.0316880081, 0.0309265463, 0.0343540426, 0.0369323021, 0.0339410141, 0.0331134986, 0.0321600676, 0.0323424325, 0.0365633933, 0.0323377799, 0.0337075893, 0.0371320637, 0.0352851133, 0.0345081644, 0.0343081360, 0.0343843873, 0.0322395532, 0.0336599025, 0.0372701432, 0.0327237778, 0.0330885488, 0.0341332881, 0.0342574843, 0.0347684232, 0.0340171002, 0.0294029961, 0.0353561972, 0.0346303050, 0.0330659311, 0.0373703421, 0.0365832062, 0.0335523539, 0.0337258084, 0.0332723263, 0.0346228748, 0.0342243297, 0.0363512253, 0.0363657924, 0.0331923490, 0.0327257110, 0.0359920284, 0.0319909798, 0.0334852850, 0.0366051866, 0.0352345872, 0.0358754296, 0.0328472885, 0.0333915701, 0.0308769045, 0.0329099140, 0.0365275563, 0.0368098133, 0.0339216667, 0.0346919781, 0.0342473437, 0.0371297663, 0.0358502444, 0.0333085513, 0.0359800969, 0.0379277218, 0.0306404832, 0.0338071977, 0.0345616307, 0.0338983399, 0.0360714427, 0.0327305034, 0.0340865449, 0.0333826404, 0.0318169554, 0.0347065655, 0.0343073272, 0.0368820302, 0.0347677302, 0.0342599702, 0.0319077937, 0.0374233366, 0.0331420158, 0.0327949070, 0.0358096319, 0.0336694535, 0.0329976501, 0.0354888910, 0.0334983516, 0.0325868733, 0.0325708545, 0.0351118570, 0.0375027754, 0.0402231508, 0.0353896396, 0.0334495202, 0.0368241430, 0.0350060018, 0.0343979019, 0.0360849577, 0.0347260359, 0.0330473435, 0.0351009264, 0.0354864600, 0.0336524823, 0.0334545504, 0.0337131260, 0.0361391392, 0.0354320117, 0.0312266113, 0.0314296311, 0.0332282234, 0.0342670367, 0.0313179985, 0.0367233682, 0.0327536427, 0.0317194783, 0.0319682940, 0.0323440360, 0.0333039168, 0.0329524456, 0.0328229387, 0.0350863498, 0.0327498121, 0.0334311158, 0.0379412906, 0.0287123766, 0.0346464080, 0.0332107385, 0.0317565614, 0.0349715783, 0.0371739054,
0.0321561001, 0.0325518083, 0.0324582919, 0.0352366576, 0.0327967406, 0.0332041419, 0.0343403711, 0.0324075898, 0.0340949014, 0.0340244516, 0.0337563964, 0.0300524893, 0.0364110378, 0.0345073651, 0.0322028956, 0.0358264741, 0.0336262467, 0.0332480653, 0.0370409233, 0.0354181796, 0.0330915593, 0.0363585086, 0.0327897815, 0.0325500837, 0.0342626019, 0.0313303196, 0.0350319536, 0.0360054927, 0.0351581472, 0.0305234765, 0.0321372815, 0.0326539790, 0.0340682343, 0.0372181010, 0.0355661597, 0.0333686047, 0.0341105543, 0.0331939185, 0.0344681807, 0.0337463076, 0.0346261591, 0.0308955369, 0.0400585193, 0.0331617913, 0.0355128450, 0.0341054130, 0.0343670716, 0.0336927804, 0.0340387969, 0.0367075274, 0.0319223957, 0.0351918214, 0.0332431009, 0.0352271561, 0.0325863900, 0.0355968793, 0.0362069728, 0.0313614304, 0.0326890506, 0.0313262468, 0.0334300688, 0.0321954063, 0.0322260367, 0.0354909274, 0.0360415048, 0.0355125512, 0.0380667896, 0.0298174277, 0.0341158142, 0.0325290938, 0.0331343602, 0.0352523635, 0.0333817702, 0.0335117829, 0.0337214195, 0.0330384426, 0.0323202388, 0.0347395295, 0.0354309718, 0.0351139237, 0.0349633520, 0.0351851800, 0.0367303058, 0.0352338103, 0.0339963345, 0.0328206661, 0.0359241500, 0.0343602565, 0.0316508200, 0.0347033146, 0.0333266554, 0.0320789265, 0.0311053396, 0.0320807531, 0.0357205849, 0.0355515396, 0.0363257125, 0.0353752309, 0.0329907701, 0.0319153504, 0.0347076233, 0.0368914788, 0.0353233866, 0.0340231862, 0.0326737167, 0.0332534013, 0.0384619083, 0.0369212584, 0.0333324169, 0.0369055476, 0.0289118334, 0.0329851526, 0.0395778845, 0.0340705313, 0.0368329974, 0.0353401853, 0.0336485111, 0.0329219524, 0.0353702030, 0.0350458188, 0.0343616734, 0.0327860817, 0.0315764937, 0.0314408060, 0.0349141714, 0.0346152924, 0.0355051871, 0.0332759279, 0.0361248092, 0.0329426961, 0.0366532551, 0.0328655734, 0.0290683619, 0.0314280802, 0.0348454320, 0.0326428137, 0.0372566026, 0.0351550843, 0.0318379542, 0.0318888453, 0.0332551152, 0.0352506488, 0.0310237631, 0.0346726507, 0.0340533042, 0.0341346322, 0.0323559175, 0.0323822997, 0.0316222348, 0.0384273891, 0.0370195929, 0.0335794831, 0.0322775815, 0.0317649474, 0.0332005886, 0.0390349282, 0.0358370073, 0.0314956599, 0.0328314065, 0.0350734846, 0.0300930718, 0.0349935441, 0.0335204195, 0.0315345989, 0.0315054612, 0.0373469979, 0.0356766223, 0.0306881258, 0.0311731736, 0.0316914958, 0.0358984621, 0.0345085877, 0.0328523686, 0.0307589907, 0.0325619140, 0.0330624637, 0.0348241782, 0.0344037950, 0.0360556733, 0.0315634221, 0.0350675734, 0.0362833220, 0.0303844924, 0.0343621467, 0.0338776093, 0.0349915006, 0.0324568309, 0.0330284578, 0.0387016639, 0.0323473919, 0.0366005230, 0.0337302735, 0.0334747501, 0.0335929346, 0.0299151435, 0.0321356981, 0.0330234866, 0.0312758562, 0.0326397289, 0.0344537397, 0.0357353527, 0.0364532062, 0.0359915000, 0.0320750153, 0.0321116729, 0.0299891793, 0.0335857619, 0.0408259644, 0.0336553511, 0.0351198700, 0.0333868928, 0.0397005432, 0.0350437126, 0.0340003083, 0.0318940274, 0.0356353574, 0.0364079413, 0.0367568829, 0.0319923095, 0.0336247263, 0.0337187203, 0.0348990852, 0.0344602527, 0.0325299709, 0.0357059921, 0.0425362501, 0.0316230098, 0.0335300385, 0.0334322512, 0.0333374828, 0.0333261098, 0.0363099619, 0.0337644888, 0.0330951399, 0.0314957616, 0.0364708477, 0.0349616747, 0.0329624557, 0.0330713081, 0.0316561498, 0.0346202915, 0.0348109897, 0.0345878492, 0.0306440898, 0.0367077013, 0.0319875495, 0.0311239107, 0.0312598049, 0.0317668814, 0.0317471745,
0.0345602430, 0.0333591414, 0.0350439640, 0.0353177608, 0.0319211084, 0.0298723056, 0.0344245505, 0.0344569052, 0.0363984021, 0.0385203055, 0.0328475219, 0.0379768153, 0.0330138460, 0.0339428409, 0.0329523104, 0.0358972952, 0.0323075219, 0.0313541947, 0.0363300988, 0.0353025757, 0.0358772380, 0.0334689172, 0.0426339175, 0.0302142806, 0.0301954931, 0.0338966454, 0.0361350450, 0.0398406931, 0.0322986553, 0.0355128930, 0.0299109706, 0.0326988648, 0.0360378958, 0.0377298095, 0.0295950951, 0.0356926529, 0.0351742638, 0.0303107313, 0.0360209559, 0.0340648389, 0.0321606838, 0.0331554730, 0.0308242163, 0.0337119671, 0.0374948581, 0.0342504105, 0.0338300751, 0.0328525577, 0.0326061438, 0.0314356178, 0.0319744169, 0.0316470437, 0.0343687698, 0.0349441269, 0.0358791519, 0.0317138241, 0.0333076142, 0.0343362006, 0.0388538772, 0.0327316011, 0.0312863492, 0.0380478831, 0.0323221138, 0.0364509508, 0.0358057484, 0.0349232801, 0.0390291366, 0.0360625259, 0.0349984621, 0.0348074988, 0.0317678898, 0.0351830784, 0.0334661208, 0.0335646744, 0.0337586280, 0.0347793271, 0.0368533683, 0.0354851090, 0.0331495410, 0.0323745700, 0.0342420118, 0.0369403345, 0.0358409327, 0.0327320390, 0.0412247638, 0.0326217227, 0.0339269664, 0.0355533469, 0.0342082060, 0.0342945355, 0.0347869500, 0.0380828984, 0.0361433366, 0.0364639866, 0.0347377337, 0.0326604795, 0.0332591454, 0.0369563140, 0.0329387751, 0.0313650065, 0.0351122378, 0.0317310851, 0.0314914847, 0.0331265829, 0.0348040639, 0.0343961788, 0.0358909347, 0.0377670017, 0.0305946042, 0.0370231559, 0.0334705314, 0.0352219859, 0.0347510718, 0.0334910913, 0.0334033324, 0.0316911197, 0.0341270030, 0.0385217381, 0.0343437294, 0.0326188079, 0.0358924845, 0.0342906622, 0.0361774028, 0.0363128253, 0.0318025029, 0.0332700434, 0.0327067800, 0.0334515980, 0.0343818187, 0.0331607903, 0.0367837199, 0.0312902889, 0.0352869535, 0.0339717419, 0.0351043390, 0.0349642942, 0.0359755711, 0.0318263935, 0.0371666307, 0.0373679581, 0.0315581604, 0.0357593801, 0.0385093925, 0.0331352715, 0.0337386303, 0.0352891393, 0.0331482829, 0.0313087226, 0.0378945332, 0.0321543653, 0.0355125400, 0.0336158666, 0.0339658925, 0.0323649723, 0.0350011742, 0.0347484454, 0.0311552009, 0.0330995383, 0.0332641552, 0.0297853104, 0.0364979163, 0.0345580395, 0.0337469216, 0.0340756007, 0.0354882739, 0.0311330448, 0.0327217526, 0.0322130205, 0.0332809958, 0.0334503080, 0.0383962907, 0.0365999629, 0.0330064559, 0.0330501123, 0.0317342212, 0.0333600425, 0.0328894426, 0.0358668178, 0.0304149712, 0.0353914491, 0.0309126755, 0.0332917931, 0.0356741657, 0.0386533999, 0.0326661369, 0.0328555960, 0.0333144392, 0.0323987033, 0.0345375561, 0.0298294309, 0.0335302868, 0.0336672130, 0.0368138609, 0.0329777781, 0.0356996023, 0.0325896776, 0.0347997509, 0.0297617154, 0.0324814137, 0.0348209731, 0.0361088946, 0.0314085719, 0.0389199369, 0.0335385371, 0.0343301359, 0.0358933989, 0.0299579996, 0.0339303043, 0.0322240916, 0.0352296303, 0.0342577613, 0.0339352050, 0.0337831485, 0.0331002454, 0.0329010463, 0.0313188562, 0.0293864125, 0.0307635565, 0.0366658174, 0.0366190409, 0.0362131220, 0.0341349779, 0.0316942375, 0.0357645933, 0.0358380697, 0.0330497765, 0.0311162822, 0.0323616034, 0.0354733385, 0.0340511484, 0.0361978209, 0.0326394720, 0.0315290633, 0.0344155484, 0.0351217699, 0.0361619959, 0.0329640457, 0.0383631108, 0.0368262396, 0.0334921793, 0.0358960753, 0.0301869838, 0.0349510688, 0.0342826255, 0.0326108953, 0.0381538655, 0.0340351076, 0.0365177578, 0.0332623954, 0.0363191737,
0.0293536837, 0.0329005176, 0.0366893418, 0.0357758770, 0.0328128036, 0.0343593678, 0.0359188066, 0.0318050461, 0.0319690181, 0.0336035260, 0.0363641634, 0.0339181522, 0.0354639025, 0.0343183705, 0.0364227189, 0.0375800020, 0.0331837355, 0.0357411446, 0.0370755983, 0.0322462288, 0.0337333611, 0.0318521747, 0.0310140612, 0.0353801691, 0.0327899844, 0.0353605325, 0.0308106809, 0.0354502080, 0.0347924354, 0.0309973556, 0.0349328907, 0.0309599647, 0.0333413712, 0.0308870409, 0.0347242241, 0.0335511580, 0.0311909248, 0.0361224734, 0.0334670599, 0.0352639827, 0.0350084338, 0.0366740867, 0.0357627195, 0.0360080716, 0.0322798570, 0.0343342018, 0.0348747935, 0.0355591818, 0.0357289193, 0.0339752948, 0.0322675738, 0.0338711514, 0.0341802572, 0.0358668691, 0.0307883430, 0.0327307602, 0.0319775457, 0.0335137044, 0.0354710589, 0.0334260988, 0.0368328429, 0.0315917591, 0.0362184804, 0.0343315336, 0.0355617278, 0.0424116315, 0.0343906900, 0.0321901498, 0.0338709399, 0.0342251292, 0.0354997745, 0.0337811564, 0.0321156626, 0.0343381373, 0.0342849777, 0.0338654098, 0.0337600203, 0.0325762443, 0.0350842348, 0.0365607126, 0.0321243576, 0.0370539298, 0.0348342900, 0.0346771486, 0.0326912210, 0.0367127247, 0.0338907644, 0.0382766794, 0.0319261804, 0.0322996570, 0.0338374157, 0.0333998118, 0.0341025305, 0.0335011546, 0.0359043809, 0.0376686127, 0.0310563468, 0.0372578228, 0.0359407594, 0.0353309039, 0.0323055511, 0.0330772805, 0.0346590306, 0.0361176241, 0.0318440137, 0.0306115947, 0.0334875860, 0.0324508127, 0.0304743628, 0.0364731227, 0.0334856491, 0.0352910508, 0.0348219463, 0.0356823891, 0.0303181897, 0.0380025811, 0.0351955175, 0.0346656919, 0.0363200489, 0.0342555942, 0.0323526927, 0.0346686974, 0.0329113753, 0.0340177900, 0.0359002417, 0.0344283273, 0.0331970399, 0.0326463043, 0.0360714424, 0.0331550981, 0.0322087683, 0.0348300942, 0.0351960666, 0.0329612730, 0.0337515078, 0.0334076520, 0.0315707958, 0.0372634115, 0.0355476786, 0.0349949689, 0.0378809943, 0.0346042378, 0.0362216647, 0.0327920694, 0.0351346447, 0.0369917136, 0.0363323032, 0.0349968491, 0.0319283782, 0.0308960118, 0.0296760907, 0.0351826281, 0.0372285708, 0.0361600401, 0.0351824195, 0.0367480774, 0.0377885216, 0.0347934045, 0.0343594004, 0.0360172437, 0.0332004741, 0.0328187484, 0.0319095283, 0.0341301662, 0.0368152226, 0.0325636887, 0.0278098444, 0.0338527584, 0.0357619561, 0.0344976289, 0.0308734936, 0.0335201837, 0.0345951360, 0.0338154139, 0.0384176165, 0.0339703122, 0.0321008854, 0.0304343495, 0.0383744231, 0.0326511530, 0.0357007692, 0.0364832862, 0.0336478745, 0.0354983104, 0.0349903528, 0.0323750578, 0.0362311983, 0.0317867627, 0.0348228931, 0.0347552913, 0.0354843736, 0.0366361339, 0.0326680627, 0.0325121978, 0.0373359480, 0.0332577694, 0.0359011959, 0.0326710008, 0.0374441123, 0.0294243636, 0.0337145872, 0.0380173731, 0.0366694190, 0.0328193935, 0.0379898684, 0.0322195023, 0.0337454946, 0.0345276309, 0.0364909653, 0.0334296165, 0.0358988742, 0.0331649512, 0.0331349341, 0.0355790826, 0.0355390276, 0.0340805740, 0.0325635992, 0.0345979215, 0.0348568556, 0.0353007498, 0.0358776752, 0.0365003484, 0.0361202812, 0.0321241206, 0.0391502926, 0.0309165817, 0.0379365085, 0.0340129424, 0.0325177785, 0.0332456390, 0.0333622897, 0.0346441767, 0.0351135120, 0.0344107385, 0.0363620369, 0.0319351166, 0.0312223492, 0.0360603563, 0.0381125251, 0.0332026160, 0.0352838661, 0.0346451541, 0.0303747736, 0.0335054672, 0.0314421632, 0.0329926348, 0.0368155443, 0.0340350988, 0.0346156710, 0.0359462684)
# to run the full simulation uncomment the following line to fit the model for every dataset and not just for the first dataset
#for (i_rep in seq_len(n_rep)){
for (i_rep in seq_len(1)) {
df = data[[i_rep]]
obj_dml_data = double_ml_data_from_data_frame(df, y_col = "y", d_cols = "d")
obj_dml_plr_orth_nosplit = DoubleMLPLR$new(obj_dml_data,
ml_l, ml_m, ml_g,
n_folds=1,
score='IV-type',
apply_cross_fitting=FALSE)
obj_dml_plr_orth_nosplit$fit()
this_theta = obj_dml_plr_orth_nosplit$coef
this_se = obj_dml_plr_orth_nosplit$se
print(abs(theta_orth_nosplit[i_rep] - this_theta))
print(abs(se_orth_nosplit[i_rep] - this_se))
theta_orth_nosplit[i_rep] = this_theta
se_orth_nosplit[i_rep] = this_se
}
g_nosplit = ggplot(data.frame(theta_rescaled=(theta_orth_nosplit - alpha)/se_orth_nosplit), aes(x = theta_rescaled)) +
geom_histogram(aes(y=after_stat(density), x=theta_rescaled, colour = "Double ML (no sample splitting)", fill="Double ML (no sample splitting)"),
bins = 30, alpha = 0.3) +
geom_vline(aes(xintercept = 0), col = "black") +
suppressWarnings(geom_function(fun = dnorm, aes(colour = "N(0, 1)", fill="N(0, 1)"))) +
scale_color_manual(name='',
breaks=c("Double ML (no sample splitting)", "N(0, 1)"),
values=c("Double ML (no sample splitting)"="dark orange", "N(0, 1)"='black')) +
scale_fill_manual(name='',
breaks=c("Double ML (no sample splitting)", "N(0, 1)"),
values=c("Double ML (no sample splitting)"="dark orange", "N(0, 1)"=NA)) +
xlim(c(-6.0, 6.0)) + xlab("") + ylab("") + theme_minimal()
g_nosplit
d
2.223562e-10
d
4.12991e-11
If the nuisance models \(\hat{g}_0()\) and \(\hat{m}()\) are estimated on the whole dataset, which is also used for obtaining the final estimate \(\check{\theta}_0\), another bias is observed.
1.4. Sample splitting to remove bias induced by overfitting#
Using sample splitting, i.e., estimate the nuisance models \(\hat{g}_0()\) and \(\hat{m}()\) on one part of the data (training data) and estimate \(\check{\theta}_0\) on the other part of the data (test data), overcomes the bias induced by overfitting. We can exploit the benefits of cross-fitting by switching the role of the training and test sample. Cross-fitting performs well empirically because the entire sample can be used for estimation.
In [51]: import numpy as np
In [52]: np.random.seed(3333)
# to speed up the illustration we hard-code the simulation results
In [53]: theta_dml = np.array([0.53136293, 0.52272831, 0.49954636, 0.48334576, 0.52869052, 0.50960096, 0.53103551, 0.45362412, 0.51957113, 0.61069451, 0.51955025, 0.52937277, 0.57872562, 0.38798612, 0.46269652, 0.50968841, 0.54554518, 0.47220716, 0.52347167, 0.4977856 , 0.4810843 , 0.4878693 , 0.46698642, 0.48443493, 0.46534302, 0.45298452, 0.52018123, 0.48756693, 0.48174886, 0.50335839, 0.52151406, 0.58889816, 0.39964656, 0.39930344, 0.49470468, 0.45137001, 0.46303227, 0.55073348, 0.48722535, 0.53190714, 0.4400922 , 0.4616436 , 0.48494513, 0.40860847, 0.53792101, 0.51689913, 0.52059786, 0.52139367, 0.5408944 , 0.52875149, 0.47727554, 0.46864492, 0.53029963, 0.54005011, 0.54397691, 0.43625533, 0.55041521, 0.53843621, 0.45583064, 0.48333893, 0.49964207, 0.53990306, 0.56825612, 0.51368379, 0.55904805, 0.44723682, 0.45519343, 0.51047696, 0.59761023, 0.55036315, 0.4866524 , 0.48804607, 0.49237478, 0.4962529 , 0.5077831 , 0.57796123, 0.52018872, 0.50510419, 0.50104404, 0.52428141, 0.50728626, 0.48344444, 0.5141088 , 0.50738986, 0.43945456, 0.51760056, 0.55749679, 0.50915542, 0.46754079, 0.53593284, 0.41350172, 0.45835974, 0.56138434, 0.51870932, 0.52875457, 0.4874456 , 0.50557897, 0.51744226, 0.56002103, 0.53739233, 0.48502689, 0.54489851, 0.44516489, 0.53242594, 0.44808442, 0.61292999, 0.50723535, 0.50036306, 0.42097857, 0.50636392, 0.4316992 , 0.51075124, 0.45559127, 0.43386408, 0.47768967, 0.4443101 , 0.50053364, 0.48279399, 0.47966487, 0.49183904, 0.45375581, 0.58152412, 0.52056005, 0.4944706 , 0.41394685, 0.55891371, 0.45624999, 0.48770709, 0.49510956, 0.45966771, 0.53445906, 0.48592879, 0.47174132, 0.49600819, 0.45099157, 0.5213728 , 0.52984893, 0.54123979, 0.42396682, 0.43733171, 0.49048566, 0.45995511, 0.56677635, 0.51760897, 0.58186166, 0.43185859, 0.41208982, 0.56619156, 0.436994 , 0.52275822, 0.53982074, 0.51239518, 0.48191852, 0.47681983, 0.53004811, 0.63301374, 0.45414726, 0.50639682, 0.44919845, 0.59455466, 0.50610284, 0.45209639, 0.54129382, 0.53827613, 0.5463622 , 0.46923036, 0.56824179, 0.50299939, 0.55194225, 0.5202513 , 0.49927143, 0.49647582, 0.49307445, 0.44855007, 0.48564176, 0.53120614, 0.52780768, 0.59618638, 0.50788421, 0.48484397, 0.4735896 , 0.55641477, 0.49123131, 0.45702819, 0.40759442, 0.50326529, 0.38615708, 0.48304318, 0.5572498 , 0.52226062, 0.48415644, 0.49909294, 0.46534393, 0.41914833, 0.5041645 , 0.53792139, 0.46867822, 0.57588454, 0.45926144, 0.46670446, 0.5029714 , 0.58832425, 0.49860647, 0.51478656, 0.48416584, 0.46848499, 0.51144127, 0.48090559, 0.55818848, 0.54492593, 0.45488626, 0.61029026, 0.44918048, 0.43848122, 0.43151488, 0.54406625, 0.48015404, 0.56150037, 0.54449333, 0.5056398 , 0.54556854, 0.48725672, 0.51484744, 0.5355666 , 0.5220055 , 0.53646095, 0.48808221, 0.53077698, 0.59040192, 0.49943543, 0.5410012 , 0.56538167, 0.65180024, 0.48622944, 0.53176816, 0.61092756, 0.52541687, 0.47786008, 0.58806778, 0.54430649, 0.48797871, 0.58191711, 0.50011255, 0.55384654, 0.47269601, 0.46901973, 0.52839749, 0.49577639, 0.43846973, 0.45726162, 0.44095152, 0.52613674, 0.517144 , 0.55747665, 0.52412769, 0.42320198, 0.49221666, 0.51332249, 0.6102125 , 0.47287016, 0.48641625, 0.44462589, 0.54614317, 0.46774837, 0.47569551, 0.53311202, 0.49764894, 0.53625808, 0.54379954, 0.4710383 , 0.42705854, 0.4602913 , 0.42062529, 0.56448918, 0.57327728, 0.55341806, 0.5221236 , 0.48277209, 0.53950506, 0.51243729, 0.52756561, 0.48170962, 0.59553086, 0.47744399, 0.53232556, 0.55442321, 0.57620493, 0.5055298 , 0.574668 , 0.43901612, 0.50379399, 0.53949653, 0.52254757, 0.47339489, 0.49258173, 0.5183886 , 0.64137403, 0.4537604 , 0.54647638, 0.41302615, 0.54680774, 0.56025252, 0.52192699, 0.52571136, 0.47495594, 0.53848808, 0.5438765 , 0.44636054, 0.57098026, 0.53867527, 0.49614499, 0.44195915, 0.50894715, 0.57024528, 0.45768364, 0.48115412, 0.32967638, 0.52638924, 0.51212732, 0.54093912, 0.61031993, 0.44329509, 0.54984334, 0.56934913, 0.53850485, 0.48548764, 0.48619359, 0.4879198 , 0.48089325, 0.48093318, 0.49012512, 0.58464257, 0.51770681, 0.45842364, 0.50154922, 0.4633238 , 0.60477447, 0.3840878 , 0.55262339, 0.46379557, 0.50155153, 0.43089435, 0.50358359, 0.45836877, 0.45155272, 0.55496056, 0.5145327 , 0.55495792, 0.48836971, 0.43117485, 0.52500554, 0.48005583, 0.55926532, 0.48117512, 0.58564901, 0.57534914, 0.5566144 , 0.45938352, 0.45330172, 0.45472652, 0.51522657, 0.48634136, 0.55942669, 0.51910296, 0.56037535, 0.5562515 , 0.53264942, 0.54919816, 0.53420782, 0.49410759, 0.44943343, 0.49828523, 0.48610302, 0.48141947, 0.45363185, 0.49028851, 0.54073859, 0.47251981, 0.46071425, 0.42744937, 0.54373207, 0.49520946, 0.5513407 , 0.48579394, 0.52918428, 0.52081621, 0.50674517, 0.52533792, 0.4287179 , 0.47300219, 0.51597441, 0.52424363, 0.56801114, 0.51765264, 0.43426476, 0.38982034, 0.49132784, 0.48535703, 0.50389229, 0.50923137, 0.49587617, 0.46628965, 0.47645684, 0.5568767 , 0.46326808, 0.48160501, 0.47352731, 0.50182387, 0.53394634, 0.47127585, 0.4531616 , 0.42814489, 0.42563885, 0.41669073, 0.46000945, 0.49680789, 0.55531998, 0.47099631, 0.42235261, 0.50826971, 0.48266141, 0.48068487, 0.47415839, 0.50339469, 0.56426391, 0.50714938, 0.54175227, 0.46615236, 0.49714452, 0.58707064, 0.5836838 , 0.46398478, 0.53507902, 0.47070714, 0.46065883, 0.49412765, 0.52365932, 0.54021214, 0.53362809, 0.45948609, 0.58916472, 0.53973573, 0.53793666, 0.5215106 , 0.50351403, 0.44731051, 0.51264616, 0.54907843, 0.46733528, 0.52374948, 0.48320222, 0.44755671, 0.52588939, 0.49270753, 0.46645489, 0.52431866, 0.54457481, 0.46103621, 0.43384568, 0.51977889, 0.52428337, 0.45370544, 0.49159615, 0.39145847, 0.47093342, 0.46258887, 0.5302714 , 0.53538872, 0.43440156, 0.53291484, 0.4806193 , 0.46224348, 0.55178615, 0.44827114, 0.50469842, 0.52082239, 0.47379301, 0.47879537, 0.47875975, 0.50922767, 0.54470212, 0.47602979, 0.55249429, 0.49562578, 0.48704513, 0.42116293, 0.52129429, 0.55841889, 0.54157331, 0.56488022, 0.51442595, 0.46919796, 0.43750352, 0.50256996, 0.45722328, 0.57957189, 0.58044095, 0.47605385, 0.39757726, 0.5204202 , 0.48244 , 0.53463332, 0.50912164, 0.47047587, 0.53972193, 0.45047608, 0.40202742, 0.55285721, 0.57729907, 0.55842033, 0.4962056 , 0.45028994, 0.51468329, 0.57998638, 0.57139213, 0.5131156 , 0.47646389, 0.54095014, 0.46397522, 0.4485555 , 0.48532192, 0.44235448, 0.51347433, 0.45575175, 0.46130362, 0.54816065, 0.5522224 , 0.55504234, 0.51953446, 0.45645616, 0.59391189, 0.50501443, 0.53681853, 0.49650863, 0.49616803, 0.46026864, 0.60486169, 0.56991945, 0.48858188, 0.47884647, 0.53930786, 0.48704381, 0.50201714, 0.47889923, 0.49059772, 0.45167014, 0.51259908, 0.50756958, 0.50177218, 0.52255236, 0.50581654, 0.51760235, 0.59012884, 0.59985609, 0.48404319, 0.54707528, 0.53690096, 0.5129813 , 0.49097044, 0.44393174, 0.41610736, 0.56533807, 0.46507743, 0.5170366 , 0.47671569, 0.52740176, 0.54509643, 0.49359077, 0.57002499, 0.55915348, 0.42686252, 0.49451919, 0.44249956, 0.49482616, 0.5534894 , 0.53262351, 0.49415316, 0.46428469, 0.45910802, 0.60898016, 0.46789951, 0.60217096, 0.55632073, 0.49215808, 0.51967744, 0.42659154, 0.40320059, 0.45542357, 0.50566995, 0.43413893, 0.47236048, 0.4730468 , 0.46542312, 0.58036831, 0.47178704, 0.48796475, 0.55344806, 0.48679801, 0.52385712, 0.52714831, 0.50505238, 0.55820019, 0.53234846, 0.48099387, 0.50714475, 0.55254672, 0.49714287, 0.50279854, 0.49271632, 0.51905668, 0.50704024, 0.51989623, 0.49840608, 0.44489991, 0.49132964, 0.46017365, 0.52238502, 0.58116519, 0.48581235, 0.56232817, 0.56636795, 0.53952668, 0.50779277, 0.54246253, 0.43242725, 0.51135409, 0.42284434, 0.51788145, 0.51975574, 0.54239929, 0.50968128, 0.65253281, 0.50373615, 0.48614475, 0.49863181, 0.61503884, 0.44949508, 0.55972135, 0.49525966, 0.49261208, 0.44111368, 0.63441355, 0.48637754, 0.55398729, 0.56778945, 0.45902377, 0.57463649, 0.46121876, 0.54155362, 0.44478171, 0.52435051, 0.57087088, 0.39245217, 0.46396707, 0.50510056, 0.5069028 , 0.48665451, 0.57847942, 0.42323615, 0.54759841, 0.51696255, 0.55060358, 0.54442527, 0.47507197, 0.48206163, 0.56839492, 0.52930626, 0.48166739, 0.43443332, 0.47844387, 0.54589331, 0.48463549, 0.55026587, 0.46140555, 0.53735227, 0.45285953, 0.47713501, 0.45401454, 0.35975943, 0.51008005, 0.55417445, 0.49993366, 0.48587355, 0.55409922, 0.51759863, 0.6103252 , 0.47978687, 0.54027196, 0.53353782, 0.48636421, 0.49506053, 0.52526545, 0.58294162, 0.45855799, 0.52657642, 0.51304469, 0.52025121, 0.53871685, 0.45504542, 0.46904091, 0.49470649, 0.55199598, 0.46424461, 0.50388651, 0.46500381, 0.48168773, 0.52653742, 0.48911706, 0.48392117, 0.49352537, 0.45431985, 0.56327712, 0.52790318, 0.48400641, 0.49237972, 0.53373982, 0.39831133, 0.5649761 , 0.54902235, 0.44074566, 0.42049476, 0.51954156, 0.52258857, 0.54230792, 0.50280809, 0.4339837 , 0.4644633 , 0.50787032, 0.52874887, 0.59047196, 0.51215975, 0.52840121, 0.51081136, 0.56514492, 0.54241922, 0.53519724, 0.52820358, 0.55715823, 0.58547163, 0.49777178, 0.52623281, 0.5017119 , 0.64372458, 0.52223454, 0.51593815, 0.44428961, 0.56491759, 0.50354986, 0.46730887, 0.47892029, 0.47908061, 0.57468579, 0.50815634, 0.50228529, 0.52348798, 0.61573092, 0.39695719, 0.55025551, 0.54751588, 0.46280727, 0.47667546, 0.53997524, 0.52529117, 0.53178652, 0.51157481, 0.51605013, 0.47286783, 0.52629508, 0.5846704 , 0.49872661, 0.45332425, 0.52898154, 0.4699794 , 0.46390095, 0.47759878, 0.37733012, 0.5115532 , 0.50444157, 0.51921322, 0.53001455, 0.50901359, 0.48552478, 0.43383384, 0.44819809, 0.55747646, 0.47719987, 0.52906354, 0.53946325, 0.49667595, 0.53863836, 0.60556717, 0.50571002, 0.4651874 , 0.56184654, 0.48148176, 0.60553531, 0.44449289, 0.50566925, 0.55150338, 0.5001125 , 0.44491697, 0.45301713, 0.49467688, 0.48813413, 0.49126065, 0.49358158, 0.51385847, 0.47953825, 0.5431249 , 0.45075482, 0.59074403, 0.56949778, 0.49203902, 0.52315143, 0.57593538, 0.48795262, 0.48162209, 0.52078148, 0.4909698 , 0.45673381, 0.53827103, 0.48172874, 0.58061229, 0.47593299, 0.54073734, 0.54522708, 0.46184543, 0.4959692 , 0.4294827 , 0.44999313, 0.44973668, 0.51974583, 0.51672712, 0.53010395, 0.5228019 , 0.59312345, 0.48991575, 0.50370248, 0.48362954, 0.40305167, 0.46827572, 0.53782401, 0.54927567, 0.55919428, 0.58823779, 0.49755018, 0.52863269, 0.5288442 , 0.48776318, 0.46883583, 0.36395434, 0.59900297, 0.47637107, 0.51653852, 0.460052 , 0.59300112, 0.43400939, 0.56324665, 0.4900747 , 0.5231264 , 0.55965705, 0.53133956, 0.54956469, 0.41375333, 0.48080361, 0.60415333, 0.48909132, 0.50683815, 0.52202015, 0.53511728, 0.48063877, 0.53188573, 0.51275626, 0.49709038, 0.45734633, 0.48069221, 0.45463481, 0.55533079, 0.50718593, 0.57828931, 0.53999175, 0.54793958, 0.52846782, 0.49284515, 0.57232448, 0.42503111, 0.57343691, 0.51694705, 0.46771321, 0.43041661, 0.59017731, 0.49683349, 0.51239223, 0.50168615, 0.4435032 , 0.56883681, 0.50566309, 0.4768778 , 0.52026538, 0.56956521, 0.41806343, 0.49501286, 0.47926238, 0.46166467, 0.49539291, 0.43903866, 0.54666996, 0.36408008, 0.51138189, 0.56013817, 0.47949159, 0.51620916, 0.46406434, 0.4321351 , 0.59831181, 0.50361498, 0.53678082, 0.47096002, 0.43574422, 0.56583098, 0.59492964, 0.50920948, 0.51323477, 0.44414516, 0.57889363, 0.49414783, 0.48518923, 0.54112235, 0.44818199, 0.55585544, 0.50517962, 0.42871447, 0.52354361, 0.55310075, 0.5062023 , 0.54722905, 0.52960227, 0.50620311, 0.47954884, 0.49630517, 0.47788121, 0.58900933, 0.53264215, 0.47771333, 0.51253034, 0.45793071, 0.49573853, 0.44799847, 0.53553779, 0.51327999, 0.61805459, 0.51404724, 0.48044965, 0.5548255 , 0.42701429, 0.46496555, 0.52774306, 0.48027187, 0.49042958, 0.47982127, 0.55689975, 0.46786333, 0.49741252, 0.4305642 , 0.53657994, 0.55687243, 0.50280181, 0.50661455, 0.54592754, 0.46576354, 0.54996808, 0.48235281, 0.43544111, 0.4979282 , 0.40185842, 0.47833626, 0.52129907, 0.46030088, 0.44938231, 0.50064939, 0.46586232, 0.55550797, 0.49609405, 0.49965942, 0.47790909, 0.45629575, 0.60067607, 0.55270055, 0.52972619, 0.52151004, 0.47740249, 0.51620499, 0.40906359, 0.58986775, 0.55217603, 0.49252861, 0.53500691, 0.52813659, 0.48438568, 0.51182359])
In [54]: se_dml = np.array([0.04451577, 0.04490176, 0.04013621, 0.04605524, 0.04977187, 0.04212784, 0.05068112, 0.05198796, 0.05196444, 0.04506378, 0.04616982, 0.04480436, 0.03900256, 0.0408141 , 0.04463835, 0.04262014, 0.05274251, 0.04588425, 0.0406925 , 0.04619202, 0.0443178 , 0.05331612, 0.04373689, 0.04470977, 0.04717082, 0.03746608, 0.04401159, 0.04484962, 0.04559701, 0.04350091, 0.03976468, 0.0404016 , 0.04144786, 0.04561865, 0.0477659 , 0.04166419, 0.04830947, 0.04266617, 0.04119545, 0.04471898, 0.04575399, 0.04373017, 0.04500419, 0.0438294 , 0.04052023, 0.04356239, 0.04462905, 0.04127356, 0.04148683, 0.04766906, 0.04087846, 0.03775995, 0.04123315, 0.04630391, 0.04460172, 0.05037454, 0.04571886, 0.04152775, 0.0427847 , 0.04150442, 0.04431639, 0.04820055, 0.04518448, 0.04374162, 0.0444376 , 0.04319458, 0.04870047, 0.03979945, 0.04596467, 0.04931064, 0.03986882, 0.03962937, 0.04125952, 0.04870504, 0.04270478, 0.04403063, 0.04529282, 0.04620437, 0.04520664, 0.04757334, 0.04328208, 0.04642612, 0.04225026, 0.04083541, 0.0420464 , 0.04743644, 0.04094565, 0.04811671, 0.04929223, 0.04734472, 0.04486223, 0.04522005, 0.03971825, 0.0494492 , 0.04246032, 0.04445648, 0.04564281, 0.04151786, 0.04590433, 0.0429455 , 0.0458209 , 0.04451129, 0.04592348, 0.04650847, 0.04314546, 0.04175605, 0.04911152, 0.0449944 , 0.04460767, 0.04476069, 0.04956918, 0.04341115, 0.04423823, 0.0454818 , 0.05354475, 0.04697678, 0.04636224, 0.04538301, 0.04269111, 0.04881574, 0.04682834, 0.05010542, 0.04094056, 0.04192807, 0.04817982, 0.04648762, 0.04513767, 0.04538384, 0.0431342 , 0.04366012, 0.04717716, 0.05104802, 0.05037837, 0.03937307, 0.04616315, 0.04925793, 0.04352705, 0.04513465, 0.04400273, 0.04393063, 0.04720379, 0.04673644, 0.04176819, 0.04758137, 0.04702437, 0.04555107, 0.04090498, 0.04103245, 0.04237081, 0.04192252, 0.04409652, 0.04748513, 0.04606071, 0.0466778 , 0.04083337, 0.04075193, 0.04389173, 0.04114668, 0.04487498, 0.04585767, 0.04381332, 0.04707023, 0.04608198, 0.04299302, 0.04885294, 0.04625258, 0.04341669, 0.04364555, 0.04496738, 0.04771595, 0.04221626, 0.04669934, 0.04655068, 0.04293044, 0.04142471, 0.0430474 , 0.04142686, 0.04910276, 0.05069312, 0.04412301, 0.03741907, 0.04366074, 0.04271793, 0.04822319, 0.04955733, 0.04480288, 0.03854231, 0.05258446, 0.04632841, 0.04218598, 0.04873542, 0.04684362, 0.04173435, 0.04979273, 0.04954611, 0.04809578, 0.04894047, 0.04714015, 0.04727152, 0.0405341 , 0.04869596, 0.04211921, 0.0472672 , 0.03973715, 0.04285718, 0.0441104 , 0.04695732, 0.04738585, 0.04666111, 0.05082641, 0.04619593, 0.04727027, 0.05069427, 0.0456792 , 0.04395541, 0.04032792, 0.04956266, 0.04489709, 0.04551175, 0.04170445, 0.039224 , 0.04439168, 0.05087589, 0.04068655, 0.04844672, 0.04605304, 0.04629987, 0.04351723, 0.04140273, 0.04298938, 0.04106323, 0.04317099, 0.04732946, 0.03829337, 0.04569024, 0.04079556, 0.04742151, 0.04236698, 0.04340556, 0.04842246, 0.04903173, 0.04463866, 0.04369826, 0.04606098, 0.04681235, 0.04796005, 0.04444611, 0.05009542, 0.04329401, 0.0476929 , 0.04254179, 0.04640316, 0.04698205, 0.04594029, 0.04440063, 0.04371973, 0.04043747, 0.04419555, 0.04642175, 0.04144921, 0.04500867, 0.04081082, 0.04333657, 0.04190279, 0.04592907, 0.04117577, 0.04978934, 0.04488768, 0.04813121, 0.04600955, 0.04739863, 0.0458207 , 0.04700503, 0.04251042, 0.04407827, 0.04381095, 0.04798204, 0.04381264, 0.0468246 , 0.04432791, 0.04262607, 0.04803382, 0.04367965, 0.04473193, 0.05151782, 0.0431806 , 0.04946927, 0.04764608, 0.04325926, 0.0458524 , 0.03946785, 0.04038225, 0.04750486, 0.03900081, 0.04092891, 0.0480587 , 0.04586711, 0.04201325, 0.04718183, 0.04768575, 0.04396952, 0.04126091, 0.04288731, 0.04131418, 0.04292477, 0.04131691, 0.04881642, 0.04769117, 0.04634296, 0.04856059, 0.05022532, 0.04751964, 0.04985441, 0.04781258, 0.04623542, 0.04786554, 0.04871963, 0.04515905, 0.0444008 , 0.05050105, 0.04165176, 0.04565505, 0.04683316, 0.04405835, 0.04556649, 0.04277391, 0.04445551, 0.04347322, 0.04471067, 0.04621331, 0.04364481, 0.04606537, 0.04067207, 0.04011425, 0.04845894, 0.04581275, 0.04729834, 0.04866388, 0.0434395 , 0.03920529, 0.041447 , 0.0425844 , 0.0455108 , 0.04510086, 0.04611818, 0.04471245, 0.04859071, 0.04859655, 0.03961201, 0.04108481, 0.04350163, 0.04066479, 0.04299716, 0.0409928 , 0.04570243, 0.04068177, 0.0424811 , 0.04828524, 0.04412345, 0.03944542, 0.04338805, 0.04922771, 0.04946275, 0.0447216 , 0.04872858, 0.04079523, 0.04283997, 0.04124148, 0.04565558, 0.04281041, 0.04773227, 0.04550097, 0.04508202, 0.04749466, 0.03991025, 0.04691873, 0.04008758, 0.03827076, 0.04817591, 0.04113973, 0.04271469, 0.04142675, 0.04383673, 0.04241571, 0.0444138 , 0.0448699 , 0.0445637 , 0.04551745, 0.04266308, 0.04128645, 0.05029066, 0.04258456, 0.04092529, 0.04228388, 0.04809675, 0.04779026, 0.04547699, 0.04681882, 0.04515109, 0.04455849, 0.04062707, 0.04394847, 0.0540176 , 0.04906232, 0.04280255, 0.04570146, 0.0404382 , 0.04646957, 0.0494561 , 0.04502105, 0.04290089, 0.04731206, 0.04166547, 0.04091599, 0.04691376, 0.05123364, 0.04381634, 0.0444253 , 0.04774744, 0.04107327, 0.04970158, 0.04363839, 0.04146108, 0.0400315 , 0.04564745, 0.04812489, 0.04502683, 0.04618436, 0.04718696, 0.04238817, 0.04991695, 0.04687108, 0.04568382, 0.05013433, 0.05055857, 0.04523953, 0.04260869, 0.04038252, 0.03702686, 0.04697013, 0.04766749, 0.04014567, 0.04355803, 0.04262662, 0.04035628, 0.04161503, 0.04735591, 0.0419495 , 0.04501374, 0.04442854, 0.04369903, 0.04695287, 0.04540954, 0.04597675, 0.04189571, 0.04684482, 0.04469593, 0.04726894, 0.04466175, 0.04596659, 0.04904686, 0.04339822, 0.04567987, 0.04510266, 0.04440095, 0.04442157, 0.04872819, 0.04540775, 0.04153831, 0.04634731, 0.04988497, 0.05146532, 0.04338413, 0.04044059, 0.04872481, 0.0423472 , 0.04788741, 0.04758564, 0.04553868, 0.0495484 , 0.04609652, 0.04553537, 0.04613947, 0.04627545, 0.04510762, 0.04362909, 0.05090735, 0.04690261, 0.04309459, 0.04375869, 0.04321443, 0.04611827, 0.04708216, 0.05039695, 0.04783286, 0.04478104, 0.04727804, 0.04231607, 0.04161096, 0.04446286, 0.04402919, 0.04460511, 0.0413567 , 0.04510582, 0.04006267, 0.04453602, 0.04417668, 0.04033842, 0.04450278, 0.04465175, 0.04769427, 0.0443506 , 0.04098464, 0.04510074, 0.04695567, 0.04279144, 0.04598814, 0.04353124, 0.04606945, 0.04433485, 0.04485745, 0.04296954, 0.04964465, 0.04690607, 0.04838417, 0.04720311, 0.0464799 , 0.04139271, 0.04371307, 0.04277626, 0.05042578, 0.0463995 , 0.04486922, 0.04572229, 0.04506653, 0.03891538, 0.04640719, 0.04188499, 0.0512673 , 0.04436281, 0.04058269, 0.04509973, 0.04520745, 0.04511468, 0.04361522, 0.04003884, 0.04579448, 0.04956273, 0.04314602, 0.04821385, 0.04319144, 0.04799197, 0.04788702, 0.04104649, 0.0374441 , 0.04357559, 0.04716883, 0.04013432, 0.0482949 , 0.04696951, 0.04355121, 0.04541856, 0.04508064, 0.04584068, 0.04093339, 0.0443579 , 0.04337625, 0.04700408, 0.04378976, 0.03908194, 0.04061357, 0.03785945, 0.05134845, 0.04326495, 0.04580763, 0.0432741 , 0.03901876, 0.04663364, 0.04619656, 0.04407017, 0.05297828, 0.04412658, 0.04390205, 0.04169626, 0.05047105, 0.0428717 , 0.04460648, 0.04339771, 0.04348521, 0.04056755, 0.04543886, 0.04911588, 0.05009616, 0.04541441, 0.04439132, 0.04260284, 0.0491047 , 0.04982572, 0.04974341, 0.04517618, 0.04280691, 0.04813894, 0.04497489, 0.04452172, 0.04441396, 0.04326458, 0.03926745, 0.04521706, 0.04309774, 0.04295556, 0.04249547, 0.03923885, 0.04729502, 0.04430447, 0.04759113, 0.05264991, 0.04923853, 0.04658972, 0.04564823, 0.0463732 , 0.04504823, 0.04608112, 0.04510274, 0.04070388, 0.04523186, 0.04283385, 0.04240228, 0.04183081, 0.04182954, 0.0461404 , 0.04899793, 0.04262142, 0.04358443, 0.04386389, 0.04425116, 0.04815725, 0.04524432, 0.04668408, 0.04142753, 0.04405384, 0.04488571, 0.04732727, 0.0427516 , 0.04166611, 0.04098165, 0.04059517, 0.04621188, 0.04878163, 0.03982029, 0.04702698, 0.04009018, 0.04044453, 0.04817704, 0.045409 , 0.04112917, 0.03883971, 0.04327644, 0.04579022, 0.04034365, 0.04854353, 0.04070647, 0.04286558, 0.05156507, 0.0463016 , 0.04050037, 0.04267794, 0.04203434, 0.04334474, 0.049374 , 0.0476435 , 0.04481233, 0.0399334 , 0.04068469, 0.04258163, 0.03980833, 0.04205589, 0.04616518, 0.04174171, 0.05102498, 0.04772019, 0.04038319, 0.04834888, 0.04541133, 0.04542193, 0.0444672 , 0.04600997, 0.0472814 , 0.04552797, 0.04271417, 0.04199389, 0.04742649, 0.04821626, 0.04620429, 0.03949069, 0.04417927, 0.04300667, 0.04681882, 0.04731583, 0.04465423, 0.04521578, 0.04240016, 0.03912794, 0.04470876, 0.0445801 , 0.04300908, 0.05041768, 0.0394401 , 0.03940126, 0.04568624, 0.04859571, 0.05189341, 0.04170926, 0.04438121, 0.04457078, 0.04517643, 0.04370614, 0.04958972, 0.03840254, 0.04512002, 0.04392756, 0.04209291, 0.0404631 , 0.04973605, 0.04544987, 0.04430782, 0.04150844, 0.04293343, 0.03735777, 0.04090707, 0.04377457, 0.04167597, 0.04646467, 0.04530597, 0.04724711, 0.04549568, 0.04503617, 0.04175847, 0.04705591, 0.04651767, 0.04638402, 0.04362851, 0.04381222, 0.04696647, 0.04563065, 0.04006881, 0.04642041, 0.04673795, 0.04505947, 0.04894623, 0.04850348, 0.04274634, 0.04661226, 0.04584635, 0.04418679, 0.0488062 , 0.04684457, 0.04166846, 0.04580169, 0.04385596, 0.04443812, 0.042286 , 0.04960309, 0.04097018, 0.04379288, 0.04165084, 0.04515601, 0.04370915, 0.04254403, 0.03945375, 0.04632602, 0.04639047, 0.04663681, 0.04179178, 0.04725791, 0.04757171, 0.04843659, 0.04273204, 0.04256972, 0.05294427, 0.05185989, 0.04727349, 0.05511793, 0.04127009, 0.05043235, 0.04191125, 0.04420794, 0.04700199, 0.04715707, 0.03940633, 0.04295491, 0.04048252, 0.04686483, 0.04432732, 0.05075038, 0.04767598, 0.04395612, 0.04282439, 0.04487319, 0.04284997, 0.04689691, 0.04344958, 0.04622157, 0.04383957, 0.04808749, 0.04143248, 0.04020974, 0.041259 , 0.04111953, 0.04542849, 0.04617783, 0.04547408, 0.05126034, 0.0449876 , 0.04431241, 0.04331047, 0.04359685, 0.04466332, 0.04352393, 0.05053184, 0.0420588 , 0.04557359, 0.04991296, 0.04589503, 0.04366415, 0.04336119, 0.05020386, 0.04424407, 0.04738682, 0.0428142 , 0.04654414, 0.04670968, 0.04839783, 0.04567707, 0.04619739, 0.04063572, 0.04939731, 0.03943886, 0.04608255, 0.04359433, 0.04050661, 0.04111041, 0.05775102, 0.04982569, 0.04418107, 0.04213239, 0.04606388, 0.04622949, 0.04454934, 0.04437447, 0.04153468, 0.04723632, 0.04337555, 0.04531656, 0.04531844, 0.04965025, 0.03908481, 0.04886395, 0.04144512, 0.05106899, 0.0435134 , 0.04010173, 0.04324871, 0.04741957, 0.04429757, 0.04478984, 0.04471098, 0.05048376, 0.0455334 , 0.04490975, 0.0461487 , 0.04498832, 0.05002286, 0.04224944, 0.03989775, 0.04229135, 0.05001072, 0.0444921 , 0.04369312, 0.049683 , 0.04144563, 0.04192496, 0.04427042, 0.04268237, 0.04406869, 0.04276288, 0.04455397, 0.0400342 , 0.04535496, 0.04483139, 0.04496589, 0.04388483, 0.04325281, 0.04742232, 0.04147194, 0.03935482, 0.0449111 , 0.04847137, 0.04637344, 0.0476185 , 0.04605648, 0.04594097, 0.04594877, 0.04271411, 0.04803218, 0.04732811, 0.04761602, 0.0447552 , 0.04163056, 0.04239556, 0.04513909, 0.04752108, 0.04820901, 0.04538986, 0.04130778, 0.04395333, 0.04485472, 0.04424779, 0.04422037, 0.04472496, 0.04255319, 0.04775233, 0.04389739, 0.03939557, 0.04467024, 0.03869845, 0.04545562, 0.04902888, 0.04442053, 0.04398383, 0.04517772, 0.0440139 , 0.04175955, 0.04460751, 0.04662963, 0.05069564, 0.04281444, 0.04561765, 0.0429581 , 0.04310135, 0.03930565, 0.04523182, 0.0416166 , 0.03961216, 0.04025454, 0.04639793, 0.04287705, 0.04865216, 0.04496899, 0.04240827, 0.04512669, 0.04363657, 0.04442822, 0.04522617, 0.04841711, 0.049549 , 0.04556039, 0.04758399, 0.04074275, 0.04459929, 0.04159275, 0.04585022, 0.04388692, 0.04480129, 0.04471656, 0.04502996, 0.04695812, 0.04245843, 0.04893408, 0.04471161, 0.04508629, 0.04459937, 0.04035969, 0.04275175, 0.04880425, 0.04554817, 0.04345368, 0.04076855, 0.04290657, 0.04557181, 0.04811059, 0.04359669, 0.04545599, 0.04268864, 0.04507617, 0.04556212, 0.04969935, 0.0472998 , 0.04697147, 0.04470329, 0.04410011, 0.0448672 , 0.04116184, 0.0454671 , 0.04731715, 0.04707245, 0.04644871, 0.04360283, 0.04479812, 0.04705519, 0.0437019])
# to run the full simulation uncomment the following line to fit the model for every dataset and not just for the first dataset
#for i_rep in range(n_rep):
In [55]: for i_rep in range(1):
....: (x, y, d) = data[i_rep]
....: obj_dml_data = DoubleMLData.from_arrays(x, y, d)
....: obj_dml_plr = DoubleMLPLR(obj_dml_data,
....: ml_l, ml_m, ml_g,
....: n_folds=2,
....: score='IV-type')
....: obj_dml_plr.fit()
....: this_theta = obj_dml_plr.coef[0]
....: this_se = obj_dml_plr.se[0]
....: print(np.abs(theta_dml[i_rep] - this_theta))
....: print(np.abs(se_dml[i_rep] - this_se))
....: theta_dml[i_rep] = this_theta
....: se_dml[i_rep] = this_se
....:
0.021332682168086814
0.0003521847425944566
In [56]: plt.figure(constrained_layout=True);
In [57]: ax = sns.histplot((theta_dml - alpha)/se_dml,
....: color=face_colors[2], edgecolor = edge_colors[2],
....: stat='density', bins=30, label='Double ML with cross-fitting');
....:
In [58]: ax.axvline(0., color='k');
In [59]: xx = np.arange(-5, +5, 0.001)
In [60]: yy = stats.norm.pdf(xx)
In [61]: ax.plot(xx, yy, color='k', label='$\\mathcal{N}(0, 1)$');
In [62]: ax.legend(loc='upper right', bbox_to_anchor=(1.2, 1.0));
In [63]: ax.set_xlim([-6., 6.]);
In [64]: ax.set_xlabel('$(\hat{\\theta}_0 - \\theta_0)/\hat{\sigma}$');
set.seed(3333)
# to speed up the illustration we hard-code the simulation results
theta_dml = c(0.526496861, 0.516423626, 0.425721911, 0.485296273, 0.461591628, 0.449992993, 0.559168622, 0.472629416, 0.451626352, 0.583773380, 0.540160800, 0.482345390, 0.467005290, 0.451166527, 0.481830043, 0.408407292, 0.462678212, 0.548922849, 0.520159040, 0.497193803, 0.440822840, 0.487791056, 0.503975646, 0.491378910, 0.524024331, 0.479086742, 0.494545415, 0.565138341, 0.503716816, 0.457255120, 0.527315906, 0.523266212, 0.594335656, 0.529733774, 0.473181690, 0.536626049, 0.515769211, 0.551253541, 0.475809536, 0.540341285, 0.533543029, 0.565422865, 0.400152176, 0.549123154, 0.600493796, 0.595089836, 0.551132521, 0.520290454, 0.500975768, 0.478196306, 0.543913201, 0.585342980, 0.596924666, 0.448544973, 0.515724987, 0.640526277, 0.494115129, 0.443993343, 0.480925887, 0.475648583, 0.529057561, 0.461694662, 0.526163020, 0.471925816, 0.522638728, 0.485680892, 0.446199496, 0.476997020, 0.595499939, 0.479949953, 0.561294909, 0.549522249, 0.465846409, 0.476958622, 0.532324561, 0.525280727, 0.537130558, 0.508431195, 0.454000570, 0.478485684, 0.465719740, 0.488619250, 0.535829149, 0.491406242, 0.496324215, 0.513651824, 0.486021785, 0.548085775, 0.450218617, 0.589061272, 0.570028944, 0.493790633, 0.457712969, 0.490754288, 0.450186566, 0.564573252, 0.514390205, 0.536154853, 0.564205629, 0.446216003, 0.474929702, 0.424738279, 0.480728948, 0.509737486, 0.470349035, 0.395730146, 0.514206304, 0.483136001, 0.505550017, 0.455673847, 0.497664192, 0.504363088, 0.472553498, 0.548180298, 0.441851680, 0.509809776, 0.514380902, 0.515111492, 0.524438567, 0.475238211, 0.518538727, 0.537238300, 0.536895285, 0.457091071, 0.479487524, 0.527342928, 0.466519739, 0.500898678, 0.492437507, 0.524065769, 0.522812378, 0.487676679, 0.482675721, 0.606135565, 0.500221561, 0.506274601, 0.571517841, 0.443503212, 0.483164272, 0.490383478, 0.534053445, 0.426995347, 0.555014827, 0.456736852, 0.506900157, 0.462229813, 0.476749648, 0.551276460, 0.541959084, 0.541989827, 0.571251150, 0.509000317, 0.586522553, 0.481518655, 0.624988641, 0.525507021, 0.416529800, 0.469122189, 0.535983548, 0.492231164, 0.590738180, 0.497245521, 0.520602377, 0.475623045, 0.612294504, 0.504006944, 0.568137487, 0.534675673, 0.525995577, 0.600662288, 0.474244759, 0.471054751, 0.478299115, 0.489695552, 0.562705161, 0.491275830, 0.463764360, 0.511402277, 0.449216851, 0.457577952, 0.461359224, 0.441720852, 0.515833161, 0.552845018, 0.445352371, 0.433054211, 0.521784596, 0.517861769, 0.428610970, 0.490151889, 0.490057116, 0.530749907, 0.493865394, 0.575550501, 0.521199371, 0.522942683, 0.476600718, 0.483198750, 0.450953100, 0.417531459, 0.504314885, 0.542740796, 0.403608989, 0.584574209, 0.602746578, 0.428342195, 0.480118563, 0.611147593, 0.540325466, 0.536663887, 0.482854898, 0.548586883, 0.507039120, 0.462672092, 0.530467193, 0.553647450, 0.587152661, 0.528806814, 0.425184460, 0.437779291, 0.517846466, 0.527048897, 0.503559461, 0.438427889, 0.507159897, 0.542511572, 0.470296932, 0.487715405, 0.507438629, 0.525845998, 0.490429927, 0.499009890, 0.481786490, 0.448895843, 0.466312236, 0.495927151, 0.550072442, 0.407513971, 0.484162554, 0.457867808, 0.491339385, 0.558883223, 0.554413612, 0.515865134, 0.491476007, 0.432321031, 0.500748821, 0.602614445, 0.495326189, 0.537322895,
0.435082303, 0.536855850, 0.521886639, 0.556088247, 0.498664916, 0.496657563, 0.438670614, 0.479836991, 0.465567611, 0.443088724, 0.549493447, 0.520863419, 0.485880843, 0.491620501, 0.497278091, 0.499753413, 0.489089688, 0.432632333, 0.542871569, 0.488010869, 0.461325007, 0.516428267, 0.569533362, 0.505236978, 0.517686862, 0.498655296, 0.519844603, 0.473304577, 0.527883827, 0.523656344, 0.504169493, 0.455283706, 0.598325421, 0.472983499, 0.517807850, 0.547712071, 0.509369960, 0.437746837, 0.569849366, 0.522265545, 0.539601063, 0.535131141, 0.490061199, 0.562085236, 0.451379302, 0.444425409, 0.457712136, 0.529186270, 0.490129726, 0.502728521, 0.458249625, 0.464209433, 0.540701533, 0.489974256, 0.577625392, 0.490314196, 0.651041738, 0.468518457, 0.512386542, 0.468917674, 0.597005437, 0.524877138, 0.442121863, 0.485403152, 0.506072072, 0.509517194, 0.541591749, 0.467503127, 0.528268571, 0.494485255, 0.561024583, 0.399094723, 0.503840347, 0.513874090, 0.524081940, 0.519125688, 0.449089827, 0.589765730, 0.486302378, 0.563737460, 0.511757073, 0.488476937, 0.533714979, 0.537573720, 0.427938777, 0.459323285, 0.535557089, 0.582036132, 0.535193543, 0.478417149, 0.383330110, 0.529422548, 0.503942314, 0.532207930, 0.586057084, 0.511947418, 0.470331103, 0.474770615, 0.507580591, 0.546605103, 0.522188853, 0.429355286, 0.572140316, 0.561258610, 0.460131188, 0.452902720, 0.496388422, 0.506668434, 0.490149450, 0.509338802, 0.501638561, 0.499779288, 0.546238717, 0.502623875, 0.432260842, 0.497627070, 0.511316043, 0.545365089, 0.538137328, 0.457859455, 0.453587162, 0.475622619, 0.572387257, 0.485639006, 0.518053657, 0.503427803, 0.503048899, 0.540383919, 0.493457345, 0.450395090, 0.534949215, 0.557382894, 0.534380782, 0.508725941, 0.551246161, 0.525896629, 0.400648336, 0.488763074, 0.527672695, 0.424693743, 0.575318311, 0.551457444, 0.537292914, 0.447912640, 0.463898168, 0.445057758, 0.535534587, 0.556765806, 0.515784047, 0.480877073, 0.572669277, 0.586097559, 0.538941708, 0.531877884, 0.438362663, 0.529896235, 0.498074681, 0.512755317, 0.501546660, 0.460365491, 0.518271383, 0.521026833, 0.524296674, 0.578400838, 0.471163716, 0.407778367, 0.531479083, 0.457065729, 0.452797248, 0.547754631, 0.535333251, 0.486621673, 0.533333502, 0.512585748, 0.573120633, 0.493075179, 0.536298318, 0.490408943, 0.492613851, 0.515634887, 0.520953978, 0.513414401, 0.565388881, 0.448519619, 0.467559204, 0.509864262, 0.442904006, 0.490109976, 0.534351365, 0.534179970, 0.557771444, 0.514516075, 0.519784049, 0.485297624, 0.536380072, 0.550809275, 0.448427472, 0.493015574, 0.532626063, 0.554964252, 0.534263750, 0.596174770, 0.568671474, 0.422851018, 0.538661804, 0.537325385, 0.503527249, 0.439025905, 0.551841010, 0.572171145, 0.518936835, 0.482281929, 0.461553681, 0.410208067, 0.448480865, 0.480915422, 0.486679326, 0.481556363, 0.537210814, 0.459726483, 0.469751432, 0.452110055, 0.508456501, 0.543183393, 0.583745800, 0.527349244, 0.464877344, 0.535844275, 0.442712049, 0.432099315, 0.557778388, 0.499356396, 0.525486405, 0.514135561, 0.536915139, 0.523707582, 0.462669642, 0.574375201, 0.538129597, 0.460811022, 0.469898047, 0.476470499, 0.597182716, 0.499428235, 0.520102652, 0.512266150, 0.507346379, 0.530595860, 0.596174529, 0.493450429,
0.467753442, 0.418983851, 0.443016650, 0.489288188, 0.519135540, 0.529404070, 0.590913702, 0.514905818, 0.595071633, 0.564989643, 0.550394820, 0.529932447, 0.555261839, 0.482325619, 0.428649183, 0.586819452, 0.577900128, 0.454670496, 0.516212671, 0.525556836, 0.495380852, 0.433787371, 0.456882873, 0.505626647, 0.480399964, 0.567110779, 0.463698835, 0.509625083, 0.419048593, 0.543423737, 0.485946550, 0.461914165, 0.504166865, 0.509459252, 0.496259111, 0.556228140, 0.448400693, 0.457535908, 0.465489789, 0.509267170, 0.477808918, 0.529695325, 0.434139856, 0.454591158, 0.429105207, 0.575914908, 0.447020348, 0.529844733, 0.373347431, 0.537159093, 0.499685444, 0.418661666, 0.456443352, 0.584872676, 0.451602520, 0.450348484, 0.453250391, 0.521788836, 0.526172956, 0.488037086, 0.533567162, 0.560744168, 0.440398033, 0.532711540, 0.530134994, 0.504209397, 0.447400211, 0.425008271, 0.516812732, 0.455672264, 0.540950117, 0.472910067, 0.414727073, 0.502684799, 0.506963329, 0.554433733, 0.554821413, 0.573707647, 0.547555131, 0.507832092, 0.593165686, 0.433090128, 0.411529072, 0.536899849, 0.468702340, 0.490093574, 0.504769803, 0.472109330, 0.549933986, 0.482994009, 0.490996350, 0.477038252, 0.469576210, 0.549356725, 0.500941790, 0.481368822, 0.548622848, 0.547555863, 0.567670275, 0.478386094, 0.447765379, 0.441149465, 0.509846825, 0.550376789, 0.468940821, 0.539059513, 0.525857246, 0.513721013, 0.489882421, 0.452975517, 0.553226219, 0.502540710, 0.488158955, 0.542927835, 0.621186936, 0.535315345, 0.479982269, 0.445903284, 0.550138165, 0.535177013, 0.529421950, 0.443045350, 0.451976715, 0.505082709, 0.452919746, 0.469870510, 0.531751243, 0.537707204, 0.527677575, 0.445564817, 0.563943631, 0.547361399, 0.484122612, 0.558962249, 0.510648089, 0.509582346, 0.504780801, 0.561277203, 0.508949258, 0.524267386, 0.442101579, 0.439593077, 0.465579898, 0.449070866, 0.493639443, 0.528873344, 0.573281801, 0.573981200, 0.453894862, 0.671048305, 0.559325727, 0.524238917, 0.480040240, 0.506184772, 0.472709532, 0.503019068, 0.528312156, 0.457797678, 0.579639824, 0.527183702, 0.492649005, 0.488461666, 0.462559863, 0.488460190, 0.498344108, 0.500522942, 0.485034370, 0.426707641, 0.418255274, 0.455822485, 0.501971487, 0.552856777, 0.464114429, 0.504789479, 0.522911187, 0.486247849, 0.494864775, 0.584647154, 0.463437017, 0.491042601, 0.489098208, 0.485216405, 0.493737063, 0.464072169, 0.529556484, 0.552803154, 0.527666056, 0.506144058, 0.487192429, 0.549385636, 0.486568486, 0.521919182, 0.445463372, 0.471090513, 0.492484255, 0.520809889, 0.416783284, 0.514696109, 0.474594277, 0.478141640, 0.528087473, 0.502851883, 0.434506318, 0.345439342, 0.503857382, 0.541490283, 0.452425178, 0.552336632, 0.499726440, 0.486331554, 0.465456844, 0.536046765, 0.557922771, 0.531833250, 0.468322143, 0.607299241, 0.524984583, 0.401490789, 0.459539340, 0.509631698, 0.516039897, 0.499634395, 0.540545514, 0.474438205, 0.534083978, 0.462877421, 0.564507870, 0.496374027, 0.449156130, 0.492138665, 0.477158718, 0.496583686, 0.541931996, 0.611491369, 0.447694369, 0.527832536, 0.495899280, 0.512490385, 0.512115692, 0.537270951, 0.512055966, 0.475501540, 0.532834960, 0.468651723, 0.492526802, 0.487603388, 0.562583651, 0.522808398, 0.588727940, 0.469481727,
0.476338523, 0.491492243, 0.547023400, 0.488819245, 0.513475593, 0.551406361, 0.482524929, 0.488237202, 0.541827499, 0.440964329, 0.597819361, 0.459415432, 0.521870553, 0.506377348, 0.526383199, 0.500716460, 0.527163335, 0.537013067, 0.533191111, 0.538923317, 0.514834296, 0.500703010, 0.410222358, 0.566701369, 0.492936524, 0.574343233, 0.547803027, 0.460683313, 0.578993390, 0.485218099, 0.482649606, 0.490506250, 0.460607633, 0.538219154, 0.510574017, 0.412753842, 0.530022133, 0.496629899, 0.551721755, 0.488979683, 0.477666640, 0.584765504, 0.558994289, 0.576869112, 0.481600458, 0.419980168, 0.589541238, 0.495017547, 0.559769645, 0.481570942, 0.513677992, 0.486732398, 0.470466567, 0.498390145, 0.440683750, 0.504212650, 0.484340159, 0.547354252, 0.436351249, 0.575151102, 0.519435183, 0.442065328, 0.474954844, 0.487734434, 0.518493617, 0.501238125, 0.522472876, 0.494242744, 0.557874318, 0.570759857, 0.545206104, 0.476331555, 0.505642659, 0.463123976, 0.389784479, 0.490964602, 0.544499658, 0.503343106, 0.469545223, 0.518358092, 0.524470435, 0.564577289, 0.507410959, 0.536679441, 0.560672146, 0.568249655, 0.511382617, 0.508237974, 0.545819583, 0.506918200, 0.456888247, 0.485467697, 0.503245587, 0.414701975, 0.501094135, 0.558824985, 0.566221109, 0.481771193, 0.503501521, 0.523207003, 0.466002136, 0.524448710, 0.525027135, 0.491718526, 0.505096445, 0.470404439, 0.524150053, 0.450207374, 0.544405665, 0.519299724, 0.504728313, 0.492213933, 0.529085791, 0.503375571, 0.495846105, 0.437172874, 0.539401410, 0.506089589, 0.431224626, 0.483530628, 0.543122255, 0.570319876, 0.476956820, 0.512402467, 0.508731622, 0.414489980, 0.490960439, 0.484874692, 0.491633363, 0.457767496, 0.494598247, 0.461205441, 0.513560810, 0.523383081, 0.533182717, 0.541803610, 0.542719828, 0.528873929, 0.527440157, 0.599518371, 0.544493122, 0.566819276, 0.527787376, 0.541638211, 0.418428540, 0.429952163, 0.518814515, 0.491632118, 0.494269810, 0.574714485, 0.590469517, 0.535498072, 0.473329934, 0.526000632, 0.530424888, 0.442788363, 0.582018897, 0.555182445, 0.401056185, 0.511486328, 0.564567275, 0.469935679, 0.570279981, 0.404521404, 0.512463207, 0.540748561, 0.564057043, 0.502089398, 0.489045252, 0.527059960, 0.486244041, 0.519106832, 0.423082151, 0.465851224, 0.461443352, 0.548314098, 0.457065014, 0.559127045, 0.575568903, 0.471134846, 0.493302706, 0.575362420, 0.536046980, 0.528900674, 0.415325148, 0.503520321, 0.542434939, 0.536627002, 0.538721496, 0.485037398, 0.519167961, 0.483395958, 0.475782570, 0.420312314, 0.502585927, 0.487045097, 0.568882355, 0.486363429, 0.518771287, 0.528704723, 0.465141994, 0.434499341, 0.540650252, 0.547986414, 0.485954670, 0.552500106, 0.552781515, 0.468090853, 0.495576849, 0.571792515, 0.567468852, 0.412088625, 0.474065400, 0.445789609, 0.507640086, 0.462192756, 0.485610917, 0.517984254, 0.531908967, 0.498897240, 0.498344967, 0.486508236, 0.488946633, 0.495260075, 0.486465657, 0.543663830, 0.483577617, 0.567011513, 0.548034817, 0.479783299, 0.512339483, 0.496316821, 0.543433184, 0.477249429, 0.384742888, 0.486166728, 0.471801654, 0.513562722, 0.545886714, 0.468391892, 0.551923479, 0.518712240, 0.574853864, 0.441880394, 0.482785165, 0.624194556, 0.535141387, 0.521832675, 0.416633931, 0.521563762)
se_dml = c(0.0459486570, 0.0427164347, 0.0421129286, 0.0426511338, 0.0435054521, 0.0445955603, 0.0447804118, 0.0491623322, 0.0427274029, 0.0391098010, 0.0430905799, 0.0402490415, 0.0437686667, 0.0452208507, 0.0403921614, 0.0403273047, 0.0474629308, 0.0541207268, 0.0495409280, 0.0407166801, 0.0474830398, 0.0491076836, 0.0472301709, 0.0470574759, 0.0375038341, 0.0450470270, 0.0442185701, 0.0438963378, 0.0469834110, 0.0528586340, 0.0443037998, 0.0444308116, 0.0464334131, 0.0456276458, 0.0523982611, 0.0451506399, 0.0394511084, 0.0436577101, 0.0447160580, 0.0485423722, 0.0392618715, 0.0477369092, 0.0458259776, 0.0501888707, 0.0433958344, 0.0475662703, 0.0437856579, 0.0513090428, 0.0459377155, 0.0399055055, 0.0496063179, 0.0439646027, 0.0453017598, 0.0448261788, 0.0457971566, 0.0408774765, 0.0495501401, 0.0511912037, 0.0430238911, 0.0423318223, 0.0467567273, 0.0451718192, 0.0398980313, 0.0464729680, 0.0479479017, 0.0427360589, 0.0413409894, 0.0446780370, 0.0436500724, 0.0421590535, 0.0428913945, 0.0451454341, 0.0460799035, 0.0419040712, 0.0434802341, 0.0450984753, 0.0456554873, 0.0422882860, 0.0428161642, 0.0410891398, 0.0416383607, 0.0411182801, 0.0436694519, 0.0463204355, 0.0429146700, 0.0406762889, 0.0403156249, 0.0450427097, 0.0499986817, 0.0425009064, 0.0423851213, 0.0420207198, 0.0382721681, 0.0480687536, 0.0493360598, 0.0420604155, 0.0483816926, 0.0446986818, 0.0430198138, 0.0444754635, 0.0414275714, 0.0467395243, 0.0413662637, 0.0451064995, 0.0459162125, 0.0446667029, 0.0384506948, 0.0436396598, 0.0451688359, 0.0435311307, 0.0460350028, 0.0432994998, 0.0454653628, 0.0429259328, 0.0451174802, 0.0492213093, 0.0483757672, 0.0476997522, 0.0409404632, 0.0428023977, 0.0400608012, 0.0417010074, 0.0512842029, 0.0439027054, 0.0398776901, 0.0414818928, 0.0445569613, 0.0473592097, 0.0425068015, 0.0410613056, 0.0399781505, 0.0419512151, 0.0465740305, 0.0432724066, 0.0456513953, 0.0497255423, 0.0473861540, 0.0455023734, 0.0431838225, 0.0436962266, 0.0405253742, 0.0474065347, 0.0481648103, 0.0429806576, 0.0432331384, 0.0425746022, 0.0441635910, 0.0438158911, 0.0413962544, 0.0361634128, 0.0463446301, 0.0473595368, 0.0428375847, 0.0509637127, 0.0499946463, 0.0450493984, 0.0450857911, 0.0455173857, 0.0465024348, 0.0466233605, 0.0477536954, 0.0484616344, 0.0423479893, 0.0437104948, 0.0462098604, 0.0413123913, 0.0435643722, 0.0445354878, 0.0484836627, 0.0465716093, 0.0411395786, 0.0468053510, 0.0407303381, 0.0423606864, 0.0473289424, 0.0468656683, 0.0446194090, 0.0436460935, 0.0458001910, 0.0483715867, 0.0456211425, 0.0437913013, 0.0478861747, 0.0479155896, 0.0447400940, 0.0428180984, 0.0446561011, 0.0438705665, 0.0476477944, 0.0427156391, 0.0445455841, 0.0451325894, 0.0421587754, 0.0444581848, 0.0437405431, 0.0497176519, 0.0468306029, 0.0447048861, 0.0413249220, 0.0520801097, 0.0428372668, 0.0421879925, 0.0445812532, 0.0441809219, 0.0442840059, 0.0477483549, 0.0454864402, 0.0427667284, 0.0449910206, 0.0445109011, 0.0464603374, 0.0504553017, 0.0468781210, 0.0449492753, 0.0497589629, 0.0458589525, 0.0462343860, 0.0473093490, 0.0461843641, 0.0439234166, 0.0447329945, 0.0437550159, 0.0435533290, 0.0450084391, 0.0440417089, 0.0471659895, 0.0456444977, 0.0404775543, 0.0416808372, 0.0451739034, 0.0455834295, 0.0434527193, 0.0458574752, 0.0468341642, 0.0414578168, 0.0422614777, 0.0400518909, 0.0450992298, 0.0431689994, 0.0406515428, 0.0437678938, 0.0411988174, 0.0424418484, 0.0478578790, 0.0386177052, 0.0444807961, 0.0413502267, 0.0405502421, 0.0443049592, 0.0492122218,
0.0415023030, 0.0406705566, 0.0408198686, 0.0457456178, 0.0416810788, 0.0429542667, 0.0471460861, 0.0440530588, 0.0439950736, 0.0441586380, 0.0427035500, 0.0383669585, 0.0485645030, 0.0444517010, 0.0421927214, 0.0478535243, 0.0453883240, 0.0444123182, 0.0491793741, 0.0456530975, 0.0443722190, 0.0487743910, 0.0413677165, 0.0415438745, 0.0423889310, 0.0446884081, 0.0477893230, 0.0488963534, 0.0448559463, 0.0383212001, 0.0432652080, 0.0447385473, 0.0446487641, 0.0517938992, 0.0465202127, 0.0432993652, 0.0437825833, 0.0433960405, 0.0442868149, 0.0442263739, 0.0468955832, 0.0404869329, 0.0568290047, 0.0429494028, 0.0449968635, 0.0483581875, 0.0442350392, 0.0446020077, 0.0440855167, 0.0472119089, 0.0417431750, 0.0445306463, 0.0439853842, 0.0442217633, 0.0436724987, 0.0459136986, 0.0470342098, 0.0419853234, 0.0424214546, 0.0429135037, 0.0439950418, 0.0438369690, 0.0460588787, 0.0457362680, 0.0463082825, 0.0469805936, 0.0486961722, 0.0383118855, 0.0458579237, 0.0431902222, 0.0429493904, 0.0473916352, 0.0439362802, 0.0427028399, 0.0437357114, 0.0411564542, 0.0408628440, 0.0431157777, 0.0455318817, 0.0444617988, 0.0427746643, 0.0449289560, 0.0464962667, 0.0473235300, 0.0442455742, 0.0443392750, 0.0462599843, 0.0460606282, 0.0414533830, 0.0447796945, 0.0435026062, 0.0446323556, 0.0406237477, 0.0409108276, 0.0443292861, 0.0447124238, 0.0457235779, 0.0464698359, 0.0411350395, 0.0402359459, 0.0461321620, 0.0481023422, 0.0451502616, 0.0435777011, 0.0408641870, 0.0462350692, 0.0518780519, 0.0512115864, 0.0439761646, 0.0474554162, 0.0382615269, 0.0470998767, 0.0475240737, 0.0436634844, 0.0469561279, 0.0450682076, 0.0457623311, 0.0433321327, 0.0485009586, 0.0438042100, 0.0436128603, 0.0418350894, 0.0403915008, 0.0422523804, 0.0464884117, 0.0459027093, 0.0463916528, 0.0413689953, 0.0464401644, 0.0426021949, 0.0498590605, 0.0437853905, 0.0382619275, 0.0408580736, 0.0481102918, 0.0440044610, 0.0485725756, 0.0479091816, 0.0418599146, 0.0433232062, 0.0437688258, 0.0440449956, 0.0403664443, 0.0447550672, 0.0474148565, 0.0428229000, 0.0435350351, 0.0449911650, 0.0414966640, 0.0491750640, 0.0498749595, 0.0429196305, 0.0426677387, 0.0415629202, 0.0462883275, 0.0481573542, 0.0467125742, 0.0420880244, 0.0429284056, 0.0441345758, 0.0404150391, 0.0474525876, 0.0443092955, 0.0417593836, 0.0422202758, 0.0467853919, 0.0470457469, 0.0405760563, 0.0422148487, 0.0432156330, 0.0465999405, 0.0448843270, 0.0448184505, 0.0400301029, 0.0435553117, 0.0433940780, 0.0410873462, 0.0433235063, 0.0463503703, 0.0442648445, 0.0466138084, 0.0467931767, 0.0402387083, 0.0456172780, 0.0428245221, 0.0472417672, 0.0437131123, 0.0438055181, 0.0508492921, 0.0427088761, 0.0456952776, 0.0463181495, 0.0440933150, 0.0451865059, 0.0395495168, 0.0448969883, 0.0428658049, 0.0427025612, 0.0413126381, 0.0454942076, 0.0443736837, 0.0450609810, 0.0476441516, 0.0413185787, 0.0410480512, 0.0395182161, 0.0454214481, 0.0553785506, 0.0424342005, 0.0440304949, 0.0436434890, 0.0519229687, 0.0465609539, 0.0435109159, 0.0427501159, 0.0471582178, 0.0447385240, 0.0476317385, 0.0434190068, 0.0451516642, 0.0441780706, 0.0452948974, 0.0429244756, 0.0428850559, 0.0470191975, 0.0570160317, 0.0409190135, 0.0431539541, 0.0437967596, 0.0435078040, 0.0425420034, 0.0484456576, 0.0454732112, 0.0425891601, 0.0417193649, 0.0492266093, 0.0445264977, 0.0461257120, 0.0415995350, 0.0403636208, 0.0471628772, 0.0463507274, 0.0488585717, 0.0416637643, 0.0470971190, 0.0424058623, 0.0408225405, 0.0417604444, 0.0412125302, 0.0425828113,
0.0482499519, 0.0446316339, 0.0456034930, 0.0460902596, 0.0444504770, 0.0379709512, 0.0448907516, 0.0507556669, 0.0459480772, 0.0499707081, 0.0426258110, 0.0488695566, 0.0415298811, 0.0414485563, 0.0429323843, 0.0468609363, 0.0419407553, 0.0438273134, 0.0472767043, 0.0475352544, 0.0470211840, 0.0447641077, 0.0580175276, 0.0371047782, 0.0404746548, 0.0427253652, 0.0477529842, 0.0528949160, 0.0434883326, 0.0482010726, 0.0355215657, 0.0406214976, 0.0435834004, 0.0470045493, 0.0385664352, 0.0472325023, 0.0444419170, 0.0400665491, 0.0498178897, 0.0442400491, 0.0416440262, 0.0418480689, 0.0398616655, 0.0430931637, 0.0479540357, 0.0450493115, 0.0467705075, 0.0450768679, 0.0403235576, 0.0405796384, 0.0428518613, 0.0422108033, 0.0456190872, 0.0439781648, 0.0487045609, 0.0397670898, 0.0439254764, 0.0461722983, 0.0515924998, 0.0416588820, 0.0414975201, 0.0502438732, 0.0410323452, 0.0459847647, 0.0456258864, 0.0467261313, 0.0501880832, 0.0467273956, 0.0462085729, 0.0472110286, 0.0393934556, 0.0454877806, 0.0451223124, 0.0439503863, 0.0429342459, 0.0461093298, 0.0489114071, 0.0466530381, 0.0434444633, 0.0432470956, 0.0459068608, 0.0475079041, 0.0479623946, 0.0423448524, 0.0534109608, 0.0446951776, 0.0425815377, 0.0472747606, 0.0451700479, 0.0428634603, 0.0462891944, 0.0494328493, 0.0474882087, 0.0499571479, 0.0419461574, 0.0447534997, 0.0440009996, 0.0473322149, 0.0421812393, 0.0435714409, 0.0465693343, 0.0412020819, 0.0425745278, 0.0419824251, 0.0451248086, 0.0424342863, 0.0471111147, 0.0496996569, 0.0386218618, 0.0483678280, 0.0427052642, 0.0467335197, 0.0456933317, 0.0407545054, 0.0428280180, 0.0408489372, 0.0454104982, 0.0481908207, 0.0438358393, 0.0385052171, 0.0443490148, 0.0470035729, 0.0469784277, 0.0450578868, 0.0426215533, 0.0422878879, 0.0427978847, 0.0438805277, 0.0432833188, 0.0445551672, 0.0456565362, 0.0400851990, 0.0464596644, 0.0433879616, 0.0440521340, 0.0472391843, 0.0468134606, 0.0432352171, 0.0514054190, 0.0477084712, 0.0396379406, 0.0456425763, 0.0457500615, 0.0478434809, 0.0455024170, 0.0455004090, 0.0438530489, 0.0413289386, 0.0448764883, 0.0404208907, 0.0488761174, 0.0442638442, 0.0460552694, 0.0417968666, 0.0468234801, 0.0442716688, 0.0413565796, 0.0444732417, 0.0440898540, 0.0386603884, 0.0512978341, 0.0456067828, 0.0427035699, 0.0420464797, 0.0484629199, 0.0418745123, 0.0433378063, 0.0411936959, 0.0409332114, 0.0420151344, 0.0470519135, 0.0476174448, 0.0444958502, 0.0429532166, 0.0407580479, 0.0435016758, 0.0443576213, 0.0475781317, 0.0393657397, 0.0462344019, 0.0406485076, 0.0459636989, 0.0482741197, 0.0500332121, 0.0413240955, 0.0401818389, 0.0442769702, 0.0458078363, 0.0474676202, 0.0425250866, 0.0432738449, 0.0434355898, 0.0474600315, 0.0438535041, 0.0459913453, 0.0416143374, 0.0460806005, 0.0368005508, 0.0395121384, 0.0472623878, 0.0464689185, 0.0425102962, 0.0512446643, 0.0462737897, 0.0443148232, 0.0471696359, 0.0380973646, 0.0436201858, 0.0430784754, 0.0439574016, 0.0432222307, 0.0452884733, 0.0450533858, 0.0437313707, 0.0457048425, 0.0422273840, 0.0364609671, 0.0400449949, 0.0497254131, 0.0469534072, 0.0459863250, 0.0422778454, 0.0426151771, 0.0475423906, 0.0477401610, 0.0459846678, 0.0390511482, 0.0442722637, 0.0458054106, 0.0428708746, 0.0461712771, 0.0432618223, 0.0419045248, 0.0444196238, 0.0456363745, 0.0468995035, 0.0408984423, 0.0508127809, 0.0459799341, 0.0430328514, 0.0465827798, 0.0416158731, 0.0458712027, 0.0478040590, 0.0422139494, 0.0479701237, 0.0448880992, 0.0479679927, 0.0417961757, 0.0464013377,
0.0385243136, 0.0423007766, 0.0467074991, 0.0447892059, 0.0409665179, 0.0445811045, 0.0470848135, 0.0443024787, 0.0397446724, 0.0433068881, 0.0517339835, 0.0462017336, 0.0455606041, 0.0472051930, 0.0508521692, 0.0493632363, 0.0445423642, 0.0488209453, 0.0477119073, 0.0409958372, 0.0453962934, 0.0442043931, 0.0412188450, 0.0443873150, 0.0429597760, 0.0478171680, 0.0397152617, 0.0451271643, 0.0444629501, 0.0400444160, 0.0447988813, 0.0404119483, 0.0419218662, 0.0411116509, 0.0438202941, 0.0435572580, 0.0422580429, 0.0474320926, 0.0437850686, 0.0472176449, 0.0465718403, 0.0484860490, 0.0516804395, 0.0495733776, 0.0417042067, 0.0464740464, 0.0463601506, 0.0446442820, 0.0453487897, 0.0457041726, 0.0424084565, 0.0429766089, 0.0416657030, 0.0457907740, 0.0409503790, 0.0437512246, 0.0448202871, 0.0433855864, 0.0463362318, 0.0444422820, 0.0499028754, 0.0414335753, 0.0467306879, 0.0420646602, 0.0459093535, 0.0534056069, 0.0432605373, 0.0397915621, 0.0398695106, 0.0463127755, 0.0458955003, 0.0472525774, 0.0427114775, 0.0482497202, 0.0435529090, 0.0441176161, 0.0447136807, 0.0434899835, 0.0454968213, 0.0499376173, 0.0408098808, 0.0470413739, 0.0454530476, 0.0453446117, 0.0425645974, 0.0454178057, 0.0461366768, 0.0490291725, 0.0428624026, 0.0422983280, 0.0431454228, 0.0430222888, 0.0443439588, 0.0451224205, 0.0461949702, 0.0468945414, 0.0418024676, 0.0479534095, 0.0476875032, 0.0465623495, 0.0452691189, 0.0417160895, 0.0465506861, 0.0453280185, 0.0405296532, 0.0424733480, 0.0426569651, 0.0429293153, 0.0402043958, 0.0502562486, 0.0432207448, 0.0436047325, 0.0446937885, 0.0449971351, 0.0407042709, 0.0482809961, 0.0464824510, 0.0449933215, 0.0481167521, 0.0450947835, 0.0401256981, 0.0435383833, 0.0413568700, 0.0408951595, 0.0470189691, 0.0498184492, 0.0445232494, 0.0401293688, 0.0483243288, 0.0441458902, 0.0447345011, 0.0467586848, 0.0450101942, 0.0431761710, 0.0421874512, 0.0430332011, 0.0410765638, 0.0495292068, 0.0450962730, 0.0443434607, 0.0495437566, 0.0431240543, 0.0433584381, 0.0442090481, 0.0458189728, 0.0481548257, 0.0448615757, 0.0449263129, 0.0388349603, 0.0396614381, 0.0377649745, 0.0452379051, 0.0501995611, 0.0467505056, 0.0443627512, 0.0468372899, 0.0466199417, 0.0468921794, 0.0463387382, 0.0486956992, 0.0430408619, 0.0428758018, 0.0407966022, 0.0473178670, 0.0460632480, 0.0432186199, 0.0372435615, 0.0413304721, 0.0460981280, 0.0459454594, 0.0428500814, 0.0431672319, 0.0415400332, 0.0440274999, 0.0491958105, 0.0432928628, 0.0422572617, 0.0409147734, 0.0485845001, 0.0425080878, 0.0496401158, 0.0478275391, 0.0419048581, 0.0454532196, 0.0451911305, 0.0418947861, 0.0458610689, 0.0398327450, 0.0437131145, 0.0442147894, 0.0448991118, 0.0456487824, 0.0439557482, 0.0436311183, 0.0474513218, 0.0477400735, 0.0454708454, 0.0437252664, 0.0485216345, 0.0388007530, 0.0426837589, 0.0490116647, 0.0489168180, 0.0427928835, 0.0487548890, 0.0406265118, 0.0426556870, 0.0452826058, 0.0460122246, 0.0456357238, 0.0478611570, 0.0432341025, 0.0455112499, 0.0481232371, 0.0488332482, 0.0438626637, 0.0431496595, 0.0448964730, 0.0446672567, 0.0477499667, 0.0456940156, 0.0472585455, 0.0466457367, 0.0420420651, 0.0516605473, 0.0400514425, 0.0468697383, 0.0425252678, 0.0414423871, 0.0418201062, 0.0433899816, 0.0466428306, 0.0456264474, 0.0442617512, 0.0432283716, 0.0413839098, 0.0413602884, 0.0459591157, 0.0482617049, 0.0444450002, 0.0468406450, 0.0438595597, 0.0403181647, 0.0434331567, 0.0420806305, 0.0423684465, 0.0456064497, 0.0410769126, 0.0434588066, 0.0489372916)
# to run the full simulation uncomment the following line to fit the model for every dataset and not just for the first dataset
#for (i_rep in seq_len(n_rep)) {
for (i_rep in seq_len(1)) {
df = data[[i_rep]]
obj_dml_data = double_ml_data_from_data_frame(df, y_col = "y", d_cols = "d")
obj_dml_plr = DoubleMLPLR$new(obj_dml_data,
ml_l, ml_m, ml_g,
n_folds=2,
score='IV-type')
obj_dml_plr$fit()
this_theta = obj_dml_plr$coef
this_se = obj_dml_plr$se
print(abs(theta_dml[i_rep] - this_theta))
print(abs(se_dml[i_rep] - this_se))
theta_dml[i_rep] = this_theta
se_dml[i_rep] = this_se
}
g_dml = ggplot(data.frame(theta_rescaled=(theta_dml - alpha)/se_dml), aes(x = theta_rescaled)) +
geom_histogram(aes(y=after_stat(density), x=theta_rescaled, colour = "Double ML with cross-fitting", fill="Double ML with cross-fitting"),
bins = 30, alpha = 0.3) +
geom_vline(aes(xintercept = 0), col = "black") +
suppressWarnings(geom_function(fun = dnorm, aes(colour = "N(0, 1)", fill="N(0, 1)"))) +
scale_color_manual(name='',
breaks=c("Double ML with cross-fitting", "N(0, 1)"),
values=c("Double ML with cross-fitting"="dark green", "N(0, 1)"='black')) +
scale_fill_manual(name='',
breaks=c("Double ML with cross-fitting", "N(0, 1)"),
values=c("Double ML with cross-fitting"="dark green", "N(0, 1)"=NA)) +
xlim(c(-6.0, 6.0)) + xlab("") + ylab("") + theme_minimal()
g_dml
d
2.315448e-11
d
2.115916e-11
1.5. Double/debiased machine learning#
To illustrate the benefits of the auxiliary prediction step in the DML framework we write the error as
\[\sqrt{n}(\check{\theta}_0 - \theta_0) = a^* + b^* + c^*\]
Chernozhukov et al. (2018) argues that:
The first term
\[a^* := (EV^2)^{-1} \frac{1}{\sqrt{n}} \sum_{i\in I} V_i \zeta_i\]
will be asymptotically normally distributed.
The second term
\[b^* := (EV^2)^{-1} \frac{1}{\sqrt{n}} \sum_{i\in I} (\hat{m}(X_i) - m(X_i)) (\hat{g}_0(X_i) - g_0(X_i))\]
vanishes asymptotically for many data generating processes.
The third term \(c^*\) vanishes in probability if sample splitting is applied.
In [65]: plt.figure(constrained_layout=True);
In [66]: ax = sns.histplot((theta_nonorth - alpha)/se_nonorth,
....: color=face_colors[0], edgecolor = edge_colors[0],
....: stat='density', bins=30, label='Non-orthogonal ML');
....:
In [67]: sns.histplot((theta_orth_nosplit - alpha)/se_orth_nosplit,
....: color=face_colors[1], edgecolor = edge_colors[1],
....: stat='density', bins=30, label='Double ML (no sample splitting)');
....:
In [68]: sns.histplot((theta_dml - alpha)/se_dml,
....: color=face_colors[2], edgecolor = edge_colors[2],
....: stat='density', bins=30, label='Double ML with cross-fitting');
....:
In [69]: ax.axvline(0., color='k');
In [70]: xx = np.arange(-5, +5, 0.001)
In [71]: yy = stats.norm.pdf(xx)
In [72]: ax.plot(xx, yy, color='k', label='$\\mathcal{N}(0, 1)$');
In [73]: ax.legend(loc='upper right', bbox_to_anchor=(1.2, 1.0));
In [74]: ax.set_xlim([-6., 6.]);
In [75]: ax.set_xlabel('$(\hat{\\theta}_0 - \\theta_0)/\hat{\sigma}$');
g_all = ggplot(data.frame(t_nonorth=(theta_nonorth - alpha)/se_nonorth,
t_orth_nosplit=(theta_orth_nosplit - alpha)/se_orth_nosplit,
t_dml=(theta_dml - alpha)/se_dml)) +
geom_histogram(aes(x = t_nonorth, y=after_stat(density), colour = "Non-orthogonal ML", fill="Non-orthogonal ML"),
bins = 30, alpha = 0.3) +
geom_histogram(aes(x = t_orth_nosplit, y=after_stat(density), colour = "Double ML (no sample splitting)", fill="Double ML (no sample splitting)"),
bins = 30, alpha = 0.3) +
geom_histogram(aes(x = t_dml, y=after_stat(density), colour = "Double ML with cross-fitting", fill="Double ML with cross-fitting"),
bins = 30, alpha = 0.3) +
geom_vline(aes(xintercept = 0), col = "black") +
suppressWarnings(geom_function(fun = dnorm, aes(colour = "N(0, 1)", fill="N(0, 1)"))) +
scale_color_manual(name='',
breaks=c("Non-orthogonal ML", "Double ML (no sample splitting)", "Double ML with cross-fitting", "N(0, 1)"),
values=c("Non-orthogonal ML"="dark blue",
"Double ML (no sample splitting)"="dark orange",
"Double ML with cross-fitting"="dark green",
"N(0, 1)"='black')) +
scale_fill_manual(name='',
breaks=c("Non-orthogonal ML", "Double ML (no sample splitting)", "Double ML with cross-fitting", "N(0, 1)"),
values=c("Non-orthogonal ML"="dark blue",
"Double ML (no sample splitting)"="dark orange",
"Double ML with cross-fitting"="dark green",
"N(0, 1)"=NA)) +
xlim(c(-6.0, 6.0)) + xlab("") + ylab("") + theme_minimal()
g_all
1.6. Partialling out score#
Another debiased estimator, based on the partialling-out approach of Robinson(1988), is
\[\check{\theta}_0 = \left(\frac{1}{n} \sum_{i\in I} \hat{V}_i \hat{V}_i \right)^{-1} \frac{1}{n} \sum_{i\in I} \hat{V}_i (Y_i - \hat{\ell}_0(X_i)),\]
with \(\ell_0(X_i) = E(Y|X)\).
All nuisance parameters for the estimator with score='partialling out' are conditional mean functions, which can be
directly estimated using ML methods. This is a minor advantage over the estimator with score='IV-type'.
In the following, we repeat the above analysis with score='partialling out'. In a first part of the analysis, we
estimate \(\theta_0\) without sample splitting. Again we observe a bias from overfitting.
In [76]: import numpy as np
In [77]: np.random.seed(4444)
# to speed up the illustration we hard-code the simulation results
In [78]: theta_orth_po_nosplit = np.array([0.44689745, 0.41560548, 0.39147078, 0.40061244, 0.41388613, 0.40812013, 0.44497593, 0.38585885, 0.44117726, 0.49159354, 0.42731988, 0.43751013, 0.45444596, 0.33483986, 0.39266598, 0.38949641, 0.41844929, 0.37425161, 0.44119905, 0.40405533, 0.38900643, 0.36211315, 0.37546086, 0.41105622, 0.39496921, 0.37376606, 0.42015518, 0.39926076, 0.38283516, 0.42610636, 0.41467175, 0.47398591, 0.33381323, 0.35597262, 0.40394593, 0.37240773, 0.38373405, 0.45020658, 0.43253392, 0.42079326, 0.37306779, 0.37176962, 0.41359098, 0.32905667, 0.43924895, 0.40621988, 0.4248174 , 0.4188524 , 0.42545551, 0.48109574, 0.39236691, 0.41109807, 0.43830774, 0.43131476, 0.46297181, 0.36525 , 0.42785268, 0.45948104, 0.37387788, 0.39193983, 0.41964355, 0.45105777, 0.45068412, 0.41531941, 0.45774125, 0.37707203, 0.37312002, 0.41251921, 0.46760764, 0.45245259, 0.38508787, 0.399372 , 0.39485543, 0.42089349, 0.42689592, 0.43747816, 0.43590912, 0.43153398, 0.40612324, 0.4246974 , 0.407134 , 0.40267842, 0.39337628, 0.39783404, 0.36343842, 0.416114 , 0.43468455, 0.42479065, 0.44861003, 0.4284088 , 0.33906023, 0.36972258, 0.46021926, 0.40991793, 0.43182578, 0.4058362 , 0.41099645, 0.40465821, 0.5049942 , 0.44719956, 0.37759362, 0.46968968, 0.39354983, 0.43817316, 0.37478456, 0.47752947, 0.4158715 , 0.42853136, 0.34761861, 0.40501635, 0.35281687, 0.40102032, 0.36820323, 0.37565568, 0.38721431, 0.38768897, 0.40365893, 0.40833054, 0.38796741, 0.38696343, 0.38890053, 0.47678995, 0.40821229, 0.39226213, 0.32782232, 0.46068932, 0.37661274, 0.39197202, 0.38255031, 0.34728656, 0.43755796, 0.36722004, 0.40465598, 0.39280923, 0.38506902, 0.45724146, 0.45084514, 0.44738245, 0.37119941, 0.40383196, 0.38649858, 0.37208117, 0.47703402, 0.41352775, 0.47243667, 0.34997923, 0.34107974, 0.4503945 , 0.37347764, 0.43770291, 0.45765222, 0.41591249, 0.37818029, 0.40079684, 0.40742744, 0.51493047, 0.38143551, 0.38922804, 0.36482121, 0.48946817, 0.40386025, 0.3789189 , 0.43472849, 0.4482401 , 0.41958351, 0.3874319 , 0.46551354, 0.40688943, 0.42076722, 0.42261826, 0.42613403, 0.3982167 , 0.40560043, 0.38510455, 0.41363841, 0.4153734 , 0.42633414, 0.48195624, 0.4064136 , 0.39629634, 0.37622216, 0.44147214, 0.42028914, 0.35511551, 0.32997176, 0.37204305, 0.34571186, 0.38392038, 0.454553 , 0.42857666, 0.37738172, 0.38752155, 0.39888209, 0.33003166, 0.39749669, 0.40623056, 0.38995359, 0.43574707, 0.39400018, 0.38395351, 0.42308851, 0.45226606, 0.39925256, 0.41512287, 0.38956918, 0.36903949, 0.39506798, 0.38304656, 0.45834436, 0.44365015, 0.3559809 , 0.49703991, 0.36755582, 0.32405813, 0.35703684, 0.45291725, 0.3862536 , 0.45430432, 0.43151156, 0.42199184, 0.45506641, 0.41894509, 0.40927734, 0.44103929, 0.41244317, 0.45798097, 0.41423993, 0.41234892, 0.49257489, 0.39316852, 0.45133683, 0.48647005, 0.52262288, 0.39411164, 0.4361213 , 0.47251784, 0.41942088, 0.38252744, 0.48580277, 0.41970908, 0.41279836, 0.4496232 , 0.39192501, 0.44793215, 0.39777389, 0.36491141, 0.42293433, 0.38311846, 0.34729731, 0.3467867 , 0.38890129, 0.41369155, 0.43718281, 0.42255496, 0.42606418, 0.34113915, 0.40785283, 0.4266964 , 0.48600678, 0.38993538, 0.37674472, 0.36469012, 0.41382762, 0.3910651 , 0.36058839, 0.4385373 , 0.3853683 , 0.45285794, 0.43859701, 0.35940378, 0.35532133, 0.36747323, 0.31895662, 0.4765736 , 0.44085975, 0.43925062, 0.43305578, 0.38225082, 0.40558291, 0.42308383, 0.44251497, 0.38819576, 0.48809178, 0.39663077, 0.43415774, 0.44212896, 0.46864931, 0.41112727, 0.46702856, 0.35242806, 0.43235724, 0.43946028, 0.41192496, 0.3837241 , 0.38252215, 0.40869806, 0.51874593, 0.3617031 , 0.45676473, 0.34502171, 0.42263263, 0.46617327, 0.44260223, 0.38736024, 0.38962609, 0.41169203, 0.42714373, 0.35953918, 0.44465238, 0.41616539, 0.40898319, 0.35428957, 0.39896774, 0.46760866, 0.35534866, 0.39458018, 0.3010404 , 0.4275348 , 0.4130533 , 0.42218207, 0.50383664, 0.35943386, 0.46271252, 0.43461421, 0.44607091, 0.38827735, 0.4151206 , 0.39796933, 0.38299143, 0.37773728, 0.39394795, 0.46817041, 0.37619518, 0.37979198, 0.39579081, 0.38947461, 0.49225938, 0.32224241, 0.45937019, 0.40716829, 0.41395486, 0.35955957, 0.41686161, 0.37045689, 0.35787435, 0.46819524, 0.42714861, 0.4317218 , 0.41438206, 0.37601954, 0.4035717 , 0.38687642, 0.46099666, 0.39478873, 0.47577063, 0.46146113, 0.43162587, 0.37506128, 0.367823 , 0.3814589 , 0.43190693, 0.37833206, 0.44924794, 0.43123303, 0.43457722, 0.45822097, 0.42546258, 0.44599857, 0.43560714, 0.4018572 , 0.35588031, 0.39884516, 0.38828861, 0.3938863 , 0.36650029, 0.38365921, 0.45357156, 0.40473875, 0.38083154, 0.34806574, 0.45759315, 0.392613 , 0.44443932, 0.3859657 , 0.42588092, 0.37958914, 0.43936553, 0.40565913, 0.37050438, 0.38075554, 0.41480746, 0.41046031, 0.44809874, 0.40125569, 0.37153855, 0.34580786, 0.39599036, 0.40912106, 0.40286777, 0.4204081 , 0.40193548, 0.38921222, 0.36955462, 0.43778725, 0.38803946, 0.40488867, 0.39969558, 0.41182898, 0.41815582, 0.39436057, 0.40924522, 0.34340729, 0.36349859, 0.32419746, 0.38380303, 0.40018357, 0.45952972, 0.38707899, 0.35145758, 0.38427105, 0.39348254, 0.41127523, 0.4128339 , 0.40017793, 0.44410231, 0.40519121, 0.43557319, 0.39673401, 0.42723438, 0.46481697, 0.46732216, 0.35065925, 0.44945291, 0.38482921, 0.38124517, 0.40981892, 0.43832261, 0.43373603, 0.41552012, 0.38114908, 0.46441949, 0.43537922, 0.44091101, 0.41058189, 0.38814734, 0.37535326, 0.43135988, 0.4162225 , 0.41918978, 0.42145809, 0.39896498, 0.37387455, 0.43116882, 0.3833774 , 0.38335557, 0.40718161, 0.45818585, 0.39677352, 0.35869148, 0.43742879, 0.4388087 , 0.39325689, 0.40411463, 0.34076527, 0.4002913 , 0.38073684, 0.42822759, 0.41065555, 0.36612017, 0.43793757, 0.38045494, 0.40073521, 0.43634931, 0.37228829, 0.38076058, 0.44028763, 0.40618622, 0.39886155, 0.39606878, 0.40121651, 0.42329785, 0.38391397, 0.48701045, 0.40852587, 0.38811837, 0.33700027, 0.43337934, 0.47638898, 0.45862387, 0.43101757, 0.39399701, 0.37816887, 0.34243431, 0.40166362, 0.3677345 , 0.48000176, 0.46025401, 0.42700476, 0.34592185, 0.41396936, 0.3651648 , 0.44434418, 0.42944586, 0.37357513, 0.45325424, 0.35135126, 0.34060331, 0.44891731, 0.45647405, 0.40584386, 0.40792945, 0.37417043, 0.40859987, 0.48744066, 0.45568601, 0.40720023, 0.38534174, 0.44418675, 0.40091154, 0.36556 , 0.40113015, 0.37258915, 0.44318192, 0.38091808, 0.40725597, 0.44666564, 0.42190333, 0.44481681, 0.41919932, 0.37032402, 0.48364551, 0.38748608, 0.44018272, 0.42203535, 0.37242747, 0.38219231, 0.46776887, 0.43337044, 0.42124322, 0.37882104, 0.44778111, 0.41846436, 0.37811701, 0.37155185, 0.39007756, 0.36378231, 0.42661165, 0.43822261, 0.40546708, 0.41911667, 0.4260626 , 0.42256451, 0.44192597, 0.47842638, 0.39611512, 0.44491116, 0.40365296, 0.39359214, 0.40003074, 0.34958918, 0.34758949, 0.45646971, 0.38078347, 0.42067093, 0.39756817, 0.41426262, 0.45689868, 0.42263988, 0.48026677, 0.44894483, 0.35304089, 0.40763657, 0.35102165, 0.40311662, 0.44168856, 0.42674332, 0.40034222, 0.37597472, 0.37988866, 0.49597058, 0.36669922, 0.50231546, 0.47759904, 0.41977028, 0.3902988 , 0.35337019, 0.29725319, 0.37085959, 0.40683181, 0.33962811, 0.3775262 , 0.40378711, 0.37522767, 0.43806618, 0.37608875, 0.38363954, 0.4213465 , 0.37685846, 0.43864085, 0.42072523, 0.41303825, 0.44085664, 0.44935243, 0.3955478 , 0.41700251, 0.42745432, 0.40076353, 0.44415625, 0.40592704, 0.42403648, 0.43164961, 0.43741185, 0.4556666 , 0.35447542, 0.36179497, 0.37609757, 0.41936549, 0.45255993, 0.39765959, 0.41473308, 0.45325442, 0.43660725, 0.41725395, 0.42584886, 0.35245624, 0.40770218, 0.34849235, 0.41924673, 0.42967222, 0.4688549 , 0.41986642, 0.5394729 , 0.41234063, 0.38889672, 0.4133863 , 0.48377739, 0.3985232 , 0.44467283, 0.40187751, 0.41207328, 0.38259148, 0.49482289, 0.40994346, 0.43842771, 0.46299092, 0.35792853, 0.43628973, 0.38300048, 0.4402988 , 0.37719632, 0.447099 , 0.46458036, 0.34070484, 0.38668328, 0.41877418, 0.39920467, 0.41402592, 0.46197827, 0.34998859, 0.44292424, 0.39554279, 0.45806025, 0.44549445, 0.41144303, 0.39757833, 0.44460934, 0.39282869, 0.40982084, 0.3594684 , 0.38035559, 0.46361533, 0.39866406, 0.45390917, 0.37363494, 0.42751835, 0.36809349, 0.3981875 , 0.38283569, 0.33765472, 0.445369 , 0.46432175, 0.3828271 , 0.40032819, 0.43784922, 0.40907506, 0.50620526, 0.39825537, 0.40494115, 0.42097051, 0.39813231, 0.42767821, 0.43632554, 0.48342796, 0.37171266, 0.4231365 , 0.39791793, 0.4475876 , 0.43187838, 0.36713241, 0.39348117, 0.37435029, 0.44677536, 0.3560875 , 0.41850455, 0.38217649, 0.37817251, 0.44770328, 0.43056134, 0.38866372, 0.42418107, 0.35271324, 0.44943607, 0.41989644, 0.4117774 , 0.40604735, 0.40890907, 0.33590615, 0.46407173, 0.43129958, 0.36415985, 0.38422561, 0.43792973, 0.41323115, 0.4359505 , 0.43611463, 0.36418954, 0.39182405, 0.39411455, 0.41871595, 0.46333644, 0.402323 , 0.44001463, 0.40980919, 0.46406623, 0.43420349, 0.40457103, 0.38281922, 0.4200635 , 0.45682402, 0.42409818, 0.43678774, 0.39269752, 0.51152078, 0.41304056, 0.44903095, 0.3799896 , 0.44601481, 0.42465758, 0.3995831 , 0.38351877, 0.39567814, 0.49077009, 0.4166894 , 0.42924459, 0.43517778, 0.49714309, 0.34822792, 0.45457007, 0.45281455, 0.37328413, 0.39353403, 0.43454553, 0.44074761, 0.4115068 , 0.41406096, 0.42452023, 0.3787139 , 0.43965875, 0.49540995, 0.41198105, 0.35261123, 0.41107871, 0.38119491, 0.37193428, 0.38633685, 0.32842662, 0.40204225, 0.42665654, 0.40166691, 0.40939109, 0.41806659, 0.3609013 , 0.37036398, 0.36511387, 0.45810674, 0.38218874, 0.40843999, 0.39675316, 0.41089759, 0.45017994, 0.48048135, 0.40035762, 0.38592168, 0.43293865, 0.38960311, 0.49343706, 0.36434481, 0.4042647 , 0.4543836 , 0.37766804, 0.37252239, 0.38376966, 0.4007445 , 0.40013226, 0.40464522, 0.39352434, 0.40585953, 0.38774478, 0.43955992, 0.38933552, 0.47584388, 0.4760254 , 0.38309879, 0.4010732 , 0.46586454, 0.37250797, 0.41343097, 0.41691298, 0.42565652, 0.39245694, 0.43781405, 0.39532442, 0.46479819, 0.37892093, 0.4500263 , 0.42672327, 0.36790712, 0.41575054, 0.36628736, 0.35657307, 0.37455605, 0.42584967, 0.44626351, 0.42206467, 0.40746804, 0.49480752, 0.40585813, 0.39371666, 0.38483937, 0.32532322, 0.37839034, 0.44029757, 0.43886105, 0.44089243, 0.48483716, 0.38290252, 0.44274085, 0.4272902 , 0.40004465, 0.37967661, 0.29926117, 0.46973071, 0.39067255, 0.40449049, 0.36781328, 0.46677963, 0.37546023, 0.455158 , 0.3808844 , 0.41382398, 0.45051137, 0.4279048 , 0.47032358, 0.34036995, 0.43181476, 0.44090817, 0.41262752, 0.42174053, 0.43467423, 0.43022294, 0.37072532, 0.43068552, 0.42300783, 0.38696066, 0.36009265, 0.34119649, 0.39879691, 0.4365699 , 0.4120221 , 0.47382627, 0.42169809, 0.40599731, 0.44671891, 0.40248377, 0.44969546, 0.35117912, 0.44837504, 0.4161964 , 0.37582456, 0.35299654, 0.447547 , 0.39787134, 0.3803233 , 0.45412813, 0.36251435, 0.45113205, 0.42711722, 0.35638102, 0.42258747, 0.45780058, 0.35630224, 0.39713166, 0.37472678, 0.37831073, 0.43158708, 0.36181206, 0.45090422, 0.32068871, 0.43781083, 0.44439732, 0.4035753 , 0.40448936, 0.37632135, 0.35958719, 0.46351404, 0.40932618, 0.4341114 , 0.38818929, 0.38353028, 0.45386913, 0.50000539, 0.42122936, 0.40654102, 0.36816651, 0.44763332, 0.41004633, 0.39898167, 0.44444691, 0.36822278, 0.4553816 , 0.3905132 , 0.36842322, 0.48547593, 0.44972635, 0.41782122, 0.45257978, 0.45659295, 0.42285171, 0.39763381, 0.41796586, 0.39381587, 0.46934179, 0.42922762, 0.38053444, 0.40544273, 0.36006186, 0.39863556, 0.3719792 , 0.39821203, 0.41790859, 0.49867499, 0.42365679, 0.395948 , 0.43571833, 0.34534889, 0.41997742, 0.41201298, 0.37578671, 0.41356327, 0.39392439, 0.44224059, 0.38535319, 0.41808058, 0.33271179, 0.46387916, 0.44893328, 0.40814264, 0.39180342, 0.43969064, 0.35560315, 0.4266996 , 0.38941229, 0.35687593, 0.40373073, 0.34140252, 0.40307342, 0.42094028, 0.39607329, 0.37907175, 0.41389525, 0.39529958, 0.45090839, 0.40201934, 0.42203752, 0.39398395, 0.39915548, 0.49363392, 0.4614152 , 0.47190431, 0.42186278, 0.40138577, 0.40398946, 0.31354679, 0.4615471 , 0.44052724, 0.40421449, 0.43339062, 0.42131888, 0.39299912, 0.43951129])
In [79]: se_orth_po_nosplit = np.array([0.03474502, 0.0373666 , 0.03254624, 0.035711 , 0.03857083, 0.03393274, 0.0382176 , 0.0390209 , 0.03947079, 0.03632686, 0.0365749 , 0.03596462, 0.03283355, 0.03305384, 0.03462809, 0.03429812, 0.04047603, 0.03751935, 0.03350184, 0.03692805, 0.03675086, 0.04425244, 0.03688826, 0.03667372, 0.04111085, 0.03012514, 0.03567342, 0.03630253, 0.03869279, 0.03591116, 0.03291957, 0.03330094, 0.03466816, 0.03615811, 0.03925414, 0.03463743, 0.0403616 , 0.03672733, 0.03673892, 0.03729507, 0.03656263, 0.0361764 , 0.03740284, 0.03674824, 0.03385807, 0.03696247, 0.03624528, 0.03540566, 0.03432279, 0.03781447, 0.03544402, 0.03242579, 0.03509169, 0.03933641, 0.03515456, 0.04092438, 0.03431629, 0.03421128, 0.03484018, 0.03407852, 0.03809609, 0.03848484, 0.0382654 , 0.03495158, 0.03619387, 0.03773587, 0.03797528, 0.03389219, 0.03663497, 0.03785326, 0.03551873, 0.03299111, 0.03513855, 0.03886106, 0.03445817, 0.03464812, 0.0355295 , 0.03802371, 0.0352706 , 0.03641776, 0.03786271, 0.03845134, 0.03586474, 0.03619399, 0.03436212, 0.03985137, 0.03525877, 0.0385634 , 0.04007947, 0.03799021, 0.03552911, 0.03579554, 0.03336541, 0.04141928, 0.03515444, 0.03668795, 0.03444747, 0.03644692, 0.03806484, 0.03519977, 0.03743296, 0.03828006, 0.03827395, 0.03657322, 0.03633241, 0.03422569, 0.03600341, 0.03973577, 0.03454769, 0.03568129, 0.0396222 , 0.03521892, 0.03596001, 0.03624428, 0.03815015, 0.03995477, 0.03826805, 0.03525698, 0.03742405, 0.03782724, 0.03844241, 0.03742421, 0.03438375, 0.03364354, 0.03845537, 0.03885686, 0.03643648, 0.03452413, 0.03600843, 0.0369478 , 0.03951252, 0.04147925, 0.03912529, 0.03211512, 0.03828445, 0.04122697, 0.03767405, 0.03814904, 0.03677246, 0.0348576 , 0.03664641, 0.03738636, 0.03694522, 0.03921987, 0.03652723, 0.03714045, 0.03334889, 0.03591158, 0.03467354, 0.03291863, 0.03650219, 0.03963865, 0.03836493, 0.03835041, 0.03321675, 0.03321789, 0.03548968, 0.03402733, 0.03633222, 0.0378797 , 0.03689956, 0.03718884, 0.03861813, 0.03437033, 0.0396973 , 0.03825877, 0.03611252, 0.03611327, 0.03678618, 0.03914326, 0.03612262, 0.03907715, 0.03660782, 0.03543502, 0.034366 , 0.03436277, 0.03323214, 0.03867857, 0.03855516, 0.03736977, 0.03163678, 0.0357661 , 0.03607132, 0.03876504, 0.04041912, 0.03634519, 0.03088368, 0.04378558, 0.03802604, 0.03509891, 0.03971062, 0.03689964, 0.03480082, 0.03983675, 0.03928926, 0.03992703, 0.04197775, 0.03720813, 0.04007649, 0.03307689, 0.03943535, 0.03518234, 0.03879254, 0.03356806, 0.03555913, 0.03467602, 0.03840389, 0.03858417, 0.03910251, 0.03937124, 0.03695466, 0.03822676, 0.04013834, 0.03398154, 0.03628905, 0.03605391, 0.03737559, 0.03754431, 0.03731786, 0.03372294, 0.03382883<