De-sparsified lasso

De-sparsified lasso contributes to construct confidence intervals and statistical tests for single or low-dimensional components of a large parameter vector in high-dimensional model.

High-dimensional linear model

$Y=X{\beta }^0+ϵ$ with $n\times p$ design matrix $X=:[X_1,...,X_p]$ ($n\times p$ vectors $X_j$), $ϵ\sim N_n(0,{\sigma }_{ϵ}^{2}I)$ independent of $X$ and unknown regression $p\times 1$ vector ${\beta }^0$.

The usual method to find the parameter is by Lasso: ${\hat{\beta }}^n(\lambda )=\underset{\beta \in {ℝ}^p}{argmin}\frac{1}{2n}{\Vert Y-X\beta \Vert }_2^2+\lambda {\Vert \beta \Vert }_1$

The de-sparsified lasso is a method modified from the Lasso estimator which fulfills the Karush–Kuhn–Tucker conditions is as follows:

${\hat{\beta }}^n(\lambda ,M)={\hat{\beta }}^n(\lambda )+\frac{1}{n}M{X}^T(Y-X{\hat{\beta }}^n(\lambda ))$

where $M\in R^{p\times p}$ is an arbitrary matrix. The matrix $M$ is generated using a surrogate inverse covariance matrix.

Generalized linear model

Desparsifying $l_1$-norm penalized estimators and corresponding theory can also be applied to models with convex loss functions such as generalized linear models.

Consider the following $1\times p$vectors of covariables $x_i\in \chi \subset R^p$ and univariate responses $y_i\in Y\subset R$ for $i=1,...,n$

we have a loss function ${\rho }_{\beta }(y,x)=\rho (y,x\beta )(\beta \in R^p)$ which is assumed to be strictly convex function in $\beta \in R^p$

The $l_1$-norm regularized estimator is $\hat{\beta }=\underset{\beta }{argmin}(P_n{\rho }_{\beta }+\lambda {\Vert \beta \Vert }_1)$

Similarly, the Lasso for node wise regression with matrix input is defined as follows: Denote by $\hat{\mathrm{\Sigma }}$ a matrix which we want to approximately invert using nodewise lasso.

The de-sparsified $l_1$-norm regularized estimator is as follows: $\widehat{{\gamma }_j}:=\underset{\gamma \in R^{p-1}}{argmin}({\hat{\mathrm{\Sigma }}}_{j,j}-2{\hat{\mathrm{\Sigma }}}_{j,/j}\gamma +{\gamma }^T{\hat{\mathrm{\Sigma }}}_{/j,/j}\gamma +2{\lambda }_j{\Vert \gamma \Vert }_1$

where ${\hat{\mathrm{\Sigma }}}_{j,/j}$ denotes the $j$th row of $\hat{\mathrm{\Sigma }}$ without the diagonal element $(j,j)$, and ${\hat{\mathrm{\Sigma }}}_{/j,/j}$ is the sub matrix without the $j$th row and $j$th column.