Nonlinear dimension reduction for regression with nearest neighbors

Quentin Paris (1)

1 IRMAR - Institut de Recherche Mathématique de Rennes

Let $(X,Y)$ be an $\mathcal X\times\mathbb R$ valued random variable, where $\mathcal X\subset \mathbb R^p$. We generalize to a nonlinear framework the sufficient dimension reduction approach, followed by Cadre and Dong (2010), for estimating the regression function $r(x)=\mathbb E(Y\vert X=x)$. We assume given a class $\mathcal H$ of functions $h:\mathcal X\rightarrow\mathbb R^p$ such that there exists $h\in\mathcal H$ with $$\mathbb E\left(Y\vert X\right)=\mathbb E\left(Y\vert h(X)\right).$$ In classical sufficient dimension reduction, $\mathcal H$ may be considered as a particular set of matrices. Here, $\mathcal H$ is considered to be a general and possibly nonparametric class of functions. In this context, we define the {\it reduced dimension} $d$ associated with $\mathcal H$ as the smallest $\ell$ such that there exists $h\in\mathcal H$ satisfying the former equality and such that $h(\mathcal X)$ spans a subspace of dimension $\ell$. Then, we construct an estimate $\hat r$ of $r$ that is proved to achieve the optimal rate of convergence as if the predictor $X$ where $d$-dimensional.

Domaines: Statistiques [math.ST]; Théorie [stat.TH]

Quentin Paris. Nonlinear dimension reduction for regression with nearest neighbors. 2013. ⟨hal-00785643v2⟩