In statistics, an adaptive estimator is an estimator in a parametric or semiparametric model with nuisance parameters such that the presence of these nuisance parameters does not affect efficiency of estimation.

Definition

Formally, let parameter θ in a parametric model consists of two parts: the parameter of interest ν ∈ N ⊆ Rk, and the nuisance parameter η ∈ H ⊆ Rm. Thus θ = (ν,η) ∈ N×H ⊆ Rk+m. Then we will say that ν ^ n {\displaystyle \scriptstyle {\hat {\nu }}_{n}} is an adaptive estimator of ν in the presence of η if this estimator is regular, and efficient for each of the submodels

P ν ( η 0 ) = { P θ : ν ∈ N , η = η 0 } . {\displaystyle {\mathcal {P}}_{\nu }(\eta _{0})={\big \{}P_{\theta }:\nu \in N,\,\eta =\eta _{0}{\big \}}.}

Adaptive estimator estimates the parameter of interest equally well regardless whether the value of the nuisance parameter is known or not.

The necessary condition for a regular parametric model to have an adaptive estimator is that

I ν η ( θ ) = E ⁡ [ z ν z η ′ ] = 0 for all θ , {\displaystyle I_{\nu \eta }(\theta )=\operatorname {E} [\,z_{\nu }z_{\eta }'\,]=0\quad {\text{for all }}\theta ,}

where zν and zη are components of the score function corresponding to parameters ν and η respectively, and thus Iνη is the top-right k×m block of the Fisher information matrix I(θ).

Example

Suppose P {\displaystyle \scriptstyle {\mathcal {P}}} is the normal location-scale family:

P = { f θ ( x ) = 1 2 π σ e − 1 2 σ 2 ( x − μ ) 2 | μ ∈ R , σ > 0 } . {\displaystyle {\mathcal {P}}={\Big \{}\ f_{\theta }(x)={\tfrac {1}{{\sqrt {2\pi }}\sigma }}e^{-{\frac {1}{2\sigma ^{2}}}(x-\mu )^{2}}\ {\Big |}\ \mu \in \mathbb {R} ,\sigma >0\ {\Big \}}.}

Then the usual estimator μ ^ = x ¯ {\displaystyle {\hat {\mu }}\,=\,{\bar {x}}} is adaptive: we can estimate the mean equally well whether we know the variance or not.

Notes

Basic references

  • Bickel, Peter J.; Chris A.J. Klaassen; Ya’acov Ritov; Jon A. Wellner (1998). Efficient and adaptive estimation for semiparametric models. Springer: New York. ISBN978-0-387-98473-5.{{cite book}}: CS1 maint: publisher location (link)

Other useful references