Jackknife resampling

In statistics, the jackknife is a resampling technique especially useful for variance and bias estimation. The jackknife pre-dates other common resampling methods such as the bootstrap. The jackknife estimator of a parameter is found by systematically leaving out each observation from a dataset and calculating the estimate and then finding the average of these calculations. Given a sample of size n {displaystyle n} , the jackknife estimate is found by aggregating the estimates of each ( n − 1 ) {displaystyle (n-1)} -sized sub-sample. In statistics, the jackknife is a resampling technique especially useful for variance and bias estimation. The jackknife pre-dates other common resampling methods such as the bootstrap. The jackknife estimator of a parameter is found by systematically leaving out each observation from a dataset and calculating the estimate and then finding the average of these calculations. Given a sample of size n {displaystyle n} , the jackknife estimate is found by aggregating the estimates of each ( n − 1 ) {displaystyle (n-1)} -sized sub-sample. The jackknife technique was developed by Maurice Quenouille (1924-1973) from 1949, and refined in 1956. John Tukey expanded on the technique in 1958 and proposed the name 'jackknife' since, like a physical jack-knife (a compact folding knife), it is a rough-and-ready tool that can improvise a solution for a variety of problems even though specific problems may be more efficiently solved with a purpose-designed tool. The jackknife is a linear approximation of the bootstrap. The jackknife estimate of a parameter can be found by estimating the parameter for each subsample omitting the i-th observation. For example, if the parameter to be estimated is the population mean of x, we compute the mean x ¯ i {displaystyle {ar {x}}_{i}} for each subsample consisting of all but the i-th data point: These n estimates form an estimate of the distribution of the sample statistic if it were computed over a large number of samples. In particular, the mean of this sampling distribution is the average of these n estimates: A jackknife estimate of the variance of the estimator can be calculated from the variance of this distribution of x ¯ i {displaystyle {ar {x}}_{i}} : The jackknife technique can be used to estimate the bias of an estimator calculated over the entire sample. Say θ ^ {displaystyle {hat { heta }}} is the calculated estimator of the parameter of interest based on all n {displaystyle {n}} observations. Let where θ ^ ( i ) {displaystyle {hat { heta }}_{mathrm {(i)} }} is the estimate of interest based on the sample with the i-th observation removed, and θ ^ ( . ) {displaystyle {hat { heta }}_{mathrm {(.)} }} is the average of these 'leave-one-out' estimates.The jackknife estimate of the bias of θ ^ {displaystyle {hat { heta }}} is given by: and the resulting bias-corrected jackknife estimate of θ {displaystyle heta } is given by: