InfluentialPoints.com
Biology, images, analysis, design...
Use/Abuse Principles How To Related
 "It has long been an axiom of mine that the little things are infinitely the most important" (Sherlock Holmes)

 

 

 

Jackknifing

Jackknife estimators

One of the problems with parametric models is they assume you are using a well-understood estimator upon a sample of a known population. Unfortunately, all too often, information regarding either your statistic's behaviour, or the population's frequency distribution, is hazy - at very best. None of which makes it very easy for non-statisticians to assess how reliable their estimates are.

In that situation what is needed is a reasonably-straightforward, albeit rough-and-ready, approximation of your statistic's bias and standard error. In other words a tool which, although not the best possible in any given situation, is better than nothing - and can be applied to a range of problems. An everyday example of this sort of item is known throughout the world as a 'Swiss Army knife' but, for reasons of their own, the Americans call it a Jack knife.

sa04.JPG from S6091 to S6093a.bmp

In this Unit (#3) we have shown how, by repeatedly calculating the same statistic from samples of a known population, you can estimate its bias and standard error and how these depend upon which population you sample. We also explore this in two hands-on sessions. The purpose of the jackknife technique is to reduce the bias of a statistic in its estimation of a population parameter, and obtain an estimate of the standard error of the statistic. It does this by repeatedly finding how that statistic differs when calculated from samples which are similar to the one you observed.

It has been used most frequently in biology for estimating sample statistics where estimation of standard errors are not straight forward, such as estimates of population size from mark-release-recapture data, population growth rates, indices of diversity and indices of species richness. Jackknife estimates require much less computation than resampling, and were heavily used in the past when less computing power was available. In contrast, jackknifing is not recommended for time-series data (unless special arrangements are made).

 

Partial estimates

    To avoid getting bogged-down by referring to any particular statistic, let us use Θ (pronounced 'theta', the Greek letter Q) is the statistic you are trying to estimate, and (theta hat) is your estimate of theta.

Jackknife statistics compare the difference between your original estimate, , calculated from n observations, with n statistics, -j, calculated from the same sample - from which one observation, the jth, is omitted. Somewhat predictably, an estimate calculated from part of a sample is known as a partial estimate.

If our statistic is an unbiased estimator, we can get an unbiased estimate from the mean of these partial estimates. More usefully, even if the statistic is a biased estimator, these n partial estimates can be used to obtain pseudo-values which will provide a relatively unbiased estimate of your sample statistic. Why this should be so may be less obvious.

 

Pseudo-values and bias

If we ignore their mathematical derivations, it turns out that, provided a partial estimate () can be viewed as some sort of mean and its errors are normal, its 'moments' can be expanded into series of terms looking something like this:
  1. E()   =   Θ + b1/n1 + b2/n2 + b3/n3...
  2. E(-Θ)2    =    v1/n1 + v2/n2 + v3/n3...

bi and vi are the biases and variances in estimating Θ, or its variance, from a sample of n observations - and E() - Θ is the overall bias.

In both cases, each term is equivalent to some constant times 1/ni so, for large samples, the (trailing) second order terms are much smaller than the (leading, i=1) first-order term. If we pool the higher order terms, the bias simplifies to E() - Θ = b1/n + O[1/n2]

  • In other words, you would expect the bias in an estimate based upon n observations ( ) to be approximately b1/n - and that, if the sample is sufficiently large, the order of error in this approximation depends upon 1/n2
  • From which it follows that, if your estimates are based upon n-1 observations ( -j ), the leading bias term should be b1/n-1
  • In order to estimate the bias we need to examine the difference between your estimate and your n partial estimates. However, because these estimates are based upon different numbers of observations, we need to weight them accordingly.

If your estimator were a simple mean this would be quite straightforward because n - [n-1]-j = Yj. Therefore nE[] - [n-1]E[-j] = E[Yj] = μY.

For more complicated functions we would not expect to obtain our original values, so j is referred to as a pseudo-value. Nevertheless, provided behaves as a mean, nE[] - [n-1]E[-j] = E[j] ≅ Θ and the bias cancels out.

The set of n pseudo-values is calculated using the function given below.

    Each pseudo-value = j = n - (n-1)-j
Where:
    j = their pseudo-value (the jth of n partial estimates, where  j = 1 to n)
    -j = a partial estimate lacking the jth of n observations
    = the estimated sample statistic using all your observations

For large samples, the mean of these pseudo-values ( = Σj/n ) is an approximately unbiased estimator of the sample statistic - and is therefore often used in preference to . If you are trying to find the bias of you could estimate it as b = -

The standard error of the jackknife estimator ( ) is commonly estimated from the sample variance of the pseudo-values ( var = Σ[j - ]2/[n-1] ) as = √[var/n]. More controversially, / is often assumed to be t-distributed with n-1 degrees of freedom - and this is used to fit confidence limits to the jackknife statistic. (We consider the principles and properties of 't' statistics in Unit 6 & 8

 

Assumptions

  1. It is assumed that the usual estimator of the parameter of interest is some function of n sample values so that you can obtain multiple estimates by omitting single values in turn. In particular it assumes the statistic behaves as some sort of mean - specifically, when calculated from large samples, it approaches normal. Jackknifing is moderately robust to departures from this latter assumption but this robustness is achieved at the expense of efficiency, and a tendency to overestimate its standard error. More importantly, jackknife estimators not cope well with statistics that do not have a smooth distribution function. These include some maximum likelihood statistics, and percentiles - such as the median or maximum.

  2. Whilst jackknifing makes no assumptions about your observations' parent population, it does assume that the pseudo-values can be treated as a random sample of independent estimates. If this is not the case, it can lead to underestimation of the standard error.

  3. The jackknife statistic is assumed to have a normal distribution. More importantly, jackknifing is particularly sensitive to outliers, contaminated, or skewed distributions - which can produce wildly inaccurate variance estimates. This arises because, although they are assumed to be independent, pseudo-values are correlated - biasing the estimated variance up or down to an unpredictable extent - which can only be determined empirically by jackknifing a range of populations. As a result, when being used for indices (which can only have values varying from 0 to 1) or for a number of statistics (such as correlation coefficients, log variances or ratios), a normalizing or variance stabilizing transformation is required - either for the observations, or the statistic, or both. This complicates matters somewhat because such transformations are liable to affect the estimated bias.

  4. Jackknifing is considered to be a more reliable estimate of bias than of standard error, and various formulae have been devised to reduce the bias of jackknife estimators to O(1/n3) or below. Unfortunately, 'higher order' jackknife estimators are much more complicated and require rather more computation than first order estimates, and do not avoid the other problems noted above. So, although jackknife statistics can provide reasonably unbiased estimates, and some measure of standard error, they of much less use in attaching confidence limits or hypothesis testing.

     

     

    How to do it

    As we noted elsewhere, when calculated from random samples the arithmetic mean provides an unbiased estimate of its population mean, so we would not expect our jackknifed estimate to differ. To illustrate the jackknife process, let us check if this is so:

    If you have 4 observations: 1.1, 2.2, 3.3 and 4.4,  their mean is (1.1+2.2+3.3+4.4) / 4 = 2.75

    You can calculate four partial estimates of this statistic:

    1. -1 = (   2.2+3.3+4.4) / 3 = 3.3
    2. -2 = (1.1   +3.3+4.4) / 3 = 2.933
    3. -3 = (1.1+2.2   +4.4) / 3 = 2.567
    4. -4 = (1.1+2.2+3.3   ) / 3 = 2.2

    The four pseudo-values can then be calculated from the partial estimates using the jackknife function.

    Hence the four pseudo-values:

    1. n - (n-1)1 = (4 × 2.75) - (3 × 3.300) = 1.1
    2. n - (n-1)2 = (4 × 2.75) - (3 × 2.933) = 2.2
    3. n - (n-1)3 = (4 × 2.75) - (3 × 2.567) = 3.3
    4. n - (n-1)4 = (4 × 2.75) - (3 × 2.200) = 4.4

    The mean of our 4 pseudo-values is 2.75 which is what we started with!

      We can readily do the calculations in R as seen below:

    Using R
    mean(pv) [1] 2.75 > # estimate bias > mean(pv)-th [1] -8.881784e-16 > # estimate SE > sqrt(var(pv)/n) [1] 0.710047

    The estimated bias and standard error were -4.440892e-16 and 0.710047. Allowing for rounding error, these were very close to the expected zero bias and standard error where SE = sd(y)/sqrt(n)

    Unlike the mean, the standard deviation is not an unbiased estimator of the population standard deviation. In a worked example in the More Information page on the variance and standard deviation, we estimated the corrected standard deviation for a small sample of lamb weights. The uncorrected sample standard deviation (s) was 2.775, whilst the corrected sample standard deviation was 2.953.

    The jackknife estimate of the standard deviation of the lamb weights is given below:

    Using R
    > mean(pv) # estimate statistic [1] 2.985590 > mean(pv)-th # estimate bias [1] 0.2107021

    The uncorrected standard deviation comes out as 2.775. This gives a corrected standard deviation as 2.986

     

     

    Some examples

    1. Effects of modern fishing methods on intertidal communities

      Ferns et al. (2000) looked at the effects of cockle harvesting on invertebrate diversity indices and bird numbers in South Wales. Ten core samples were taken from harvested and control sectors of plots on the day before harvesting and immediately after harvesting. The species richness of the pre-and post-harvesting invertebrate communities was measured with 'alpha' in the log series.

      The indices for each site and sampling occasion were jackknifed to obtain pseudo-values of 'alpha'. This was done by omitting each one of the ten samples in turn to provide partial estimates of 'alpha', and then using the jackknifing function to obtain the pseudo-values. The unbiased means and standards error of 'alpha' were then calculated from the pseudo-values for pre- and post harvesting. Assuming that distributions approximated to normality, pre- and post-harvest means were then compared using a two-sample t-test. Significance levels of differences between pre and post-harvesting are shown below where *** denotes P < 0.001.

      {Fig. 1}
      Ecofer03.gif

      The diversity index dropped slightly (but apparently significantly) in muddy sand after harvesting, but initially increased in clean sand.

         It is surprising that the very small drop in muddy sand resulting from the loss of three rather rare species was so highly significant, but unfortunately the estimated standard errors of 'alpha' are not given, so we cannot evaluate this ourselves. Should perhaps have considered a variance-stabilizing transformation for the diversity indices.

       

    2. Population growth of the spruce bark beetle

      There are relatively few data available on the effects of temperature on life history and population growth of the spruce bark beetle Ips typographus despite it being one of the most important forest pest in Central Europe.. Wermelinger & Seifert (1999) looked at the effects of temperature on reproduction and population growth rate.

      Oviposition rates and developmental periods at different temperatures were determined using phloem pieces of cut spruce trees. A jackknife was used to estimate the variance for all the life table parameters - the net reproductive rate, the intrinsic rate of natural increase, the generation time and the doubling time. For each parameter, partial estimates were made by recalculating the parameter n times using only n-1 of the replicates each time. Pseudo-values were obtained using the jackknife function, and means and variances of the parameters were calculated from the pseudo-values. The figure below shows the generation time and the doubling time for each of three temperatures:

      {Fig. 2}
      Ecower02.gif

      The fastest population growth rates were between 25 and 30°C. This confirmed previous experience that a temperature of 28°C is optimal for mass rearing of the species.