Sturges's rule: Difference between revisions

From testwiki
Jump to navigation Jump to search
imported>Sprint99
Reverted my earlier change which was based on a misunderstanding
 
(No difference)

Latest revision as of 14:41, 18 October 2024

Template:Short description


Sturges's rule[1] is a method to choose the number of bins for a histogram. Given n observations, Sturges's rule suggests using

k^=1+log2(n)

bins in the histogram. This rule is widely employed in data analysis software including Python[2] and R, where it is the default bin selection method.[3]

Sturges's rule comes from the binomial distribution which is used as a discrete approximation to the normal distribution.[4] If the function to be approximated f is binomially distributed then

f(y)=(my)py(1p)my

where m is the number of trials and p is the probability of success and y=0,1,,m. Choosing p=1/2 gives

f(y)=(my)2m

In this form we can consider 2m as the normalisation factor and Sturges's rule is saying that the sample should result in a histogram with bin counts given by the binomial coefficients. Since the total sample size is fixed to n we must have

n=y(my)=2m

using the well-known formula for sums of the binomial coefficients. Solving this by taking logs of both sides gives m=log2(n) and finally using k=m+1 (due to counting the 0 outcomes) gives Sturges's rule. In general Sturges's rule does not give an integer answer so the result is rounded up.

Doane's formula

Doane[5] proposed modifying Sturges's formula to add extra bins when the data is skewed. Using the method of moments estimator

g1=m3m23/2=1ni=1n(xix)3[1ni=1n(xix)2]3/2,

along with its variance

σg12=6(n2)(n+1)(n+3)

Doane proposed adding log2(1+|g1|σg1) extra bins giving Doane's formula

k^=1+log2(n)+log2(1+|g1|σg1)

For symmetric distributions |g1|0 this is equivalent to Sturges's rule. For asymmetric distributions a number of additional bins will be used.

Criticisms

Histogram of 10,000 samples from a Gamma(2,2) distribution. Number of bins suggested by Scott's rule is 61, Doane's rule 21, and Sturges's rule 15.

Sturges's rule is not based on any sort of optimisation procedure, like the Freedman–Diaconis rule or Scott's rule. It is simply posited based on the approximation of a normal curve by a binomial distribution. Hyndman has pointed out[6] that any multiple of the binomial coefficients would also converge to a normal distribution, so any number of bins could be obtained following the derivation above. Scott[4] shows that Sturges's rule in general produces oversmoothed histograms i.e. too few bins, and advises against its use in favour of other rules such as Freedman-Diaconis or Scott's rule.

References

Template:Reflist

  1. Template:Cite journal
  2. Template:Cite web
  3. Template:Cite web
  4. 4.0 4.1 Template:Cite journal
  5. Doane DP (1976) Aesthetic frequency classification. American Statistician, 30: 181–183
  6. Hyndman RJ. The problem with Sturges' rule for constructing histograms. Monash University. 1995 Jul 5:1-2.