Comments on: It always seems worse than you think

By: Jeff Beck

Jeff Beck — Wed, 07 Oct 2009 14:20:44 +0000

here’s a hasty blog post explaining why this ‘bias’ is rational from a Bayesian statistical perspective:
http://justaddnoise.blogspot.com/2009/10/here-is-probability-that-true.html
but the short answer is just that the median estimate is greater than the mean and maximum a posterori estimates of small quantities and so it’s actually likely that small quantities are underestimated.

By: Jeff Beck

Jeff Beck — Wed, 07 Oct 2009 13:52:24 +0000

It’s actually good statistical reasoning to assume that rare events are underestimated in their frequency. A Bayesian statistical analysis can demonstrate this. In short, if you have N observations of a Poisson distributed (rare) random variable, then the maximum likelihood estimate of the mean of the underlying process (lambda) is N.
However, one could take a proper Bayesian approach and ask, what is the probability that the true mean, lambda, is greater than this estimate? It turns out that the smaller N is the more likely it is that the true rate is greater.
To see this, place the conjugate Gamma prior on lambda so that the posterior distribution on lambda is also Gamma distributed to obtain,
p(lambda|n)
Now we can ask, what is the probability that lambda is actually greater than the mean of this distribution or even greater than the maximum aposteriori estimate of lambda? It turns out the answer is more or less independent of the parameters of the prior in this case. And its quite high. See figure here:
http://4.bp.blogspot.com/_QrGZ_bdq8kE/SsyMyTgcrwI/AAAAAAAAAmw/b7dykNNKUZk/s1600-h/underestimation.jpg
This result also holds for binomial distributions with uniform prior. x=lambda in the figure. For the Poisson case, the observed rate has been normalized by 1000 to fit on the same plot.

By: Rhodora Online

Rhodora Online — Sat, 26 Sep 2009 15:13:39 +0000

To me it seems like that same bias for hypothesis-proved rather than hypothesis-disproved results. The latter ones are more often filed away and neglected and the former ones have a better chance of being sent and accepted for publication.

By: raestyr

raestyr — Sat, 26 Sep 2009 00:10:23 +0000

Yet Google also says:
Results 1 – 10 of about 284 for “the true number may be higher” -”the true number may be higher or lower”. (0.19 seconds)
Ah, search engines. And english.

By: Neuroskeptic

Neuroskeptic — Fri, 25 Sep 2009 22:57:04 +0000

This may just be my cynicism showing but I think it’s because generally the people responsible for estimating the prevalence of a given disease are the same people with an interest in making that prevalence seem high.
Say you’re an academic who researches GID (or anything else). If you can say “GID affects maybe 1 in 100 people” that looks a lot better when you’re asking for grant money than if you say “GID affects 1 in 100,000 people”.
Or say you’re an activist for the interests of people with GID, you will naturally want to make it seem as common as possible because this is, unfortunately, equated with “normality” in most people’s minds.

By: Maia Szalavitz

Maia Szalavitz — Fri, 25 Sep 2009 22:06:56 +0000

It’s simply because if it’s a bigger number, it’s a bigger story.

By: WhiskerBiscuit

WhiskerBiscuit — Fri, 25 Sep 2009 21:43:32 +0000

> “the true number may be higher” 20,300 hits
Yes, but the true number may be higher.