Comments on: Voodoo II: this time it isn’t personal

By: phineasgage

phineasgage — Wed, 29 Apr 2009 22:00:49 +0000

@Stephen:
I think this is an error that people will keep making, because it consistently biases results in favour of finding an effect. We are all more inclined to believe convincing results, sadly.
The Kriegeskorte paper goes some ways beyond this basic point however. For me, the most interesting point made was that using ‘orthogonal’ contrasts in place of independent data is inherently risky, since orthogonal contrast vectors do not ensure that the contrast as such is orthogonal (see in particular the supplement for the NN paper). There are some very nice simulations to back this up.

By: Jennifer R. Ewing

Jennifer R. Ewing — Wed, 29 Apr 2009 17:23:48 +0000

Oops..I mentioned the phrase “learning disabled” twice in one sentence. Maybe *I’m” learning disabled .

By: Jennifer R. Ewing

Jennifer R. Ewing — Wed, 29 Apr 2009 17:22:04 +0000

It’s very sad to me that “pop psychology” has taken such a foothold in American society that there are now literally storefront, strip mall locations where you can get MRIs performed, then read by–get this–social workers, who will then inform anxious parents whether their child is learning disabled, hyperactive, has a learning disorder, etc. This is why the information in the above article bears repeating, and bears repeating often, and loudly. As the article points out, the selective “reading” of such MRIs by unlicensed (in the fields of neurology/radiology) “professionals” is nothing more than shooting multiple arrows against a wall, drawing a circle around the arrows which land closest together, then calling that a “meaningful result”. And, unfortunately, too many well meaning parents, and too many uninformed patients, simply don’t know enough to tell the difference between meaningful results and “voodoo radiology”.

By: Stephen Hampshire

Stephen Hampshire — Wed, 29 Apr 2009 13:33:17 +0000

Interesting stuff, but from the point of view of a survey researcher/statistician it seems surprising that this is only now being raised as an issue.
Naive post-hoc tests, “fishing expeditions” and using the same data for generating/confirming hypotheses are all well-established problems surely?