Sunday, February 26, 2006

Confirmation bias

From a recent reader review of The Conscious Universe. This review provides a nice demonstration of the confirmation bias: If you believe that psi does not exist, but then you encounter evidence indicating that psi does exist, the confirmation bias will cause you to find reasons to reject the positive evidence, which in turn will make you more convinced -- and hence confirms -- that your original hypothesis was correct. Because this bias is rife throughout science, I discuss it and other biases in The Conscious Universe.

Incidentally, I doubt that the reviewer read the book carefully, because I never, ever use the term PSI. The correct word is psi. The reviewer writes (excerpt):

The main reason why I found this book unconvincing is that if PSI existed it would be easy to demonstrate. No meta-analysis of a huge number of studies is needed, one good experiment suffices.

This argument might seem appealing, except for one very important problem: Human performance is highly variable, even among those with world-class talent. E.g., some people claim to be able to hit baseballs thrown at them at 90 miles/hour. If that obviously ridiculous claim is true (because human reaction time is far too slow to hit such a fast-moving object), then it would be easy to demonstrate. We shouldn't need fancy baseball statistics. One good swing of a bat ought to prove the claim once and for all.

Back to the real world, we know that the best baseball players in the world get a hit only about once every three times at bat. If our skeptic happened to look at the plate only when the batter missed the ball, which was the majority of the time, he'd conclude that the baseball claim is false. Especially if he distrusted statistics.

The fact is that many excellent, highly significant experiments have been published, but "one good experiment" isn't convincing (and shouldn't be) because the results might have been due to an undiscovered flaw. This is why independent repeatability is important in science, but also why we always need a way to measure replication rates. Thus the need for meta-analysis. There's no way to escape this because all measurements contain error.

The test I found most impressive is also the simplest one: to have subjects mentally affect the result of random number generators. This experiment is easy and cheap to set up, is so simple that errors can easily be removed, and is easy to reproduce (the book claims that all subjects were successful in nudging the RNG outside of its operational parameters). If it works it would prove that PSI is real.

The book absolutely does not claim that all subjects are successful in these tests, nor does it claim that this is an easily reproducible effect. But the fact remains that some individual RNG experiments have provided extremely strong evidence in favor of psi, and yet as noted above skeptics dismiss those studies.

The fact that PSI believers instead of concentrating on one simple experiment are instead all over the place describing results of disparate, old, and complex studies is a bad sign.

This assumes that all researchers are clones and interested in exactly the same thing. Well, cancer is a very important health problem, but does that mean all biologists should be working exclusively on cancer research? Of course not. Researchers decide what they wish to focus on for dozens of reasons. Discomfort about "disparate, old, and complex" studies indicates a disinterest in history, avoidance of published experiments with strong results, and inability or unwillingness to understand experimental details that provide confidence that results are what they appear to be.

Another way to easily prove the existence of PSI would by using casino statistics. If there is one place where PSI should be present it is there. The author got some daily data about the money dropped and money won at different games such as roulette. If he had only gotten data about the number of roulette plays he could have easily demonstrated that the money won was more than expected thus demonstrating PSI. In the book he claims that casinos are reluctant to share such data, but I wonder why.

Do you know of any business that freely shares its detailed profit/loss details with anyone who asks? I don't. Casinos in particular are extremely tight-fisted about their data because they deal in cash. Public lottery information is another matter. For that data a few large studies have been conducted, as I describe in the book, and in fact they do provide evidence for psi.

So in conclusion, even this book did educate me about the arguments in favor of PSI and made me wonder and think about this issue, the net effect has been to strengthen my belief that PSI does not exist.

Lo! The confirmation bias at work.


Anonymous said...

thank you for sharing your concept of confirmation bias. after years of doing research into life after death it was a complete surprise to me to discover that that the skeptics were the most rigid in their thinking and bias. finally I realized that they have to be rigid because one unexplainable phenomena would create a slipply slope effect and destroy their most cherished paradigms.

Ian Pegler said...

Another example: Two qualified scientists each perform the same experiment and get the same results as each other. One interprets his result as a failure and the other scientist interprets his own result as a pass, even though the raw data is basically the same. (e.g. Wiseman and Sheldrake doing the experiment to see if a dog knows when it's owner is coming home). Refutations, counter-refutations, counter-counter refutations ad nauseum. How is the lay-person with an interest in psi supposed to judge who is right and who is wrong?

Thomas Herold said...

Hello Dean Radin,

I am following your publications and I am looking forward to have your new book in my hands.

I doing some experimenting with random generators since a while, but I am struggling a bit with the hardware.

Following the influence of the collective consciousness on the random events I believe an idividual also influences a random generator near by. From my research most of the modern radionics devices work like that.

Here is random generator that I found on the Internet, which I intend to use:

However I am not sure that this works as it has some extended anti-bias functions build in. I am worried that this would block or minimize the influence from a single consciousness field.

I am looking forward to hear from you.

Thomas Herold

Dean Radin said...

On Ian's post: In the Wiseman-Sheldrake example, Wiseman's data closely matched Sheldrake's data, so it's quite true, oftentimes the same raw data is interpreted differently. Of course, it takes some digging to discover this, so when it comes to any controversial topic, if you are really interested in judging what's what, you either find a researcher who seems credible and is reasonably neutral, or you do your homework and decide for yourself.

Dean Radin said...

On Thomas' post: The RNG I've used most often in field consciousness and microPK research is the Orion. You can find it here: This device uses a logical filtering (XOR) between two truly random bit streams to help ensure an unbiased mean output. This does not seem to affect the results of the experiments, so PK effects do not appear to be happening at the level of the "raw" noise. Rather, the effects seem to "operate" holistically on the outputs of the RNGs.

Adam said...

Hi Dean

Psi has been a fascination of mine for years and I've had more than a few experiences, including encounters with non-physical entities. Psi seems to happen when we least expect it - like when I received a projected thought that preceded a vocalisation at an Aikido class. Or when I consciously influenced dice rolls while in a good mood. Yet when focussed on the psi outcome with my normal, everyday mind too hard the effects seem to evade me - relaxing and opening to the unconscious or (paradoxicly) being in a heightened state of alertness seems to facilitate it for me. The ready, mind-free alertness of Aikido seems akin to what helps psi. Perhaps the sceptics muddy their psi with the wrong mental states?

Spooks said...

It is like a big catch 22. Some will only believe it when they "see" it but what they won't understand is that some things will be seen when you can belive.
Like the "sheep and goats" in Professor Schemeidler's esp experiments in the 40's.

Mike T said...

Regarding the design of the RNG, shouldn't everyone with at least some basic knowledge of AIT, Shannon-Entropy and all that stuff take issue with the claim that the application of logical functions - such as XOR to the output of a (not perfectly) random source will increase entropy (i.e. randomness) in any significant way?

Either you have truely uniform distribution - in which case you don't need any fancy XOR-ing, or you do not - in which case the XOR merely masks the problem, i.e . the non-randomness is a little harder to detect.

How do you account for this in your studies? And what sense does it make to obscure non-randomness, given the effect size to begin with?

Dean Radin said...

"take issue with the claim that the application ... XOR ..."

The best answer to this issue is reported by physicist York Dobyns from the Princeton PEAR Lab: