Wednesday, June 09, 2010

Maybe the check is in the mail?

The Wikipedia entry on Masaru Emoto is a good example of why no one should trust an encyclopedia written by anonymous amateurs. I know it is possible, at least in principle, to edit Wikipedia pages to make corrections. But it is also possible for pranksters to change information on any page just for fun. And I know teenagers who regularly do this to confuse their classmates.

The case in point was brought to my attention by a friend. I will correct the entry here. I've tried making corrections to Wikipedia in the past, and I'm not willing to go through that waste of time again. I'll italicize the Wikipedia entries:

In 2003, [the magician] James Randi publicly offered Emoto one million dollars if his results can be reproduced in a double-blind study.

I was coauthor on such a study, which was co-sponsored by the Institute of Noetic Sciences and published in 2006. You can find it here on PubMed. As far as I know Emoto hasn't received the one million dollar check. I know I haven't.

In 2006, Emoto published a paper together with Dean Radin and others in the peer-reviewed Explore: The Journal of Science and Healing (of which Radin was co-editor-in-chief).

Yes and no. Yes: I published such a paper. No: I became a co-editor-in-chief of this journal in 2009, three years after publishing that paper. I had no connection with Explore prior to that. Nor did I have any affiliation or financial interest in Emoto's work then, or now.

The paper itself was not peer-reviewed, as the journal only conducts peer reviews of articles submitted within the 'scientific' category, a label which Emoto and Radin chose not to apply to their work.

The citation attached to the above sentence refers to a photo essay about Emoto's crystals, published in Explore in 2004. I am not a coauthor of that article. I had nothing to do with it. The double-blind paper we published in 2006 was indeed peer-reviewed, and it showed a statistically significant difference between water that was "exposed" to intention vs. identical water set aside as a control. The magnitude of the observed effect was smaller than is implied in Emoto's books, but the direction of the effect was consistent with his claim.

A better-controlled "triple-blind" follow-up study published in the Journal of Scientific Exploration did not yield positive results.

No. The cited reference points to an article in a popular magazine that got it dead wrong. The abstract of the original journal article, of which I am a coauthor, reads:

An experiment tested the hypothesis that water exposed to distant intentions affects the aesthetic rating of ice crystals formed from that water. Over three days, 1,900 people in Austria and Germany focused their intentions towards water samples located inside an electromagnetically shielded room in California. Water samples located near the target water, but unknown to the people providing intentions, acted as ‘‘proximal’’ controls. Other samples located outside the shielded room acted as distant controls. Ice drops formed from samples of water in the different treatment conditions were photographed by a technician, each image was assessed for aesthetic beauty by over 2,500 independent judges, and the resulting data were analyzed, all by individuals blind with respect to the underlying treatment conditions. Results suggested that crystal images in the intentionally treated condition were rated as aesthetically more beautiful than proximal control crystals (p = 0.03, one-tailed). This outcome replicates the results of an earlier pilot test.

There were, however, potential problems with the "triple-blind" follow up. As the study explains:

Yes, as I explained. Empirical studies regularly contain sections discussing the limitations of the design. No experiment is perfect.

"In any experiment involving intention, the intentions of the "investigators" cannot be cleanly isolated from those of the nominal participants and this in turn constrains how one should properly interpret the results....

All quite true. But the above snippet starts in the middle of a paragraph. The actual article begins this paragraph with the following:

"These design elements excluded obvious environmental differences and conventional subjective biases as plausible explanations for the observed results, and the combined results of the two experiments appear to exclude chance as an explanation (unweighted Stouffer Z = 3.34, p<0.0004)."

In other words, when it comes to Wikipedia, reader beware. Meanwhile, I'm still waiting for my prize check. Maybe it's in the mail.