Sunday, September 05, 2010

Feeling the future

Daryl Bem's article, "Feeling the future," is now in press in the Journal of Personality and Social Psychology, an American Psychological Association high impact journal. You can download a preprint of the article from here.


Feeling the Future: Experimental Evidence for Anomalous Retroactive Influences on Cognition and Affect

Daryl J. Bem

Cornell University

The term psi denotes anomalous processes of information or energy transfer that are currently unexplained in terms of known physical or biological mechanisms. Two variants of psi are precognition (conscious cognitive awareness) and premonition (affective apprehension) of a future event that could not otherwise be anticipated through any known inferential process. Precognition and premonition are themselves special cases of a more general phenomenon: the anomalous retroactive influence of some future event on an individual’s current responses, whether those responses are conscious or nonconscious, cognitive or affective. This article reports 9 experiments, involving more than 1,000 participants, that test for retroactive influence by “timereversing” well-established psychological effects so that the individual’s responses are obtained before the putatively causal stimulus events occur. Data are presented for 4 time-reversed effects: precognitive approach to erotic stimuli and precognitive avoidance of negative stimuli; retroactive priming; retroactive habituation; and retroactive facilitation of recall. The mean effect size (d) in psi performance across all 9 experiments was .21, and all but one of them yielded statistically significant results. The individual-difference variable of stimulus seeking, a component of extraversion, was significantly correlated with psi performance in 5 of the experiments, with participants who scored above the midpoint on a scale of stimulus seeking achieving a mean effect size of .42. Skepticism about psi, issues of replication, and theories of psi
are also discussed.

21 comments:

Tor said...

Thanks Dean!

It's on my list on articles to read.

I'm happy to see psi articles published in mainstream journals. Hope to see more of that happening.

MickyD said...

This publication and an in press article by Emily Kelly on anomalous cognition in mediums (in JNMD) together with the Psych Bull meta analysis by Lance Storm. Appears to be a greater willingness from mainstream psychology towards psi as of late. Would you agree Dean?

Dean Radin said...

> greater willingness from mainstream psychology towards psi ...?

It may be that as new discoveries dissolve past prejudices, like the prohibition against "action at a distance", that younger scientists are becoming more open to new possibilities. A few generations from now I suspect that psi will still be controversial, but no more so than dozens of other leading-edge topics in science. By then the question of existence, of something real worth studying, will be replaced by mainstream debates over which theories seem more viable, and how to best interpret the latest experimental evidence.

Mark Szlazak said...

MickyD, would you happen to have a soft-copy of the Kelly paper in JNMD that you could share? I can't find it online.

MickyD said...

Hi Mark,

It's in press so not yet!

Best,
Michael.

Pikemann Urge said...

Just as an aside: a friend of mine is studying psychology. One or two years ago, one of his lecturers made a casual aside about presentiment (and how we don't yet have an explanation), and then moved on with his presentation. The point here is that quite a few psychologists already accept psi (or parts of it).

EJ said...

Hello,

We just submitted a reply to the Bem paper, outlining the problems with this study (see http://dl.dropbox.com/u/1018886/Bem6.pdf for the complete paper). Here is the abstract:

Does psi exist? In a recent article, Dr. Bem conducted nine studies with over a thousand participants in an attempt to demonstrate that future events retroactively affect people’s responses. Here we discuss several limitations of Bem’s experiments on psi; in particular, we show that the data analysis was partly exploratory, and that one-sided p-values may overstate the statistical evidence against the null hypothesis. We reanalyze Bem’s data using a default Bayesian t-test and show that the evidence for psi is weak
to nonexistent. We argue that in order to convince a skeptical audience of a controversial claim, one needs to conduct strictly confirmatory studies and analyze the results with statistical tests that are conservative rather than liberal. We conclude that Bem’s p-values do not indicate evidence in favor of precognition; instead, they indicate that experimental psychologists need to change the way they conduct their experiments and analyze their data.

Dean Radin said...

> in order to convince a skeptical audience of a controversial claim, one needs to conduct strictly confirmatory studies and analyze the results with statistical tests that are conservative rather than liberal...

Ideally, yes, given enough time and resources. But in the real world Bem's article presents a radically new experimental technique (radical for orthodox psychology, anyway) that goes far beyond what is usually presented for novel methods. Not just a single, small-scale, exploratory study, but multiple conceptual replications, large subject pools, and transparent methods and statistical analyses. From this one hopes that many people will attempt to replicate the effect. That has the potential of convincing a skeptical audience, which no single paper, regardless of how well executed or conservative the underlying experiments, can ever do. Bayesian methods have their place, but if one starts with extremely strong priors in favor of the skeptical position, no evidence can ever be good enough.

Pikemann Urge said...

EJ, I always appreciate criticism of anything I'm interested in, and so I hope that I'm able to get something out of your paper. However, I'm not very knowledgeable in statistics - but, I do understand Bayesian theory just enough to know what it's for.

In fact one historian wants to apply it (among other things) to the study of the historical Jesus. I was against it at first. But later I thought, well, why not? There's nothing to lose.

But at the same time it seems that skeptics of psi always try to stack the deck - or, if you like, shift the goalposts. Skeptics get to commit the fallacies that they are quick to point out when other people commit them. Prof. Richard Wiseman pretty much does that:

"I agree that by the standards of any other area of science that remote viewing is proven."

Yet he suggests that higher standards are necessary for the paranormal. He doesn't understand that this rigour is always inherent in the results.

As soon as Dean mentioned "if one starts with extremely strong priors" I got suspicious. That doesn't mean that EJ's paper is useless, but it looks like the same old thing again.

And if Dean is not a good example of a conservative scientist I don't know who is.

Dean Radin said...

EJ et al's paper seems thoughtful and detailed, but alas, it is utterly wrong for many reasons. I will just name three.

(1) They state that "these experiments highlight the relative ease with which an inventive researcher can produce significant results even when the null hypothesis is true...," yet they fail to acknowledge that the exact opposite is true of analysts who disagree with empirical results. I.e., if a clever person starts with the assumption that a hypothesis cannot be true, then all sorts of reasons, valid or not, can be imagined to confirm that hypothesis.

(2) Their bias is clear: "First, there exists no mechanistic theory of precognition (see Price, 1955 for a discussion)." This concern is common among psychologists, who, not being trained as physicists, implicitly hold 17th century views of physical reality. Today's physics allow for all sorts of strange effects, including retrocausation. They also fail to state that Price later apologized for his article and published a retraction in Science.

(3) "there is no real-life evidence that people can feel the future (e.g., nobody has ever collected the $1,000,000 available for anybody ...)" This line of argument seriously confuses real-life with entertainment.

Enfant Terrible said...

Hi, Dean

your criticisms are important (and I agree specially with your point 3: the authors are not familiar with literature, since the psychic Malcolm Bessent showed excelent evidence for precognition), but they don't attack the central argument in EJ's article, which is:

Out of the 10 critical tests, only one yields “substantial” evidence for H1, whereas three yield “substantial” evidence in favor of H0. The results of the remaining six tests provide evidence that is only “anecdotal” or “worth no more than a bare mention” (Jeffreys, 1961). [...]Overall, the Bayesian t-test indicates that the data of Bem do not support the hypothesis of precognition.

You can attack the authors for their lack of knowledge of literature, but they really seem to have noted important flaws in Bem's article. I would like to know what Bem himself has to say about this, or some statistician like Jessica Utts, before come to a conclusion.

Dean Radin said...

As the authors of the critique say, the statistical issue they raise is not just relevant to Bem's paper, but holds for virtually all experiments that rely on classical (or frequentist) stats rather than Bayesian.

This is an ongoing debate in the experimental sciences and is not likely to be resolved any time soon.

However, it is representative of arguments with double-edged swords, i.e. if the criticism is valid, then the majority of published research in most domains, including most of what we know from conventional disciplines, is equally problematic. It is certainly valid to question prevailing methods, but it represents a problem for psi research.

We design studies that are as transparently simple as possible to avoid critiques that the methods used in psi studies are too foreign, or untested, or new. Then we are criticized for using those very same methods because for one reason or another they are wrong. I.e., you're damned if you do and damned if you don't.

For a good discussion of the Bayesian vs. frequentist debate, see this paper: http://www-stat.stanford.edu/~ckirby/brad/papers/2005NEWModernScience.pdf

EJ said...

Hi Dean,

Our main points are straightforward:

1. one should not confuse exploratory research with confirmatory research; 2. extraordinary claims require extraordinary evidence; 3. A default statistical test shows no evidence for Bem's assertion. I'm sure we agree on these three crucial points.

By the way, Bem himself defines psi as follows: "The term psi denotes anomalous processes of information or energy transfer that are currently unexplained in terms of known physical or biological mechanisms". So we did not really make this up. Nevertheless, we have already received one theory from a psychologist/physicist who claims it can explain psi. So this issue might be indeed be debatable, although it is not central to any of our main arguments.

Please also note that the skeptical prior you mention was only used to formalize the adage that extraordinary claims require extraordinary evidence. It does not affect our other claims or Bayes factor analyses.

Finally, at the end of our paper we outline several guidelines that can be followed to really convince people. So we aren't claiming that it cannot be done.

Cheers,
E.J.

Dean Radin said...

> 1. one should not confuse exploratory research with confirmatory research.

True. But Bem's studies are not just exploratory. They are confirmatory in the sense that they were designed to extend results of previously reported precognition experiments.

> 2. extraordinary claims require extraordinary evidence

Yes, and extraordinariness is in the eye of the beholder. I.e., if one starts with the assumption that this cannot be true, or that it violates one or more laws of physics, then it is super extraordinary. But if one knows about many previous experiments describing similar results, and also knows something about physics, then it is rather ordinary.

> 3. A default statistical test shows no evidence for Bem's assertion. I'm sure we agree on these three crucial points.

No. From a frequentist perspective the odds against chance for Bem's series of experiment is well beyond a million to one.

> Finally, at the end of our paper we outline several guidelines that can be followed to really convince people. So we aren't claiming that it cannot be done.

On this we fully agree.

EJ said...

Hello Dean,

You mention that "They also fail to state that Price later apologized for his article and published a retraction in Science."

I looked up the "retraction", and, as it turns out, the **only** thing Price does is apologize for suggesting that Soal cheated. So I take issue with your summary and the deeper conclusion that comes with it.

Funnily enough, the consensus now seems to be that Soal did in fact cheat (see http://en.wikipedia.org/wiki/Samuel_Soal).

Cheers,
E.J.

Dean Radin said...

Price apologized to both Rhine and to Soal partially because of his conversion from an atheist to a Christian. In his initial letter to Science he had assumed that Rhine was trying to support his religious beliefs (which was absolutely not true). And in the latter case it seems reasonable to assume that he was now in favor of Rhine's work for that very same reason. In other words, after his Bayesian priors shifted what was initially viewed as impossible now suddenly became possible.

The fact that Soal seems to have cheated (based on a statistical analysis many decades later) is irrelevant because Price's harsh accusation was made without the benefit of any evidence at all. I.e., it was pure prejudice.

In addition, in 1961 Price conducted an ESP card experiment and obtained odds against chance of 300 to 1, which impressed Price (but will not persuade a Bayesian with low priors!). This is described in the book, "The Price of Altruism: George Price and the Search for the Origins of Kindness," by Oren Harman.

Enfant Terrible said...

E.J.,

you should mention the work that many scientists did with Malcolm Bessent. References:

01 - A precognitive dream study with a single subject. (1971). Journal of the American Society for Psychical Research, 65, 192-203. KRIPPNER, S., ULLMAN, M., & HONORTON, C.

02 - Automated forced-choice precognition tests with a “sensitive.” (1971). Journal of the American Society for Psychical Research, 65, 476-481. HONORTON, C.

03 - A second precognitive dream study with Malcolm Bessent. (1972). Journal of the American Society for Psychical Research, 66, 269-279. KRIPPNER, S., HONORTON, C, & ULLMAN, M.

04 - Precognition and real-time ESP performance in a computer task with an exceptional subject. (December 1987) Journal of Parapsychology, 51, 291-320 Charles Honorton

05 - Precognition and real-time ESP performance in a computer task with an exceptional subject: a replication attempt. The Journal of Parapsychology, 54, June 1990. Bruce E. McDonough, Charles A. Warren, Norman S. Don

Marylo said...

Dear Dean Radin,

Thank you for the disentanglement you provide in such a wonderful way and with such exciting results.

I happened to have chosen a similar title to a blog of mine: "Tangled Communication About to Advance".
This is my latest input there:

All born to be each others servant
I try to be increasingly observant
of what’s most significant at heart
and then just strive to do my part.

Love from Lo Snofall

Falk Lumo said...

Being a theoretical physicist, I wonder how this is discussed. Is there no curiosity in the topic itself?

First, I find it hard to read Bem's paper. To get to the point: He had 3600 questions for erotic images and 1912 were answered correctly (53.1%, 112 more than the average). The normal fluctuation would have been +/-30 or so. So, in one out of 9 experiments, he saw 82 good answers he cannot explain.

We all know that such a thing can happen. We've all seen it happen in our own experiments. Make enough experiments and it happens. There is no need for fancy statistical methods and fights of schools of thought.

Rather, I wonder why the most simple procedure to clarify this isn't followed? 82 good answers where 100 answers cost 5$ and that's it?

If Psi exists then it shouldn't pass away if more experiments are made. Right?

So, why not increase 3600 to 14400 and see if 112 increases to 448? Because if it only increases to 172 then the effect is nil. Quite simple.

So, the meta conclusion is that "scientists" in this field have an interest to discuss and publish. But not to do science and find out truth. That's really bad actually.

Dean Radin said...

> Is there no curiosity in the topic itself? First, I find it hard to read Bem's paper....

I hope that the paper stimulates lots of independent replications. Then the statistical power question will be settled fairly quickly. The design is extremely simple, as are the statistical analyses. There ought to be thousands of academics who are well equipped to replicate these studies.

Sandy said...

A new paper is out that claims to refute Bem's work:

http://papers.ssrn.com/sol3/papers.cfm?abstract_id=2001721

One issue seems to be that Galek et al claim to be replicating Bem's procedure, but have introduced data from an online experiment. The online experiment allows for a variety of ways to cheat that are not controlled for in the analysis.

One very simple way to cheat on the online experiment that wasn't addressed in the paper, and that would be difficult (if not impossible) to do in the lab under supervision, would be to have two people work on the test together. One person does the first part of the test where you are shown words and try to remember as many as possible. A second person does the "study" section of the test where you only see half of the words (the half you should do better on in the first section if psi is in effect).

The response I've gotten from skeptics was that there is no evidence that Bem supervised participants to prevent this sort of cheating in his lab experiments, so the fact that online cheating is possible wouldn't make a difference to Galek's results. I just don't don't buy that argument.

Any thoughts on this?