Diminishing Returns with Repeated Replications

As noted earlier, replication is vital to progress in science. It provides a way to check up on surprising or important results. If something is true, it should remain true even when a skeptical scientist in a different laboratory tests the idea. Replication failures are always unsettling to the original researchers. When somebody else fails to replicate their findings, the original researchers take a close look at their procedures and try to find out what went wrong. Sometimes small differences in experimental procedures explain the failure to replicate. In such cases, after appropriate adjustments, replications succeed.

On other occasions, however, replication failures continue. This is bad news for the original researchers, because it means their finding was inaccurate or a fluke (a one-time finding). False claims—including those which start as honest mistakes—produce a distinctive pattern during successive attempts at replication: the effects get smaller and smaller as more replications are conducted. This happened, for example, in the case of cold fusion (in which a desktop apparatus was said to produce fusion energy), and in the field of psychology it happened to claims of cardiac conditioning (claims that rates could learn to alter their heart rates directly through classical conditioning).

What pattern occurs with attempts to replicate a false claim?

Diminishing effects with repeated replications occur not because an actual effect is disappearing, but because scientists learn to run the experiments with better controls. A solid, scientific finding will gain more support as people continue to test it. A false lead or quack science claim will become less solid as people continue to test it.

How do quack scientists react to failures of replication?

Quack scientists sometimes reject any attempt at replication at all. They may decline to spell out the details of their research, claiming they intend to file a patent and must keep it secret, or that the work is not yet finished (even though the results are in!). If there is a failure to replicate the findings, the quack scientist may accuse the researchers of bias or conspiracy. They may assert that the very idea of replication is a fallacy, because in nature there is never an exact repetition of events. (This ignores the replicability of other scientific claims and misrepresents the idea of replication, which requires only repeatability of practical results.)

What tends to happen when a "quack science" theory is criticized or discredited?

In general, quack science is not open to correction. If the theory is addresed by serious scientists who offer intelligent objections, the original theory continues to be offered in the original form, as if the criticisms never registered or were ignored. If the quack science theory is discredited and ignored by most scientists, a band of True Believers holds on, forming their own society or institute. For interesting Quack Science web sites, see <http://www.quackwatch.com/>.


Write to Dr. Dewey at psywww@gmail.com.

Don't see what you need? Psych Web has over 1,000 pages, so it may be elsewhere on the site. Do a site-specific Google search using the box below.

Custom Search

Copyright © 2007 Russ Dewey