The importance of replication

Not just definitions but individual acts of experimentation must be checked for reliability. Research must be repeated before a finding can be accepted as well-established. Findings obtained at one time might not hold true at another time with different researchers or different experimental subjects. To check the reliability of a finding, one must replicate the research. That means to repeat the research in all its important details .

What is replication? Why is replication "vital to science"? How are operational definitions helpful?

Replication is vital to science. It helps make science a self-correcting system. Any time a result is surprising, researchers will try to replicate it, to see if the phenomenon is dependable or just a fluke (a one-time occurence). Operational definitions are critically important in aiding replication. Why? An operational definition spells out exactly how to measure something. To replicate an experiment, one must know how the original researcher performed measurements. Hence operational definitions must be known, and known precisely, in order to replicate research.

What are common reasons a replication fails?

If a surprising result from research cannot be replicated, that does not mean somebody lied or cheated. Most scientists are honest, and most non-replicated research is due to some difference in the way the research was performed, differences the subjects involved, or other details (perhaps the weather, the location, the time of day, or the instrumentation). Seemingly minor details might influence the outcome of an experiment. If so, scientists need to find out about it. That is why replications are performed.

What does the word "prove" mean?

Until a phenomenon can be reproduced reliably by researchers who are skeptical about it or have no strong motivation to report a positive outcome, one cannot be confident that the finding will prove true in the long run. Indeed, the word prove means test (as in "proving ground" which is a place where automobiles are tested). A finding cannot be accepted as true until it is "proven" in a variety of circumstances.

An example is research on extra-sensory perception or ESP. Positive findings about ESP are often reported, but they always seem to disappear when somebody who is skeptical about ESP tries to replicate them (see the page in Chapter 4 titled Selective Reporting). The same thing can happen in education and health research. Honest researchers often produce positive results for their favored ideas, perhaps because their enthusiasm is conveyed to participants in the research. Skeptical researchers may not be able to repeat the finding. If so, the finding is open to doubt.

Sometimes a dishonest researcher commits a fraud. This is also detected by failures of replication. An example is the "spotted mice" scandal at the prestigious Sloan-Kettering Institute in the early 1970s. A research scientist was working under pressure to produce successful skin grafts from black-furred mice to white-furred mice. In all previous experiments, such grafts failed to "take" because of genetic differences between the mice. This was important research, aimed at reducing the problem of rejection in transplanted organs.

What is a "powerful incentive for honesty among researchers"?

The scientist who faked the skin graft was riding in an elevator early one morning with a batch of his white-furred mice and a permanent ink marking pen. Suddenly and impulsively, he leaned over and made some dark spots on the fur of his white mice. Then he claimed success in transplanting patches of skin from black to white mice. His report of a successful transplant surprised other scientists, so they tried to replicate the experiment, and they could not. Then they knew there was something wrong with the research report. Eventually the scientist confessed to what he had done.

The consequences of fraud are devastating for a scientist, leading to dishonor and usually to the loss of job and career. Scientists know that any important result will be subjected to attempts at replication, and this provides a powerful incentive for honesty among researchers.

Why must earlier researchers be contacted, sometimes? What is the usual result?

Students are taught that a research report should include all necessary details to permit replication. However, it is often impossible to find all relevant details about how research is conducted in a published report of research. Sometimes, to carry out an exact replication, one must contact earlier researchers to learn details of a procedure. Gasparikova-Krasnec and Ging (1987) found that earlier researchers were generally cooperative in providing needed information for later researchers. A month's wait was normally all that was required.

How is even a failed replication "not necessarily bad news"?

Researchers typically are willing to provide details of their procedures to a scientist planning a replication, If a replication fails, the original findings are cast into doubt. However, serious researchers realize this process of double-checking is important to science. Naturally they hope for confirmation of their earlier efforts. However, if the attempted replication fails, this is not entirely bad news (as long as no fraud is involved). A failed replication may have a stimulating effect on a field of research, inspiring new studies to figure out why an attempt to use the "same" procedures led to different results on different occasions or with different populations.


Write to Dr. Dewey at psywww@gmail.com.

Don't see what you need? Psych Web has over 1,000 pages, so it may be elsewhere on the site. Do a site-specific Google search using the box below.

Custom Search

Copyright © 2007 Russ Dewey