What it means to be “Scientifically Proven”

Podcast: Download

by Nick Josh Karean from Science, Reason & Rationality
Article ID: 148

“We live in a society exquisitely dependent on science and technology, in which hardly anyone knows anything about science and technology.”

- Carl Sagan, American Scientist (1934-1996)

How many times have you heard someone claim that their products, theories or beliefs are “Scientifically Proven”? They often do this before completely taking over your brain and gaining absolute access into your wallet.

Almost anyone can claim that their ideas are “scientifically proven” or “scientifically tested”. Judging by how gullible and ignorant most of us are, it really doesn’t take much for this claim to convince someone.

What is science?

So, how can you tell which claim is truly scientifically proven and which one is not? To understand, we first need a basic understanding of what science is, and how to ensure that those who claim to be “scientifically proven” have followed the required procedures to really deserve that title.

Okay, let’s get to work. What exactly is science? Science is a method, a procedure, a technique by which we use to examine our surroundings and gain knowledge from them. With sometimes painstaking accuracy, science produces precise unbiased data. It’s not a belief system. Science is fact-based, not faith-based.

For example, mathematics is a form of science: one plus two equals three. ‘One plus two’ is the method and ‘three’ is the answer derived from that method. The answer ‘three’ does not require your belief. It is a fact whether you believe in it or not. This factual approach is the best and the most reliable method for achieving accuracy. No matter how many times you repeat the process, it will produce the same results

Expecting others to believe that one plus two equals four (without using any systematic scientific method to prove it) is not the right method of gaining knowledge and it is always prone to error.

Another example of a factual approach is in the court of law. The court looks for evidence first before making a verdict. The judge and jury do not start with a conclusion – they develop one based on available evidence.

“While anybody is free to approach a scientific inquiry in any fashion they choose, they cannot properly describe the methodology as scientific, if they start with the conclusion and refuse to change it regardless of the evidence developed during the course of the investigation.”

- Judge William Overton of the U.S. District Court for the Eastern District of Arkansas (1939-1987)

The scientific method

The 10 Commandments of Science (inspired by science journalist “Potholer”)

1. Thou shalt base thy conclusion on the evidence.
2. Thou shalt measure objectively, not guess subjectively.
3. Thou shalt back up thy statements with evidence. Just claiming something is a fact doesn’t make it a fact.
4. Thou shalt use large sample sets.
5. Thy tests shall be double-blinded.
6. Thy tests shall have observable controls.
7. Thou shalt cite thy sources of information.
8. Thy sources of information must be reliable, verifiable and backed up by evidence.
9. Thy opinion is not a fact.
10. Thou shalt not bear false witness. Don’t lie!

Well, actually, the above are not really divine commandments, but they’re a good basic guide to scientific observation. All scientific knowledge is based on two things: observation and logic. Thus, good research follows these rules to produce true scientific data and genuine knowledge. As we said earlier, such rules are applied in the court of law to preserve or uphold justice. The rules exist to ensure impartiality, fairness and the most truthful of results.

Now that we’ve understood the basics of science, let’s go a little deeper to see what exactly is involved in this whole scientific procedure thing. We’ll create a scenario, apply the basics of the scientific method, and see what happens…

1. The Problem

Based on the flow-chart image above, let’s imagine a scenario:

One day a young scientist named “Nick” was researching a seemingly supernatural or mysterious natural phenomenon. At this stage, we say, “Nick is working on a problem.”

2. The Hypothesis

Later, Nick gets an idea on how to explain this mystery. At this stage, we say, “Nick has a hypothesis.”

3. The Prediction

To see whether his hypothesis is right, Nick enters the next stage. He makes a prediction about what should happen with this phenomenon under certain circumstances.

4. The Experiment

Nick then performs several experiments and measurements. He collects plenty of data, and continuously tests his hypothesis using the scientific method. When his scientific research and experiments are finally over, Nick may discover that his prediction was actually wrong. If so, he can change his hypothesis and start over again.

However, if Nick’s prediction was correct, he writes down his results on a paper and submits it to a scientific journal. This is not just any publishing house. It’s not a group of friends or select scientists, and not to his own selection of people whom he thinks is best to be peer reviewers for his results. It has to be a scientific journal. This is so Nick’s work can be peer reviewed.

Note: There are many science and pseudoscience magazines that are not really scientific journals. Please refer to this link to a partial list of legitimate scientific journals.

5. Peer Review

The scientific journal then gets other scientific experts of the same field (often from all over the world) to go through Nick’s paper specifically looking for mistakes or errors before it’s approved to scientific standards. It doesn’t matter if Nick is a well-known, experienced scientist. Nick could also be a highly intellectual person, with all the necessary credentials, never been wrong before. He could hold numerous degrees and represent a prestigious scientific organization. Nick may also have others who agree with his ideas. These credentials still do not excuse his paper from being submitted to be peer reviewed and thoroughly scrutinized.

Regardless of the excuses, scientists who have not had their paper peer reviewed have a higher probability of errors or fraudulent results. This is what makes legitimate science more reliable and keeps out the proponents of pseudoscience or bad science. It is this same process that helps guarantee legitimate results. The peer review process isn’t absolutely 100% error-free, but it’s the best process we have for analyzing claims and determining which claims are least likely to be wrong.

Psychologist Dr. Keith Stanovich explains it this way:

“…one important way to distinguish charlatans and practitioners of pseudoscience from legitimate scientists, is the former often bypass the normal channels of scientific publication and instead go straight to the media with ‘their findings.’ One ironclad criterion that will always work for the public when presented with scientific claims of uncertain validity is the question: “Have the findings been published in a recognized scientific journal that uses some type of peer review procedure?” The answer to this question will almost always separate pseudoscientific claims from the real thing…

Not all information in peer reviewed scientific journals is necessarily correct, but at least it has met a criterion of peer criticism and scrutiny. It is a minimal criterion, not a stringent one, because most scientific disciplines publish many different journals of varying quality. Most scientific ideas can get published somewhere in the legitimate literature if they meet some rudimentary standards. The idea that only a narrow range of data and theory can get published in science is false. This is an idea often suggested by purveyors of bogus remedies and therapies who try to convince the media and the public that they have been shut out of scientific outlets by a conspiracy of ‘orthodox science.’

…the failure of an idea, a theory, a claim, or a therapy to have adequate documentation in the peer reviewed literature of a scientific discipline is very diagnostic. Particularly when the lack of evidence is accompanied by a media campaign to publicize the claim, it is a sure sign that the idea, theory, or therapy is bogus…

“The peer review process is far from perfect, but it is really the only consumer protection we have. To ignore it (or not be aware of it) is to leave ourselves at the mercy of the multimillion-dollar pseudoscience industries that are so good at manipulating the media to their own ends….”

6. Publication

Coming back to Nick’s progress: If Nick followed the scientific method meticulously while conducting his experiments and got all of his data and calculations accurate, then there shouldn’t be much of a problem for his paper to be published after being peer reviewed. Otherwise, Nick would be advised to make some amendments or adjustment before it is published, if it was not rejected completely altogether.

7. Replication

The next stage is the replication process, where other independent scientists re-check the results. Like the peer review process, the replication process is also an essential scientific element.

Again, psychologist Keith Stanovich explains it this way:

“Scientific knowledge is public in a special sense… scientific knowledge does not exist solely in the mind of a particular individual. In an important sense, scientific knowledge does not exist at all until it has been submitted to the scientific community for criticism and empirical testing by others. Knowledge that is considered ’special’ – the province of the thought processes of a particular individual, immune from scrutiny and criticism by others – can never have the status of scientific knowledge. Science makes the idea of public verifiability concrete via the procedure of replication.

In order to be considered in the realm of science, a finding must be presented to the scientific community in a way that enables other scientists to attempt the same experiment and obtain the same results. When this occurs, we say the finding has been replicated… It ensures that a particular finding is not due simply to the errors or biases of a particular investigator. In short, for a finding to be accepted by the scientific community, it must be possible for someone other than the original investigator to duplicate it…

Both creation science and intelligent design propose that life on Earth was created magically by some supernatural creator the humans call God. These ideas have been characterized as pseudoscience by the mainstream scientific community for the very reasons explained above. And so, the proponents of creation science and intelligent design (which are pretty much the same thing), often turn this scientific replication requirement back on the theory of evolution. This how their argument sounds:

“Evolution involves events that cannot be repeated or reproduced in the laboratory, so it does not depend on science in the usual sense of the word.”

- creationism.org

The fact is, such experiments on evolution can indeed be repeated or reproduced in a laboratory. Science is based on observations. It does not make claims without evidence to back itself up. So, I suggest the proponents of creationism to read the writings of or watch the video Cosmos by the American scientist Carl Sagan (1934-1996).

In one applicable part of the video, Dr. Sagan says:

“We mix together and spark the gases of the primitive Earth: hydrogen, water, ammonia methane, hydrogen sulfide all present, incidentally, on the planet Jupiter today and throughout the Cosmos. The sparks correspond to lightning also present on the ancient Earth and on modern Jupiter. The reaction vessel is initially transparent: the precursor gases are entirely invisible. But after ten minutes of sparking, we see a strange brown pigment slowly streaking the sides of the vessel. The interior gradually becomes opaque, covered with a thick brown tar. If we had used ultraviolet light simulating the early Sun, the results would have been more or less the same. The tar is an extremely rich collection of complex organic molecules, including the constituent parts of proteins and nucleic acids. The stuff of life, it turns out, can be very easily made.

Such experiments were first performed in the early 1950’s by Stanley Miller, then a graduate student of the chemist Harold Urey… Under the right circumstances in the test tube, short nucleic acids can synthesize identical copies of themselves… (Of course) no one has so far mixed together the gases and waters of the primitive Earth and at the end of the experiment had something crawl out of the test tube (yet)… But we have been performing such experiments for only some thirty years. Nature has had a four billion year head start. All in all, we have not done (too) badly.”

There are other examples where proponents of pseudoscience have taken to undermine legitimate science in order to support their own “scientifically proven” claims. Nevertheless, a pseudoscience remains a pseudoscience until proven otherwise by legitimate scientific method. Here are links to partial lists of known pseudosciences:

Lists of pseudosciences (from Wikipedia)

List of topics characterized as pseudoscience (from RationalWiki)

8. Falsifiability

Falsifiability is another important scientific concept. A claim cannot be scientific or be subjected to scientific inquiry unless it is falsifiable.

Other than passing the replication process, Nick’s experiments must be able to be refuted and retested. The term “falsifiable” here does not mean that Nick’s experiment is false or has to be made false. For example, evolutionary biologist Richard Dawkins, in elaboration of the original quote from geneticist J.B.S. Haldane, said that if there was a single fossil mammal (like a rabbit) in the Precambrian rocks, it would completely blow the theory of evolution out of the water.

Evolution is based on three principles: variation, heredity and selection; if any of these were shown to be flawed, then the theory would also be false. In addition, any of the following would also falsify the theory as well:

• If it could be proven that mutations do not occur.
• If it could be proven that, although mutations do occur, they are not passed down through the generations.
• If it could be proven that selection or environmental pressures do not favor the reproductive success of better adapted individuals.

“If it could be demonstrated that any complex organ existed which could not possibly have been formed by numerous, successive, slight modifications; my theory would absolutely break down. But I can find out no such case.”

- Charles Darwin (1809-1882), English naturalist

Evolution is falsifiable, unlike the ideas of intelligent design or creation science which are deemed not falsifiable at all – they make no predictions that can be tested. If a claim can’t be falsified, it doesn’t mean it’s good or bad. It just means it’s not based in science. That’s one of the primary characteristics of pseudoscience or bad science.

Anti-evolutionists may claim that scientists will never ever allow the theory of evolution to be falsified. They may say that scientists use dishonest methods to prevent this from happening. They may also say that the theory of evolution is actually the theory of “evil-ution” from the Devil and is a scientific conspiracy to stop people from believing in the existence of God.

This is certainly not how science works.

One difference between a scientist and a pseudoscientist is that a legitimate scientist is willing to reexamine facts objectively. A legitimate scientist is also willing to reconsider a theory (even one as widely-accepted as the theory of evolution) if observations and experiments present a new discovery that contradicts or falsifies old ideas.

“…a scientific theory must be tentative and always subject to revision or abandonment in light of the facts that are inconsistent with, or falsify, the theory. A theory that is by its own terms dogmatic, absolutist and never subject to revision is not a scientific theory.”

- Judge William Overton of the U.S. District Court for the Eastern District of Arkansas (1939-1987)

“The criterion of the scientific status of a theory is its falsifiability, or refutability, or testability.”

-  Karl Popper (1902-1994), Austrian Philosopher of Science

“No amount of experimentation can ever prove me right; a single experiment can prove me wrong.”

- Albert Einstein (1879-1955), German Scientist

9. Theory versus fact

As you can see from the above scientific stages, it’s not easy to be “scientifically proven”. There are many rigorous standards of filtering processes that need to be overcome, and only after this painstaking screening processes does a hypothesis finally become a theory.

Scientific “theory” is not the opposite of “fact”. For example, the theory of evolution, like the theory of gravity and the theory of electromagnetism, is indeed a fact. You might think, “Well, if it is a fact, then why call it a theory? Why not simply call it a fact?”

In science, nothing is infallible. Nothing is absolute, like it is with creation science and other pseudosciences. Scientific theories are supported by observable evidence. The theory has to also be consistent with new theories in other fields. New technologies based on that theory have to work. For as long as this occurs, then a theory – for all intents and purposes – is a fact and can be added to school text books and other scientific journals.

At the same time, this does not mean it can’t be updated or modified when new and improved data is discovered. Even pillars of science like the theory of gravity and the theory of electromagnetism are always open for new updates or modifications. Such updates and modifications might not overturn the previous theories; they can just compliment them and give us a better understanding of how something works.

Science is self-correcting, cumulative knowledge

As said earlier, nothing is infallible. Errors occur from time to time, but with the technology, experience and cumulative knowledge, errors are minimized and the process is self-correcting. It’s unwise to dump all scientific theories and go back in believing in the supernatural, superstitious assumptions, or unsubstantiated claims.

Science can’t guarantee perfection, but it does guarantee that mistakes will be corrected.

Back to our example of Nick the researcher. Let’s say Nick is secretly a mad scientist, and he comes up with a different kind of hypothesis, and also plans to abuse his scientific knowledge by taking advantage of a gullible public to make money from it.

Nick realizes that if gets his data inaccurate, he will not pass the peer review process. His paper would be rejected. So instead, Nick cunningly fakes the results to make it seem as though his hypothesis was accurate. Based on his fake data, Nick’s fraudulent paper manages to convince the peers and it gets published. To make it more interesting, let’s imagine that Nick has also managed to corrupt some peers with bribes. Unable to resist the offer, the peers approve his paper and Nick becomes published.

This is one of the reasons why some people say they have no confidence in the scientific procedures – it can be manipulated to support a hypothesis. But, what these people don’t realize is that even if Nick’s paper gets published, there are other independent researchers and scientist out there. When they can’t replicate Nick’s results, when Nick’s evidence isn’t consistent the hypothesis, the theory will no longer be accepted by the scientific community. Nick will also suffer loss of credibility – he can say “goodbye” to his career as a legitimate scientist. He can however, try his luck in a field that would accept his dishonesty – say that of pseudoscience.

Bad science will always be exposed for what it truly is. In the meantime, it’s important to ensure that whatever your children learn in science class has been thoroughly tested, verified and supported by evidence.

Fundamentalists don’t criticize researches when they come up with theories that explain electricity, or the role of vitamins in health, but they think scientists get things hopelessly wrong whenever they research about the origins of our species and the universe.

What do you think would happen if legitimate science also worked like religion? What if pseudoscientists presented their own unfalsifiable ideas and called it science, without going through all the processes mentioned earlier? The video below presented by Professor Richard Dawkins will provide you with the answer.

“Science is the single most consistently reliable method we have for determining an understanding of reality.”

- Matt Dillahunty

Other articles related to this topic:

• Evolutionary science and creationism: A skeptical response to Duane Gish’s “Creation Scientists Answer Their Critics”
• Intelligent Design in Michigan
• Research methods: How to find answers
• Faith versus the scientific method
• Macro-evolution observed in the laboratory