The myth of military testing standards

Podcast: Download

By Andy Kaiser
Article ID: 152

[Pretend you're hearing a bunch of click-clacky gun-loading noises right now. Shotguns being loaded, pistol slides being racked, ammo cartridges being slammed into place.]

Anyone who’s seen a shoot-em-up Hollywood movie in the last 60 years knows: That is the sound of Awesome.

You’ve got the sound of heavy artillery being prepared for bloody, violent action. Some may be precision instruments, military tools. High-tech, superbly-designed weapons, all proven to outlast their civillian counterparts. These tools of destructive peacekeeping are all tested and passed by the most rigorous military testing standards.

It certainly sounds cool when I say that something was “designed per military specifications“. But what does that really mean?

Without more information, there’s a problem. The claim can mean whatever the tester wants it to mean.

To use an example from my own life, let’s look at a cellphone.

I review consumer technology, stuff like ereaders, computer stuff and yes, cellphones. A few months back, I got my hands on a demo unit of the Casio G’zOne Brigade cellphone, a high-tech yet super-durable phone. It was so tough, the marketing material said that the Brigade “Meets Military Specifications 810F standards for Water, Shock & Dust Resistance, Immersion, Vibration, Salt Fog, Humidity, Solar Radiation, Altitude, Low and High Temperature Storage.”

This was a pretty cool opportunity. The Verizon sales rep gave me special permission to not only review the phone, but to put it through the military standards that the phone was tested for! I could put the phone in the oven, in the freezer, and even under water!

So I did.

Now, to paint the whole picture, I’d done this before with a similar phone. And when I reviewed that one, I was given some very specific military testing standards, a big list of tests like this:

“The Drop test was performed in accordance with Method 516.5, Procedure IV. The G’zOne Type-V was sequentially dropped in non-operating mode, onto each face, edge and corner for a total of 26 drops from a height of 1.50 meters (4.9 feet). The drop surface was defined as two-inch-thick plywood over a steel plate over concrete.”

That was an older phone. I had a little problem with this newer, phone, though, because for some reason I couldn’t find any specifics on what military spec tests had actually been performed. I tried to pull information from Casio, from Verizon and the Internet at large. Nothing.

Military standards are important, and needed, because they can tell you what kind of equipment you want to take with you in certain extreme conditions, conditions that could be a life-or-death situation. But without detail, the claim of testing is meaningless, because we don’t know what standards were tested against, or what kind of failure rate the subject experienced.

In this most recent case, simply saying “military standards” won’t tell me if my phone is suited more to Death Valley, or a warm Caribbean beach.

If you see claims for military testing standards, realize the following:

• The “Military Specifications 810F” standard is not a test. It’s a collection of tests. Each test has variables that can be set by the tester. If you say that something “conforms to MIL-STD-810F“, it’s meaningless unless you also say what specific test was performed, and under what conditions.
• You can easily claim a device was designed for or tested by military specifications. That doesn’t mean it passed!

With this latest ultra-rugged cellphone, I took it upon myself to perform specific tests. Yes, the phone passed all of them, and gave me some awesome pictures of me doing things to a cellphone you really shouldn’t do. There were multiple cool tests. My favorite was testing for a high-velocity rainfall: I got to spray an expensive cellphone with a hose.

At a low level, I learned a lot about the 810F military test standards and how they can be abused or misunderstood. I learned that when they’re used properly, such standards are very important, and can help guarantee that a tool or a weapon will behave in the way that you think it will.

At a high level, the lesson here is even simpler: An impressive-sounding claim means nothing if there’s no way to verify it. Because for some tools and weapons, from the simplest Leatherman to the best lightsaber, knowing the limits can be the difference between life and death.

Other articles related to this topic:

• Double-blind testing
• Orgone chips review: New Age tech versus the scientific method
• The Dyatlov pass accident and the fatal “unknown compelling force”
• Private Military Companies, civilian contractors and the Global War on Terror
• A shocking lesson in human nature