« Underwater computing? | Main | Thoughts about rugged handhelds -- the Juniper Archer »
November 15, 2007
Tests and reviews - how much punishment?
I love rugged machinery, and so does everyone else here at RuggedPCReview.com. When a new machine comes in, everyone wants to see it, touch it, comment on in, and speculate how much abuse it can take. And this is where it gets interesting, the degree of abuse.
Rugged machines are, by design, conceived and built to take a beating and survive. But the only way to know for sure if they indeed CAN take a beating is to administer one. And whether or not we should do that is a sensitive issue. A lot of this equipment is not inexpensive. So do we take a $4,000 computer, drop it, twist it, spill coffee on it, try to see if the screen is really scratch-proof and whether it's really water-proof? And then send back, at best, a severely banged-up machine, and at worst, one that is destroyed? Dvorak may get away with that and maybe some of the few remaining big print magazines, but I am not sure most eval unit coordinators would look upon such a reputation with great favor.
That puts us in an interesting situation. We really think that rugged equipment should be just as rugged as manufacturers say it is, and sometimes we have doubts. We also see some stuff we are not very fond of. For example, glossy metallic surfaces that can and will get scratched in an instance simply should not be on a rugged machine, no matter how cool they look. But even there, do we just mention that in a review, or see just how badly it scratches (or not), document that, and then send it back?
Most rugged machines come with ruggedness specs. MIL-STD results are listed and perhaps compliance with other testing procedures as they may vary from country to country. That can include inhouse testing and third-party independent tests in labs. Now I have seen many of those torture chambers -- the ones of Panasonic, GD-Itronix and Intermec, to name a few. I've seen machines being baked, shaken, rattled, dropped, scratched, exposed to extreme humidity, vibration, pressure, materials fatigue testing and more. The tests are real, and they certainly reveal weak points that are then addressed.
Problem is that the reported testing results are not always very informative. MIL-STD testing means just that; a piece of equipment has been tested in accordance with the procedures mandated in a MIL-STD document. Often it is not reported what the outcome was, or if the machine even passed. Or only part of the test results are included in the specs. So prospective customers often do not have enough data to really compare. Some of the big companies in the field are guilty of not including truly meaningful ruggedness specs, and that doesn't do anyone a favor.
Sometimes we do go beyond simply describing a machine and administer our own torture testing. When Trimble/TDS claimed their Nomad handheld was waterproof to the extent that it would survive for an hour in a full meter of water, we decided to see if that was really so. I made sure they were okay with that. We used scuba gear and actually took it for a dive. I used it underwater and pushed the specs. The Nomad went down to maybe seven feet, it stayed underwater for a good while, and it survived. It worked underwater and I even used it for handwriting reco underwater. It's all on video and up on YouTube.
As a result, some manufacturers may be reluctant to send us their gear because -- hey -- those guys at RuggedPCReview may actually check the ruggedness specs for themselves. Others send us gear with the specific request to do so.
A current example: Toshiba makes a remarkable machine, the R500 notebook. It is an ultra-light and definitely not fully rugged. But it has an awesome outdoor-viewable display and was designed to take the kind of punishment that may occur on the road. I think a Toshiba rep called it "executive-rugged". The R500's display case is very flexible, so much so that we had our doubts if it'd hold up to any abuse. Well, Toshiba explained it was designed that way, and there is even a video showing the machine take abuse and the LCD being twisted to a frightful extent, and survive. We're tempted to see if we can duplicate that, but should we? The last thing I want to do is send the R500 back with a busted display.
For the most part, all this doesn't pose a dilemma. Most of the time the official test results are very clear and we see no reason to doubt them, nor would we have the ability to duplicate the torture testing. But the question does come up at times, and hence this column.
What we would like to challenge the rugged industry to do is this: State all ruggedness specs fully and clearly enough so readers will know what exactly the machine passed, and, more importantly, what it means.
Posted by conradb212 at November 15, 2007 2:43 PM