Some hams depend on Lab reports and ratings to make purchase decisions. If they are to be believed any time the rankings from one of the Labs changes that they quickly order the new top dog and sell off their present radios.
What a load of crap. Few actually change their radios each time the latest lab results are published, even fewer will argue that that ratings have any impact on user experience (you can only hear to certain limits), and even less will argue that the lab result is important enough to offset every other aspect of the user experience.
Reality is everyone wants their bit of kit to be the top rated one, and they don’t like it much when some other kit gets rated higher.
I’ve written before about the bias of the various labs – whether the lab is part of an organization also selling advertising to the manufacturers or whether they are seeking other forms of payment. An economically sponsored lab is encumbered by the expectations of the sponsor.
Bias issues aside, what are the lab really measuring? If the limits of another part of the experience, whether the human factor or other parts of the audio/RF chain have their own limits overriding any benefits, does the measurement matter?
These questions came to us as long time ago in terms of HiFi gear and Photographic performance measures.
Several of the Labs have gone into print (well more like published opinions on the internet) that functionally the top say 30 radios they have tested all perform to a level that the differences between these top 30 or so radios have no impact on the user’s experience.
I’d further argue that for the typical ham there are performance differences between most of the recognized good radios that the operator compromises and operating environment conditions overwhelm. If you have a very limited antenna, ground and station RF silence situation, coupled with average operator’s techniques it won’t matter if you have a solid good radio or the top radio in lab testing.
Sometimes a radio that the lab may feel is compromised may actually provide a “better” user experience.
I’ve written before about “balance” in equipment – last reposted in 2011 based on a 2007 original post – https://k9zw.wordpress.com/2011/03/12/repost-well-balanced-machinery-how-some-machines-just-work-and-feel-right/
The same applies to the radio operator’s experience, certain radios “just work good” whether the lab reports champion them or not.
Unfortunately the lab “measures” more than it “evaluates” – so if you are a statistics junkie lab reports are defining, but it you are an actual equipment user the lab reports always fall short.
Some items labs don’t test that make a difference in user experience include:
- How easy is the radio to set up
- How easy it is to “fine tune”
- Are controls sensitive, too coarse or overly sensitive?
- Are features usable?
- Are compromises a fair trade off when costs are considered?
- Is the audio good for long periods of listening?
- Is the transmit audio effective?
- Are the menus easy to understand and navigate?
- How easy is it to return settings to an out of box/factory set up?
- Is the physical arrangement comfortable?
- And many more…
As so many of these non-lab measurements are “Goldilocks Evaluations” (the too warm, too cold, JUST RIGHT sort of evaluations) the pure lab-type numbers are a poor measure. It is actually better to let groups of users respond to a survey type of measurement to get a ranking.
And we have to remember that while it may be a “Goldilocks Evaluation” few of us are true Goldilocks persons, so YMMV (You Mileage May Vary) applies. If you have great vision and manual dexterity you may prefer a radio that would rate poorly for a vision impaired ham fighting hand arthritis. The features that matter for a contest station may not even matter for a Sunday morning net sort of ham. When I lived in Britain I often heard “Horses for Courses” reflecting that some tracks favor certain types of horses. The 2019 Kentucky Derby became so wet as to favor a “mudder.”
In the end Lab Reports can be used to assist our decision making, but shouldn’t replace our efforts at making good equipment choices for ourselves.
73
Steve
K9ZW
With Varying Frequency - Amateur Radio Ponderings 
One of the top rated evaluators, whose adherents pay attention to as divine word from above, once tested a used Flx 6300. It didn’t perform all that well, as it turned out, the radio had a defect. But he published the results. To this day I hear the true believers uttering the numbers from a broken radio as gospel truth. It was at that point I decided that radio evaluations are exactly what you say they are. Radios of unknown provenance, tested by peopel with human outlook, perhaps even the Ford Versus Chevy adherents.
I take these reports with a hugge grain of salt, while so many others treat them as scripture.
[…] “Some hams depend on Lab reports and ratings to make purchase decisions. If they are to be believed any time the rankings from one of the Labs changes that they quickly order the new top dog and sell off their present radios. What a load of crap…” […]