I no longer subscribe to any magazines that offer reviews of home theater-type equipment and I don’t generally read online review sites either. Why? Because, I’m sick of reading worthless and inaccurate information and that doesn’t help me pick a product and understand its limitations. I’m convienced the only way to get a useful opinion is through aggregation of forum posts, where multiple individuals can give multiple opinions to allow once to reach their own, overall consensus. This methodology is reinforced by the book The Wisdom of Crowds (Why the Many Are Smarter Than the Few) by James Surowiecki. Paperback link, Audio book link . But I have some additional reasons.

First of all, I never read any strongly negative opinion in a magazine review. Then I’d buy a product, find a major design annoyance, go to an Internet forum/message board such as AVS Forum and find others with the same exact complaints. Perhaps the magazine authors are trying to please the advertisers? The opinions posted on forums do not generally any tie to the banner add at the top of the forum page, and that is a positive.

Secondly, the magazine authors don’t always seem to have a full understanding of what they are testing and what the results mean. For example, this article in Home Theater Mag says that the measured brightness of the projector was 27 ft-lamberts. For the non-engineer, this probably doesn’t seem like a problem. Here’s the issue: this unit measures the brightness of the projected image on the screen, which is a result of projector brightness AND screen size AND screen gain ratio. So, the ft-Lambert result doesn’t directly describe the performance of the projector, which is the focus of the review. How would a person know if that projector is any brighter than any other projector? It’s like saying “this flashlight is blindingly bright” without saying whether it is 1 inch or 1000 feet away from your eye. I used my home theater calculator to reverse-calculate the projector ANSI brightness (based on the screen the tester used) and posted it in the corresponding Engadget entry so others could maybe make sense of it. It’s beyond my why the author didn’t include it in the article. Check the box off for reporting brightness and move on, regardless whether it is usable by the reader. At least they are not covering up a flaw of the projector.

Unfortunately, I have an even worse example: the review of the Sony HS-51 projector conducted by AV Review. The author states that the projector has a resolution of 1366 x 768 and that Sony erroneously lists it as 1280 x 720. In general, manufacturers don’t under-promote resolution, so I e-mailed the author to see how he reached this conclusion. To paraphrase, he said he “used his professional-grade test equipment to send a 1280×720 test signal to the projector and it did not fill the screen, so it must have more pixels than 1280×720 and 1366×768 seems reasonable.” The guy was dead wrong. It wasn’t that the projector had more pixels, it was that it was blanking out the pixels it had, ignoring the outer portion of the signal being sent to it. So when you watched something at 1280×720, you would only get 1250×690. It would not fill your screen edge-to-edge (black bars all around) and you would not see the full image. This was a major design bug and Sony eventually issued a recall. I gave the author and the editor more-or-less conclusive proof of this and yet the article remained unmodified. Don’t say anything bad about Sony, don’t inform customers, and don’t ever question your test equipment or methodology.

The one exception I have at this point in time is the website Audioholoics. I’ve found their reviews to be extremely detailed and adequately critical. Here’s hoping they maintain this position.