How are product reviews like good martinis? Both are best when dirty. That's clearly your feeling (at least when it comes to reviews). An overwhelming portion of you support our testing of products on dirty, well-worn PCs--meaning they're full of Windows detritus, just like yours and mine--instead of pristine, clean PCs, and let the chips fall where they may.

But here's the rub: No two boxes are dirty in the same way. This issue cropped up when we tested Pinnacle Studio MovieBox DV, for example. It did great in our tests. But fully 86 percent of CNET readers rated it el crappo. I supected that maybe our test system was a bit too clean, giving our PC an unfair advantage over your real-world ones.

Well, after investigating the matter, I found that the disparity in performance had more to do with a firmware revision than with the condition of our test bed. (In other words, we were testing a later version of the product, while you probably stayed up for three nights trying to make the earlier, buggy one work.)

But that doesn't discount that Achilles' heel of testing PCs: no two PCs are alike. These aren't standardized TVs or cell phones that can simply be reset to their original, unsullied states. A PC is like a child (and often a very rotten one); it's shaped by its earlier experiences. So even when you have the latest hardware, firmware, and drivers, you may still get stung by an otherwise good product because it conflicts with your system's various past-life experiences.

How do we control for this? I told you I had the ear of our Labs team. Here's what Daniel Begun, manager of CNET Labs, had to say:

"Testing software on a dirty system is an excellent idea, and it is something we endeavor to do whenever feasible. However, it's important to note that it is next to impossible to standardize on what exactly a dirty system should have installed on it. Applications and drivers are frequently updated by manufacturers, and users typically install and uninstall applications over time that leave varying amounts of detritus behind. One user's flotsam and jetsam are another user's mission-critical application files. As a result, any testing done on a dirty system needs to be relegated to anecdotal testing only--in other words: your mileage may vary."

Well, that's a start. Daniel has promised me he'll have his team come up with a formal dirty-box policy, and over the last week, they've been deep in a vigorous dialogue about it. I'll run that policy by you as soon as they draft it. But I can tell you already that sometimes your machine won't have the 11 herbs and spices needed to make even the best-rated add-on work with it. If cars were like that, I'd be feeding my horse right now.

Get your story straight
My wife, her sister, and my mother-in-law. Sounds like the start of dumb bar joke, but no. The three of them are going to Italy in a few days and want to make sure they'll be able to place cellular calls in the country, as well as back to the States. So Kitty (mother-in-law) calls her provider, T-Mobile, to check on its overseas rates. A customer rep tells her that any calls from Italy made back to her home market (New York) will be deducted from her plan as regular minutes. That's quite the little deal.

Now, I love a company that gives its customers choice, but a choice of facts?

But Kitty's an Italian from Brooklyn, so she mutters sono cazzate and calls me, the son-in-law who works for CNET (as opposed to the other one, who makes dentures). Posing as her husband (an unsettling but useful premise), I call T-Mobile, and they describe to me a very different rate structure: $1.19 per minute from Italy to anywhere in the United States. Feeling we need a specialist, I call in Joni Blecher, who gets under the hood with T-Mobile's people and finds out that the rate is really 99 cents a minute.

Now, I love a company that gives its customers choices, but a choice of facts? I bet you've been there. We have to get these companies to simplify this stuff. I don't think they're evil, I just don't think they understand their rate plans either.

Oh, wait there's more--check out this other way your cell phone can bite you in the ass.