More ammo for my vendetta against Consumer Reports
Jonah Lehrer’s recent article in Wired.com perfectly illuminates one of my pet peeves: Consumer Reports magazine. I am a longtime subscriber and I do like to flip through each issue when it arrives, although unlike my father I don’t save the back issues just in case I might buy something in the future — after all, you know how I feel about material possessions. No, I flip through each issue mainly because I like to look at the pictures of car interiors. And because I think it’s a good idea to be somewhat informed about consumer issues. And because, I have to say, it’s fun to read the bad reviews, especially of the weird products that they sometimes highlight.
However, I’ve been arguing for years that the ratings themselves are useless, or even worse than useless. The problem is that raters have to find a reason to distinguish between products that are often nearly identical. I remember reading an article about shampoo that began by stating that all shampoos clean your hair equally well. Then it went on to rate a bunch of different brands… and I ask you, if they all clean your hair equally well, how do you possibly rate them? The only way they can come up with rankings is by focusing on distinctions without a difference. And then we read about those differences and get all het up thinking oh I couldn’t possibly buy that shampoo because it has a twist top instead of a flip top, oh those stupid manufacturers.
Lehrer’s article describes a study where college students sampled strawberry jams. Mostly, their preferences agreed with Consumer Reports’ rankings. But when they were asked to explain why they preferred one to another, it all went haywire. It turns out having to explain and analyze and look for reasons to choose one brand over another seriously affects the actual choices. The researchers concluded that “human reason has nothing to do with finding the truth, or locating the best alternative. Instead, it’s all about argumentation.” Says Lehrer:
Ever since the Ancient Greeks, we’ve defined ourselves in terms of our rationality, the Promethean gift of reason. It’s what allows us to make sense of the world and uncover all sorts of hidden truths. It’s what separates us from other Old World primates. But Mercier and Sperber argue that reason has nothing to do with reality. Instead, it’s rooted in communication, in the act of trying to persuade other people that what we believe is true. And that’s why thinking more about strawberry jam doesn’t lead to better jam decisions. What it does do, however, is provide up with more ammunition to convince someone else that the chunky texture of Knott’s Berry Farm is really delicious, even if it’s not.
Indeed.
Same thing could be said about book reviews: the ratings, generally, are meaningless. If you liked both books, why is one a four star and one a five star? Makes no sense. I prefer to know about an item/book/movie rather than just a rating.
Oy. Yes. Big pet peeve.
I let my online subscription to Consumer Reports end a few years ago. They asked me why, and I e-mailed that their coffee maker article had not warned me that the model I bought would not keep the coffee hot (so much for the “thermal” carafe), would always drip when poured, and was easily broken: within a couple of months, a flimsy latch-mechanism had broken off, ending the convenience of pouring of a cup while brewing was still under way. Even the digital clock was illegible. (Which may be another whole topic: Why do my toaster, stove and coffee maker all have built-in clocks?)
I think the CR coffee maker article had focused on how quickly the brewing cycle was completed, or whether the fill lines on the side of the machine were easy to read. They may be too wrapped up in their lab work and have forgotten what really matters to a consumer.
And really, you just can’t mess up on the coffee maker evaluation. It’s used every day, first thing, by people who are cranky. Myself included.
Melissa, yes! Although I think ratedreads.com does a pretty good job… ;-)
“They may be too wrapped up in their lab work and have forgotten what really matters to a consumer.” Sara, so true. In all the articles I read about computer printers (I was obsessed with them for quite a while) I have never seen ANY mention of the fact that inkjet ink is water soluble, i.e. you probably shouldn’t use one to print legal documents, or envelopes. Kind of a large oversight, I think.
On my dermatologist’s advice I’ve been using moisturizing creams. About these also CR says they are all about equally effective at moisturizing. But the overall ratings differ for good reason: one of these creams hides and retains ca. 1/8 of its stated net weight inside the diabolically designed tube so as to make retrieval impossible. I have to cut off the bottom of the tube with scissors in order to pry out numerous remaining doses of this shamelessly priced — $10 a tube (production cost estimated at 17 cents) — medium. Others, e.g., Cetaphil are sold in a plain container, and cost less to boot; you can get out every last tittle of cream. I believe this lotion jolly well deserves a higher rating for that alone even though other things are equal.
Archie, that is a bummer. But for me, if I had read that article, then what? I would say oh, great, I want Cetaphil. And then I would have to go find it. I would either have to call around to Kroger and Target and CVS to see if they have it — itself a frustrating process with the “press 2 now” & the waiting on hold — or drive around from store to store looking. Is the extra time, frustration, and mileage worth what you save on one lousy tube of cream? Especially knowing that all moisturizers work the same?