Software
Houzz Logo Print
jopopsy

Consumer Reports Reliability Data 2008

18 years ago

Hey Gang,

How's this for a good thread starter? In order of most reliable to least, according to CR:

Troy Bilt

Honda

Toro

Craftsman

Lawn-Boy

Yard Machines/Man

Snapper

John Deere

(John Deere was almost 2x as unreliable as the Troy Bilt)

Wow. Some of this flies against everything I've read on this forum .... particularly the Snapper.

Comments (35)

  • 18 years ago
    last modified: 11 years ago

    Consumer reports data on mowers is completely unreliable and bogus to put it mildly. The scientific methods we all learned in 7th grade science never crossed their desks. They are limited by money, time, reliable data input, and real world experience. Much of their input is from angry customers. Happy customers seldom bother to report data. Angry customers go all out to trash products and their manufacturers. Any organization that would rate Craftsman mowers in the middle is totally without a clue. Anyway..... I just ignore their idiotic ratings and continue to assess equipment like I always have. Real world use over a long time, combining reliability, price, support, and performance together with any other influencing factors all contribute to my opinion.

  • 18 years ago
    last modified: 11 years ago

    Ditto's.
    Keep in mind that Yard Machines/Man and Troy Bilt are for all intents the same stuff from MTD. Yet they have different ratings.

    I've told this truth before but its been a while. Back in the mid 80's I forever threw away CR for their bogus reports. At that time Snappers were only sold at servicing dealers, the mower were never intended to leave the sales department until they were assembled and serviced. Since that time Snapper has flirted with box stores sales but once again they are out of the box stores. There may be some left overs but new models shouldn't be found anywhere but service dealers.

    Anyway, back to this bogus mid 80's report. CR aquired a couple of Snapper mowers from some dealer who sold them to CR at a discount in the box. This was not suppose to happen. CR proceeded to downgrade the Snapper line heavily, primarily based on their testers having difficulty assembling the Snappers. Their testers were never supoose to have to do final assembly yet they raked Snapper over the coals because their idiot testers didn't know how.

    This is not unbiased, accurate testing.

  • 18 years ago
    last modified: 11 years ago

    Any subjective ratings Consumer Reports does are very questionable. Their car reviews have become completely worthless. No way to know what the difference is between their good and bad ratings, nor do they separate out major differences in drivetrains, options, etc.

    The fact is they do not have reliable data. But that doesn't stop them from pretending they do.

  • 18 years ago
    last modified: 11 years ago

    Consumer Reports reviews are probably as useful for for predicting which consumer goods including mowers will be reliable as the opinions on internet discussion boards. Forum opinions are typically limited in number and tend to be heavily weighted by brand loyalty. Or said another way, while a given source of information can be useful, it should not be the only basis for making a decision.

    If Consumer Reports consistently produces reports that draw the wrong conclusion and directs consumers towards problem laden machines I would think that consumers would have abandoned the magazine long ago. Lets face it folks - there has to be a reason for their incredible success. And it isn't because consumers are mindless drones.

  • 18 years ago
    last modified: 11 years ago

    I've been burned by CR a few times now. On the other hand the results make for entertaining postings. I'm seriously considering a Snapper w/ the Kawa engine in it b/c I wanted something rock solid.

  • 18 years ago
    last modified: 11 years ago

    roadbike I agree that opinions are a dime a dozen. However, to downgrade a product because it is difficult to assemble when that product was never intended to be assembled by the buyer is false reporting. Were they being deceptive or just plain ignorant of how the manufacture intends their product to be sold? Either way they have serious credability problems.

  • 18 years ago
    last modified: 11 years ago

    CR is only one source that can be used when making purchases. I have a question for those that feel their compiled results are skewed or biased. What SHOULD they report when they receive feedback from consumers? CR has no control over which consumers choose to fill out a survey and submit it, and which ones don't. If by simply reporting the numbers of failures, warranty issues, etc. experienced by consumers that have purchased products one brand/model is found to be more reliable or less reliable than another, should they NOT report the results they've received.....because it'll ruffle some feathers?

    The issue about difficult assembly is a valid one, but I'd think any consumer reading the report(s) and noticing the difficult to assemble comments would also have taken into consideration that the assembly was only one part of the products' equation. They may have even already visited their power equipment dealer and seen the assembled and prepped machines for sale.

    I'll ask again.....the reliability numbers CR reports are based upon data compiled from consumer-submitted surveys. Should they NOT publish those numbers because it'll ruffle some feathers? What would you do instead.....take the results from the submitted surveys, and then "arrange" the manufacturer reliability ratings based upon YOUR opinion rather than the hard numbers you have in front of you?

    I rarely consult CR myself when making purchases, but I have filled out their surveys in the past. I hope that the time I spent doing so wasn't a waste because someone with an agenda was disregarding the data I provided and instead substituting their own.....

  • 18 years ago
    last modified: 11 years ago

    broke not, the problem is they do not publish their survey results. They publish their interpetation of those surveys. What is the real statistical difference between a black circle and a red circle? Only CR knows, and they aren't telling.

    CR has also been caught numerous times doing flawed testing and giving some brands the benefit of any doubt.

  • 18 years ago
    last modified: 11 years ago

    Broke note, I think reader surveys have some use in evaluating a product. I believe some of CR's data is reliable but on the other hand I have seen bogus reports like their rating of the mowers as presented in the 1st post above. I think part of their problem is the summaries. They know that readers want to be able to glance to the end of the article, see what rates high, as they walk out the door to go but a mower somewhere. The rating summaries place horrible mowers unrealistically high and better mowers way to low in the ratings. I feel like these summaries are the weak point and should be taken with a grain of salt.
    To emphasize the point wouldn't it be interesting to see how they rated certain cars from the past to see how they have turned out now that decades have gone buy? Cars that we know to be horrible like the Vega, Pinto Wagon? Absolute pieces of crap that allowed brands like Toyota to get a foothold on American soil by building great 200,000 mile Corollas and Celicas when American cars were trashed at 60,000 miles. My point is that real world evaluation of products can't always be made from a few reader's surveys and a few days testing. Placing these ratings out there like they are gospel is good entertainment I suppose and interesting reading but the consumer is still taking a risk when buying a product and is sometimes actually mislead by the magazine as in the lawnmower ratings posted here. I'll take a Toro, Lawn-Boy, John Deere, or Snapper any day over an MTD product or heaven forbid a Craftsman.

  • 18 years ago
    last modified: 11 years ago

    CR's generic reliability ratings on Appliances are useful. Lawn Mowers don't work because the type of issue can vary greatly. A loose screw is a problem - as is a broken tranny - according to CR. If you want a broken tranny in 3-4 years buy the AYP (Craftman) or MTD (Yardman, Cub Cadet) crap. If you want a real lawn mower buy the high end Snapper, Toro, etc. and it will run for 20 years IF you maintain it. If you don't, it will have problems too. So if you want a disposal mower buy the cheap stuff and replace it often. Your call.

    A fridge requires no maintenance - hence a generic survey result is somewhat meaniful. Any problem will likely require a service call on appliances. Or a straight replacement - heck mircowaves are so cheap it makes no sense to service it, just buy another one.

    Jim

  • 18 years ago
    last modified: 11 years ago

    Lets not mix up CR reports. CR has two kinds of evaluations that they present.

    1. They conduct a product review that is pretty darned comprehensive considering the number of units reviewed. The format is consistent and the text while tending to be brief is informative. We have to realize that their tests are just like all other tests. Ultimately they are a distillation of test results and yes they are opinions. But for the average consumer looking for a source of information they are probably a better source of comparative information than brand-specific mower collectors or the guy who will only own a given brand.

    2. They present a summary of owner surveys. Such surveys are most useful when viewed over time. A one-time blip probably does not mean much. And yes I think CR is completely justified in not releasing the raw database of owner comments. Such a file requires interpretation and culling of errors. Tasks that most people are likely to screw up.

  • 18 years ago
    last modified: 11 years ago

    It is one thing to not include raw results in the magazine. It is quite another to not release them at all, or to even explain the methodology used in coming up with their ratings.

    Nobody can check their data (that I am aware of anyway), that makes it questionable. We are supposed to simply put faith in their rating method.

    Maybe they do have comprehensive data, but I usually find their reviews to be anything but comprehensive and tending to favor subjective opinion over objective facts.

  • 18 years ago
    last modified: 11 years ago

    Honda HRX217HXA $800 (86)
    Toro Super recycler 20092 $500 (83)
    Honda HRX2172VKA $700 (81)
    Lawn-Boy 10785 $460 (81)
    Honda HRR216K3VXA $580 (80)
    Lawn-Boy 10697 $500 (79)
    Lawn-Boy 10685 $370 (77)
    Lawn-Boy 10695 $450 (77)
    Toro recycler 20066 CR Best Buy $350 (74)
    Honda HRR216K3TDA $440 (72)
    Toro recycler 20064 $280 (69)
    John Deere JS35 $480 (66)
    Cub Cadet 12A18MC $270 (64)
    Craftsman 37077 $350 (63)
    Craftsman 37182 $480 (63)
    Husqvarna 7021RS $360 (63)
    Craftsman 37712 $400 (63)
    Snapper SPV21675E $380 ((62)
    Snapper SPV22675HW $300 (61)

  • 18 years ago
    last modified: 11 years ago

    "It is one thing to not include raw results in the magazine. It is quite another to not release them at all, or to even explain the methodology used in coming up with their ratings."

    I'm no more surprised that CR does not release their polling database than I am for any other poll. When's the last time you saw the underlying raw data for consumer or political polls released to the general public.

    And on a related note do you really think that the test data would be of any use to prospective purchasers of lawn mowers? It would likely be of more use to those who would like to shoot CR down for their success.

    However, have you taken the time to call CR and ask for a summary of how the little circles are populated? While they may not let you peek at the data they may describe weighting, trimming and averaging used to come up with those circles.

    "Nobody can check their data (that I am aware of anyway), that makes it questionable. We are supposed to simply put faith in their rating method."

    RB> Since when should their data be available for checking when no other consumer or political poll makes their data available for public viewing. Don't forget that this is not a new method of managing DNA published in the JAMA, but the review of consumer lawn mowers. When S&P or Moody's rates a company they don't open all their data files for public inspection. C'mon be realistic here.

    "Maybe they do have comprehensive data, but I usually find their reviews to be anything but comprehensive and tending to favor subjective opinion over objective facts."

    RB> That's not new news. Any review or test is in the final analysis an opinion.

  • 18 years ago
    last modified: 11 years ago

    I'm just wondering - What kind of professional assembly and servicing do Snapper mowers require? Most, if not all, of the OPE I have purchased for as long as I can remember (I am 71) only required removal from the shipping carton, folding the handle into position, the addition of oil and gas, and pulling the rope to start. What's the big deal about a Snapper?

  • 18 years ago
    last modified: 11 years ago

    Sigh...we've been over this in threads before, folks. I have hesitated to weigh in because I think using the "search" function should be required before starting a new thread on an old topic. But here I go anyway.

    CR purports to "report" on "research" they do of reliability and consumer satisfaction, etc. And Roadbike says he knows of no publication or poll that publishes raw data from their surveys. Well, if you are truly doing research then _EVERY_ publication publishes their test instruments (in this case, the survey itself); the procedures (who was sent surveys and how they were chosen, the return rate, any skew in sample of those that returned completed surveys, etc); the raw data; statistical analysis of the data (which then shows whether the results are statistically significant); the possible interpretations of that data (note that there is almost _always_ more than one possible meaning of survey data); possible errors; and limitations to the generalizability of the results. Oh, and all research begins with a review by a board, who serve as a quality check on the research methodology _before_ the research is published. CR has none of those, and never could because they could never get past that initial review board. Never. Not even close. They have zero control over their sample, no idea what industry averages for reliability might be, etc. They don't even account for the fact that most Snappers/JDs and high-end Toros are going to be in service for decades and so small issues are likely to be repaired, while a $250 MTD is going to be in service for a just a few years and simply be replaced when a small breakage occurs. It's a complete and utter joke. Keep in mind there are no national databases for auto repairs, much less lawn equipment, so there is no way to gather reliable data.

    This is the same crap that happens with supposed reports of automotive reliability. Ever notice, for example, that almost _all_ reliability surveys are for cars 90 days after purchase? Does that _any_ relevence to what you and I consider reliability to be? Have you ever checked out what the results of a 5-year reliability survey are? Check JD Powers and you'll find some suprising brands at the top of _that_ result, and some suprising names at the bottom (and that survey is far from perfect, but it's better than CR). I wonder what CR's mower surveys would look like if they surveyed mower owners after 5 years of ownership??? Remember the CR report that showed all the 5-speed automatic transmissions in Honda minivans and Accords failing? No, you wouldn't, because CR never did a report about that (neither did anyone else). That's because Honda never issued a recall and was able to keep it pretty well hidden. But it's a known issue, especially in the mechanic trade, but Honda keeps getting the highest marks from CR as being "reliable."

    And I haven't even mentioned the issue of the CR self-fulfilling prophecy: people read the 07 survey of mowers and tend to buy what CR recommends, so they are then of course biased to corroborate that recommendation when they get their survey. I've owned/used Snappers for over 30 years. I'd love to fill out a survey on them. In all those years, CR never sent me a questionaire. I wonder why? It must be because I don't pay CR to send me magazines in the mail. No bias there - nosiree!

  • 18 years ago
    last modified: 11 years ago

    mattv21, okay, that's a fundamentally sound argument. I knew I would enjoy getting this thread started. But back to your post, given what you know of mowers and comparing that to CR's results, would you buy a Snapper (not the value line) or a Honda Masters? Features aside, just going for a well made product?

  • 18 years ago
    last modified: 11 years ago

    xlindax
    Can't blame a youngin like yourself for not knowing. Prior to selling in box stores Snapper was not a quick assembly and for someone who hadn't done so before they were quite an edjucation to do so.

    The handles were not pre assembled and folded down using wing nuts and bolts already in place. Drive speed control and drive clutch controls were not flex cables but hard rods so they could not be assembled and folded for boxing. All these parts had to be assembled.

    The chute was not installed and the bag and its hardware had to be assembled. Also keep in mind since they weren't intended to be assembled by the average joe end user there was no instructions included.

    These things were changed when Snapper decided to try their hand at box store sales and have continued with those re-designs even though they have returned to being a service shop only sale. Imagine not knowing the insides of a computer, buying a new one and finding you need to install half of the innnard components with no instructions. That's about as close as I can think to put it in perspective with current times.

    Yet CR took these never intended to be sold in the box Snappers and slammed them for difficult assembly. See if you can find the old reports from 82-84 time frame for yourself if you want. You folks can argue the merits of opinion poles all you want but these are facts about CR's lack of creditbility, not opinion.

  • 18 years ago
    last modified: 11 years ago

    jopopsy, I have no direct experience with a Honda Masters. I am also admittedly biased as a life-long Snapper user. So take my opinons for what that makes them worth. My impression is that the Masters and Commercial Hondas are/were super-tough, well built, slick-running machines. I almost bought a used Masters once. They probably break rarely, and cut very well. The Snapper is not maintenance free, and will require adjustments and replacements of belts and such to keep working its best. It may require more frequent repairs, but on a Snapper those repairs involve parts that are cheap and easy to find and you can do the work yourself if you want. Unless the deck itself falls apart, you can _always_ keep a Snapper running. The Snapper is probably nicer to maneuver in tight places with lots of turns and pull-backs, and probably bags/vacuums better (though the Honda might do better bagging in the wet). For a similar-sized engine (IIRC, the Masters came with a smaller GVX140), the Snapper is probably a lot lighter than the Honda. I think the biggest issue would be price (comparing new to new - but they don't make the Masters anymore, do they?). The Honda would have been nearly double the price of a residential Snapper. And the Honda may require less frequent repairs, but I _think_ the parts and labor will be significantly more. So it would all depend on what you value most, and what kind of dealer support for each brand exists in your area.

    With the above said, I would defer to Nevada_Walrus or any other tech who has sold and serviced both brands. I promise they can give a better perspective than I can!

  • 18 years ago
    last modified: 11 years ago

    I believe the HRX line is considered the 'Master' line for Honda. I read that on another lawn site one time or another. Thanks for responding!

  • 18 years ago
    last modified: 11 years ago

    You said it well Matt.
    If that list of mowers someone put up are the models in the test, I believe those Snapper representations are the new box store design for Sears. These are not considered true Snappers by Snapper fans. Kinda like all modern Lawn Boys are not true Lawn Boys by their fans.

  • 18 years ago
    last modified: 11 years ago

    "Well, if you are truly doing research then _EVERY_ publication publishes their test instruments (in this case, the survey itself); the procedures..."

    RB> I'm hoping that you are not equating Research with consumer product testing. It is simply not true that every entity that publishes the results of polls and studies also publishes the underlying data. Where did you come upon that rubbish. If anything the exception would be the one that does publish such information.

    Lets also realize that CR is not a research organization in the sense that most of us understand that term. They don't do R&D work to create the latest and greatest lawnmower. They test the finished products of others in simulated working environments. And they report the level of owner (dis)satisfaction with a variety of mostly moderate priced consumer goods. Their goal is to give consumers the information to allow them to identify reasonably priced products that will get the job done.

    CR is not partnering with the NIH doing advanced research into genetic modification to solve cancer. They are testing lawn mowers. Somehow I think most of us understand that peer review isn't really needed in the latter.

    "They don't even account for the fact that most Snappers/JDs and high-end Toros are going to be in service for decades and so small issues are likely to be repaired, while a $250 MTD is going to be in service for a just a few years and simply be replaced when a small breakage occurs."
    RB> I do not get your point at all. CR is simply giving consumers a starting point. An MTD mower could be kept running for 20 years while a Honda could be trashed after 5 years depending on what the owner does or does not do for maintenance.

    I realize that it must be unsettling to think that someone would pick a $100 MTD mower over a $1,000 Honda, but there are a lot of people who see a lawn mower as nothing more than a tool to get a job done. They don't get excited about the decal proclaiming a Honda or Kawasaki motor. Nor do they care that it has 8 forward speeds or whether the valves are above or beside the piston. They just want it to turn long blades of grass into short ones.

  • 18 years ago
    last modified: 11 years ago

    "These things were changed when Snapper decided to try their hand at box store sales and have continued with those re-designs even though they have returned to being a service shop only sale. Imagine not knowing the insides of a computer, buying a new one and finding you need to install half of the innnard components with no instructions. That's about as close as I can think to put it in perspective with current times.

    Yet CR took these never intended to be sold in the box Snappers and slammed them for difficult assembly. See if you can find the old reports from 82-84 time frame for yourself if you want. You folks can argue the merits of opinion poles all you want but these are facts about CR's lack of creditbility, not opinion. "

    RB> Snapper is staffed by adults making adult business decisions. If they decided to market a lawn mower at an advanced level of disassembly while other lawn mower companies were more in tune with their customers, then Snapper should be held accountable. They should be criticized for selling a mower that would likely be difficult for most people to assemble. CR should not be soft-pedaling bad decisions like that. I think CR called it right.

    Maybe Snapper learned from that lesson.

  • 18 years ago
    last modified: 11 years ago

    as a life long snapper user also -i am not at all suprized by the last place showing in reliability. i take 3 to work every day so at least 1 MIGHT still be working at days end. worst sp desighn ever conceived yet very easy and cheap to fix. i would not use them if they didnt preform a certain task that no other mower can do

  • 18 years ago
    last modified: 11 years ago

    Lawnmowerdan: Wow, looks like Honda Master for me !!! When I pull the cord I want it to 'just work'. I don't mind a expensive repair every 4 or 5 years when I know everything will work in between episodes.

  • 18 years ago
    last modified: 11 years ago

    roadbike you are so wrong. Apparently you too are not familiar with the older Snappers, sad. The consumer models of the time had lighter duty wheels and did not have the handle bracing so they could support the weight of the commercial 13cid Briggs, but otherwise they were built just like the commercial models. Same drive and controls. Those controls were hard rods for shifting and clutching the drive. They could not be folded as cheap flimsey flex cables. They were nut and bolted with proper lenght bolts, not over long bolts to allow a hand knob wing nut to be loosened enough to stay on but fold the handle.

    Making the handles so they and controls can un-fold out of the box with only tightening a hand knob, no tools required make the modern Snapper less durable and stout then those older models. That more durable design of yesteryear is why they had to have assembly, but that wasn't of concern to Snapper because the consumer was not intended to do so. Why is it so hard for you to understand that! CR found some shady dealer to sell the mowers in a box for a discount to save money went into slam mode.

    Had I not worked for a Snapper dealer at the time of reading that report I would be as ignorant as the rest of CR's defenders. That or y'all think I'm a liar. All ya need to do is dig up the report for the time and or find an old Snapper and disassemble, reassemble and inspect it for design durability compared to todays models.

    But it's easier to call someone a liar isn't it! That's OK, I know the truth on this matter and if you wish to go through life in a fog that's OK with me.

  • 18 years ago
    last modified: 11 years ago

    "roadbike you are so wrong. Apparently you too are not familiar with the older Snappers, sad. The consumer models of the time had lighter duty wheels and did not have the handle bracing so they could support the weight of the commercial 13cid Briggs, but otherwise they were built just like the commercial models. Same drive and controls. Those controls were hard rods for shifting and clutching the drive. They could not be folded as cheap flimsey flex cables. They were nut and bolted with proper lenght bolts, not over long bolts to allow a hand knob wing nut to be loosened enough to stay on but fold the handle."

    You are missing the point completely! You wrote that Snapper marketed a mower through box stores that required a significant amount of assembly. That was a very poor decision likely the result of Snapper being late to the box store sales game. And it is an issue that most potential lawn mower buyers would want to know about. Snapper screwed up and should be held accountable. Most companies learn from such snafu's. Hopefully Snapper did.

  • 18 years ago
    last modified: 11 years ago

    Roadbike, of _course_ I'm comparing CR's reliability reports to those of other research journals. Gathering data, reporting on it, and analyzing it is what CR purports to be doing. What else should they be compared to? I am _not_ talking about their individual product tests - I'll get to that in a minute since you brought it up - but rather these data compilations from which CR expects its readers to draw conclusions. There are literally thousands of research journals in all kinds of fields, and their processes are standardized and transparent. And there's a reason: because you can't draw conclusions about sample data or generalize such conclusions beyond the sample if the research isn't rigorous, well controlled, and transparent. The fact that CR is sampling owners of mowers is no excuse for their piss-poor work, anymore than if they were surveying patients in a research study for medication compliance. The research is only useful if it meets research standards. Such things as interrator reliability (where different-branded mowers that are the same under the paint and made in the same factory should have the same result) matter. If the reports aren't reliable or valid, then what is the point of buying the magazine!?! Why would _anyone_ buy CR if they didn't think the info in it was reliable or valid? That would just be plain stupid. But it's not, and all the advertisement of their _journalists_ in white lab coats won't change that.

    Speaking of their product tests, I recall one test of an Echo blower that was tested in some standardized way (ANSI? OSHA? I forget) and reported with some sort of decible rating for that lab test. CR purported to replicate the test and got a louder result, and called Echo liars in writing. Unfortunately, they didn't know all the original test procedures and therefore couldn't possibly replicate the standardized testing. But that didn't stop them from smearing the Echo product all to hell and back. Even after Echo wrote a letter protesting the incorrect procedures, CR wrote a rebuttal saying "Echo does have a choice: tell the truth about their product!" That not only shows a lack of willingness to listen, but also a fundamental misunderstanding of proper testing procedures.

  • 18 years ago
    last modified: 11 years ago

    Roadbike, _you_ are missing N_W's point: the Snapper that was sold in the box store was substantially modified to allow consumers to assemble it. But that was not the mower that CR tested. CR tested the dealer-only model that required a lot of tools, time, and some training to assemble; and it was _never_ sold to consumers out of a box at any time, ever. Yet they reported on it as if it was, which completely misrepresented the product and turned what was a big asset of Snapper Hi-Vacs (dealer support that comes with the premium price) into a supposed liability.

  • 18 years ago
    last modified: 11 years ago

    roadbike I think you are mixed up. Nobody said Snapper marketed a difficult to assemble mower through a big box store. What was said is that CR bought a mower that was supposed to be assembled by the dealer that sold it to them.

    And you have missed my point. You have absolutely no idea what the difference is between a red circle and a black circle. Something that is below average is not necessarily bad. But CR gives you no way of figuring this out.

    You can justify their methods any way you want, but it doesn't change the fact that they can not be used as a reliable reference for any product comparison. That is the point I am trying to get accross. Sure, for some oddball products that nobody else tests CR can be useful. But people need to understand their ratings are NOT as comprehensive as CR would like them to (and leads them to) believe.

  • 18 years ago
    last modified: 11 years ago

    yes i use the comm models-the only advantage is steel wheels and better handle bracing. the drive sys is the same sorry pitifull desighn . hear is where snappers fault lies. this is snappers ONLY fault. they are as good as any other mower and better than most in every other way. the drive sys looks like something from the mouse trap kids game and is about as durable

  • 18 years ago
    last modified: 11 years ago

    "And you have missed my point. You have absolutely no idea what the difference is between a red circle and a black circle. Something that is below average is not necessarily bad. But CR gives you no way of figuring this out.

    You can justify their methods any way you want, but it doesn't change the fact that they can not be used as a reliable reference for any product comparison. That is the point I am trying to get accross. Sure, for some oddball products that nobody else tests CR can be useful. But people need to understand their ratings are NOT as comprehensive as CR would like them to (and leads them to) believe."

    Most CR readers are looking for a quick summary, much like a road test in one of the car magazines. The reality is there is no other comprehensive source of comparative information about consumer goods that even comes close to the kind of work CR does. None. Most CR readers are better off with summaries from CR than they would be with no information before trying to sort out the choices.

    It's worth understanding that most CR readers don't have the training to get into the methods and numbers behind the charts and text as you suggest. Since you appear interested, have you asked to speak with someone in either their testing or statistical analysis staff? While they may not share the data I'm sure one of the numbers guys would be happy to talk about processes at some level. Let us know what you find out.

    And just so that there is no misunderstanding about the Snapper issue, here is what was said by Mr. Walrus: "Anyway, back to this bogus mid 80's report. CR aquired a couple of Snapper mowers from some dealer who sold them to CR at a discount in the box. This was not suppose to happen. CR proceeded to downgrade the Snapper line heavily, primarily based on their testers having difficulty assembling the Snappers."

    That is a Snapper issue and not a CR issue. And it happened what - 25 years ago???

  • 18 years ago
    last modified: 11 years ago

    You weren't addressing me in your last post, roadbike, but I'm goint respond to your question about asking CR their methods. I have corresponded with a CR tester/journalist. For the most part, he wouldn't say _what_ methodology and standards are in place. All he would do is provide the same rebuttals you are doing. What process he did provide (this particular discussion was in regards to their automotive "tests") confirmed that there is very little rigor or control in any way. IOW, their methodology is all over the map, not very standardized, not in line with any recognized formal test procedures, etc. In short, they aren't carrying out testing at all - just editorials.

    If CR did tests that provided very little info with occassional useful facts coming out, and told readers up front that they weren't conducting research and their results had absolutely no generalizability to products in the store, I'd say "Okay, what's the harm." But the fact is that CR's readers are exactly as you describe them: generally "regular" people without a background in research who have no idea that CR's results are nearly irrelevent to reality. And yet CR represents itself as an objective source of reliable and valid product research. So it is misrepresenting itself to customers who don't have the background to know the difference. Beyond that, it provides _misleading information_ about products, probably on a fairly regular basis. I mean, I don't personally jack about can openers, so if they do a test of can openers I can't pick it apart. But I know enough about mowers to know that their comparison surveys are completely bogus; their reliability reports on automobiles are ridiculous; and their advice on hi-fi equipment is just plain wrong. That is much worse than not providing much useful info - it is making the public _more_ uneducated about entire product classes!

    And don't even get me started on automotive journalism! There is some pretty sad crap being published these days, and lots of payola/bribery deals going down in the publishing world. Go research how Car and Driver determines their "Car of the Year" award and get back to me.

  • 18 years ago
    last modified: 11 years ago

    Looks like this thread has died down, and I'm just a visitor (didn't even know this forum existed until today) but here's a nugget about Consumer Reports from somebody who's old enough to remember where the rot set in. If your library has back issues, go look at the multi-factor ratings for two cars: Mitsibushi Eclipse and Eagle Talon. The former, being Japanese, was rated high -- the latter, being American, was rated lower. The only problem was that they were both built in the same factory, with different final badging and trim -- otherwise the same car. So for whatever reason readers aren't being objective, or fairly sampled, or something when these ratings are compiled. (I know a brilliant guy at work (not being sarcastic) who offered me a ride one day...he was telling me how he was about to buy a newer Japanese car because his had been so outstandingly reliable...but when I saw his car 3 of the windows had been duct-taped to the roof because, he said, otherwise they'd fall into the door interiors. Sometimes people believe the myths of quality more than their own observations -- and there's one explanation for CR's ratings.)

  • 18 years ago
    last modified: 11 years ago

    stever sl - I am absolutely with you on this one. I work at a Ford factory and despite the fact that we recently bested Toyota in quality ratings, I find it difficult to get friends (even family!) to consider buying Ford. The perception that Japanese or German engineering and quality as being top notch is so deeply rooted in today's consumer culture that it seems everyone has a bias against American produced goods. The people that review products are certainly tainted by this stereotype as well.
    I am sure the Honda mowers are fine machines, but I feel that they have an immediate advantage going for them when the tester walks up and sees "Honda" on the engine or nameplate. I bet he immediately has the thought in his head that this will be a well built mower.