Do you define "reliabilty" like Consumer Reports?
Lexus Champion
Joined: Aug 2006
Posts: 12,540
Likes: 273
From: Illinois
According to this, my quick math shows there were about 130 million vehicles sold from 2000-2017
https://www.statista.com/statistics/...es-since-1951/
cr says they have 500,000 responses to surveys on vehicles from 2000-2018.
that's .4% of vehicles sold (peanuts) who also responded to a cr survey. I maintain cr's subscriber base is an inheritantly biased and unrepresentative sample... call it the 'middle class sensible consumer' demographic. I believe many will respond with a virtue signalling survey to show how smart they were in purchasing what they did, or trashing a purchase to make sure no other 'sensible consumer' makes the same mistake.
i also imagine the sample size of loads of vehicles they 'rate' are minuscule, and not meaningful. Imagine how many, i dunno, porsche 911 turbo reviews they get, or more topical, tesla model x. Tesla and porsche don't sell many to begin with and even fewer buyers read cr and even fewer of those bother to return a survey. So let's say cr gets 3 tesla x surveys (unlikely) and 1 owner had a horrible experience for whatever reason. Even if the other two had no issues, it's going to have a bad rating.
cliff notes: cr survey is worthless, and worse than worhless because people actually rely on it for buying decisions.
https://www.statista.com/statistics/...es-since-1951/
cr says they have 500,000 responses to surveys on vehicles from 2000-2018.
that's .4% of vehicles sold (peanuts) who also responded to a cr survey. I maintain cr's subscriber base is an inheritantly biased and unrepresentative sample... call it the 'middle class sensible consumer' demographic. I believe many will respond with a virtue signalling survey to show how smart they were in purchasing what they did, or trashing a purchase to make sure no other 'sensible consumer' makes the same mistake.
i also imagine the sample size of loads of vehicles they 'rate' are minuscule, and not meaningful. Imagine how many, i dunno, porsche 911 turbo reviews they get, or more topical, tesla model x. Tesla and porsche don't sell many to begin with and even fewer buyers read cr and even fewer of those bother to return a survey. So let's say cr gets 3 tesla x surveys (unlikely) and 1 owner had a horrible experience for whatever reason. Even if the other two had no issues, it's going to have a bad rating.
cliff notes: cr survey is worthless, and worse than worhless because people actually rely on it for buying decisions.
Lexus Champion
Joined: Jul 2013
Posts: 2,300
Likes: 4
From: RightHere
The Kardashian-Jenner celebrity clan could endorse a product tomorrow on various online platforms and you would see a huge spike in sales, marketing and ad hoopla about let's say product xyz and this wonderful widget abc. Now you know they don't do tweets, instagrams and facebook for shameless dollar shilling don't you? They merely wear, drive or are coincidently seen in ohh, various outfits, cars, SUVs and fashion brands while going about their daily business of being famous. It's not at all deliberately contrived in conjunction with a vapid tv show.
Does that mean a potential customer should be looking at a NPS metric? Or that 108 million Kim Kardashian followers who love her brand are more accurately reliable about any product she endorses vs some sort of independent consumer organization? Apparently Teen Vogue tells a potential buyer that they need to be seen in Range Rovers, Rolls Royces, Maybach and G-Wagons.
In return for this product "endorsement" they will tell you they love this "brand", and urge their followers to love this "brand". And eventually this cascades into that sort of buyer love of celebrity endorsed brands and cachet. So it's not reliable or durable? Who cares? If Kanye-Kardashian love it, so must I.
Nitpicking on a consumer-based organization is great sport for marketing, advertising and survey types. That's because they want money, and bashing something is easier than coming up with a better solution. Bashing methodolgy while serving up some sort of commercial, profit driven business motives is disingenuous to say the least.
Darryl CheapPants gives 4 star reviews to every survey he ever fills out (he gets a discount coupon and sometimes $25 too). He gives everything 4 stars, from his dentist's free toothbrushes to the fancy sandwhich he had recently. He's a good indicator of what's good and what isn't. Get enough Darryl CheapPants and you have a great product and the critics of your product are fewer than the promoters.
Half a million consumers who are members of a consumer based organization that doesn't focus on brands specifically but asks "what went wrong with your vehicle while you had it",are apparently, not.
Does that mean a potential customer should be looking at a NPS metric? Or that 108 million Kim Kardashian followers who love her brand are more accurately reliable about any product she endorses vs some sort of independent consumer organization? Apparently Teen Vogue tells a potential buyer that they need to be seen in Range Rovers, Rolls Royces, Maybach and G-Wagons.
In return for this product "endorsement" they will tell you they love this "brand", and urge their followers to love this "brand". And eventually this cascades into that sort of buyer love of celebrity endorsed brands and cachet. So it's not reliable or durable? Who cares? If Kanye-Kardashian love it, so must I.
Nitpicking on a consumer-based organization is great sport for marketing, advertising and survey types. That's because they want money, and bashing something is easier than coming up with a better solution. Bashing methodolgy while serving up some sort of commercial, profit driven business motives is disingenuous to say the least.
Darryl CheapPants gives 4 star reviews to every survey he ever fills out (he gets a discount coupon and sometimes $25 too). He gives everything 4 stars, from his dentist's free toothbrushes to the fancy sandwhich he had recently. He's a good indicator of what's good and what isn't. Get enough Darryl CheapPants and you have a great product and the critics of your product are fewer than the promoters.
Half a million consumers who are members of a consumer based organization that doesn't focus on brands specifically but asks "what went wrong with your vehicle while you had it",are apparently, not.
Last edited by MattyG; Oct 28, 2018 at 08:02 PM.
Lexus Fanatic
Joined: Sep 2010
Posts: 25,992
Likes: 4,312
From: Alberta
According to this, my quick math shows there were about 130 million vehicles sold from 2000-2017
https://www.statista.com/statistics/...es-since-1951/
cr says they have 500,000 responses to surveys on vehicles from 2000-2018.
that's .4% of vehicles sold (peanuts) who also responded to a cr survey. I maintain cr's subscriber base is an inheritantly biased and unrepresentative sample... call it the 'middle class sensible consumer' demographic. I believe many will respond with a virtue signalling survey to show how smart they were in purchasing what they did, or trashing a purchase to make sure no other 'sensible consumer' makes the same mistake.
i also imagine the sample size of loads of vehicles they 'rate' are minuscule, and not meaningful. Imagine how many, i dunno, porsche 911 turbo reviews they get, or more topical, tesla model x. Tesla and porsche don't sell many to begin with and even fewer buyers read cr and even fewer of those bother to return a survey. So let's say cr gets 3 tesla x surveys (unlikely) and 1 owner had a horrible experience for whatever reason. Even if the other two had no issues, it's going to have a bad rating.
cliff notes: cr survey is worthless, and worse than worhless because people actually rely on it for buying decisions.
https://www.statista.com/statistics/...es-since-1951/
cr says they have 500,000 responses to surveys on vehicles from 2000-2018.
that's .4% of vehicles sold (peanuts) who also responded to a cr survey. I maintain cr's subscriber base is an inheritantly biased and unrepresentative sample... call it the 'middle class sensible consumer' demographic. I believe many will respond with a virtue signalling survey to show how smart they were in purchasing what they did, or trashing a purchase to make sure no other 'sensible consumer' makes the same mistake.
i also imagine the sample size of loads of vehicles they 'rate' are minuscule, and not meaningful. Imagine how many, i dunno, porsche 911 turbo reviews they get, or more topical, tesla model x. Tesla and porsche don't sell many to begin with and even fewer buyers read cr and even fewer of those bother to return a survey. So let's say cr gets 3 tesla x surveys (unlikely) and 1 owner had a horrible experience for whatever reason. Even if the other two had no issues, it's going to have a bad rating.
cliff notes: cr survey is worthless, and worse than worhless because people actually rely on it for buying decisions.
Lead Lap
Joined: May 2010
Posts: 3,991
Likes: 1,616
From: San Diego, CA
The CR data set is enormous. What they said was they had responses for 500,000 vehicles in their latest survey.
To put that into further content, a poll with a sample size of just around 1000 people can reflect the views of the adult population of the United States of America with a 3% margin of error/95% level of confidence.
To put that into further content, a poll with a sample size of just around 1000 people can reflect the views of the adult population of the United States of America with a 3% margin of error/95% level of confidence.
Lexus Fanatic
Joined: Nov 2011
Posts: 35,436
Likes: 372
From: ON/NY
The CR data set is enormous. What they said was they had responses for 500,000 vehicles in their latest survey.
To put that into further content, a poll with a sample size of just around 1000 people can reflect the views of the adult population of the United States of America with a 3% margin of error/95% level of confidence.
To put that into further content, a poll with a sample size of just around 1000 people can reflect the views of the adult population of the United States of America with a 3% margin of error/95% level of confidence.
You don't think anyone on Youtube or here is going to be subject to the same issues you bring up about CR subscribers?
Lead Lap
Joined: May 2010
Posts: 3,991
Likes: 1,616
From: San Diego, CA
Also, this kind of information can't really be gathered at random. it's not trying to predict an election. It's just trying to get a better insight into actual owner experience. No one is compelled to act on it either as a respondent or as a reader.
Lexus Fanatic
Joined: Nov 2011
Posts: 35,436
Likes: 372
From: ON/NY
I agree it is not trying to predict an election, however when it comes to studies or scientific ones, the drawbacks of a study are very important in the overall big picture. You are right, nobody is compelled to act as a respondent, however those who are subscribers would definitely be a drawback. 500,000 participants is a great thing, but insight into the way CR comes to their conclusions is just as important as what those conclusions are. So would be the question of whether the study could be easily replicated etc etc The methodology is completely vague in this study.
Thread Starter
Pole Position
Joined: Mar 2014
Posts: 3,354
Likes: 555
From: fl
The Kardashian-Jenner celebrity clan could endorse a product tomorrow on various online platforms and you would see a huge spike in sales, marketing and ad hoopla about let's say product xyz and this wonderful widget abc. Now you know they don't do tweets, instagrams and facebook for shameless dollar shilling don't you? They merely wear, drive or are coincidently seen in ohh, various outfits, cars, SUVs and fashion brands while going about their daily business of being famous. It's not at all deliberately contrived in conjunction with a vapid tv show.
Does that mean a potential customer should be looking at a NPS metric? Or that 108 million Kim Kardashian followers who love her brand are more accurately reliable about any product she endorses vs some sort of independent consumer organization? Apparently Teen Vogue tells a potential buyer that they need to be seen in Range Rovers, Rolls Royces, Maybach and G-Wagons.
In return for this product "endorsement" they will tell you they love this "brand", and urge their followers to love this "brand". And eventually this cascades into that sort of buyer love of celebrity endorsed brands and cachet. So it's not reliable or durable? Who cares? If Kanye-Kardashian love it, so must I.
Nitpicking on a consumer-based organization is great sport for marketing, advertising and survey types. That's because they want money, and bashing something is easier than coming up with a better solution. Bashing methodolgy while serving up some sort of commercial, profit driven business motives is disingenuous to say the least.
Darryl CheapPants gives 4 star reviews to every survey he ever fills out (he gets a discount coupon and sometimes $25 too). He gives everything 4 stars, from his dentist's free toothbrushes to the fancy sandwhich he had recently. He's a good indicator of what's good and what isn't. Get enough Darryl CheapPants and you have a great product and the critics of your product are fewer than the promoters.
Half a million consumers who are members of a consumer based organization that doesn't focus on brands specifically but asks "what went wrong with your vehicle while you had it",are apparently, not.
Does that mean a potential customer should be looking at a NPS metric? Or that 108 million Kim Kardashian followers who love her brand are more accurately reliable about any product she endorses vs some sort of independent consumer organization? Apparently Teen Vogue tells a potential buyer that they need to be seen in Range Rovers, Rolls Royces, Maybach and G-Wagons.
In return for this product "endorsement" they will tell you they love this "brand", and urge their followers to love this "brand". And eventually this cascades into that sort of buyer love of celebrity endorsed brands and cachet. So it's not reliable or durable? Who cares? If Kanye-Kardashian love it, so must I.
Nitpicking on a consumer-based organization is great sport for marketing, advertising and survey types. That's because they want money, and bashing something is easier than coming up with a better solution. Bashing methodolgy while serving up some sort of commercial, profit driven business motives is disingenuous to say the least.
Darryl CheapPants gives 4 star reviews to every survey he ever fills out (he gets a discount coupon and sometimes $25 too). He gives everything 4 stars, from his dentist's free toothbrushes to the fancy sandwhich he had recently. He's a good indicator of what's good and what isn't. Get enough Darryl CheapPants and you have a great product and the critics of your product are fewer than the promoters.
Half a million consumers who are members of a consumer based organization that doesn't focus on brands specifically but asks "what went wrong with your vehicle while you had it",are apparently, not.
I initially brought this up because I see Tesla towards the very bottom of CR's "reliability" list, yet they have the highest NPS of any auto company, and they have been the highest in owner satisfaction in Consumer Reports every year since 2013. That makes no sense. Something doesn't jive here, and to me, it's CR's definition and/or methods of classifying something as reliable or unreliable.
Lexus Fanatic
Joined: Oct 2003
Posts: 94,422
Likes: 249
From: Virginia/D.C. suburbs
The fact that they are the subscriber base makes little or no difference. It is still the same vehicles, having the same problems, no matter who is reporting them, whether they subscribe to CR or not. Of course, opinions can differ.....one person can claim that something is a "problem" when another person does not (like if a video-system, for example, is difficult to use), but that will apply whether the person is an actual subscriber of CR or not.
Lexus Fanatic
Joined: Nov 2011
Posts: 35,436
Likes: 372
From: ON/NY
The fact that they are the subscriber base makes little or no difference. It is still the same vehicles, having the same problems, no matter who is reporting them, whether they subscribe to CR or not. Of course, opinions can differ.....one person can claim that something is a "problem" when another person does not (like if a video-system, for example, is difficult to use), but that will apply whether the person is an actual subscriber of CR or not.
Any study that is done with data will have limitations or drawbacks, these do not make the finding invalid per say. They actually can in fact make the findings or claims more valid or reliable.
Last edited by Toys4RJill; Oct 29, 2018 at 07:01 AM.
Lexus Fanatic
Joined: Jan 2005
Posts: 20,237
Likes: 274
According to this, my quick math shows there were about 130 million vehicles sold from 2000-2017
https://www.statista.com/statistics/...es-since-1951/
cr says they have 500,000 responses to surveys on vehicles from 2000-2018.
that's .4% of vehicles sold (peanuts) who also responded to a cr survey. I maintain cr's subscriber base is an inheritantly biased and unrepresentative sample... call it the 'middle class sensible consumer' demographic. I believe many will respond with a virtue signalling survey to show how smart they were in purchasing what they did, or trashing a purchase to make sure no other 'sensible consumer' makes the same mistake.
i also imagine the sample size of loads of vehicles they 'rate' are minuscule, and not meaningful. Imagine how many, i dunno, porsche 911 turbo reviews they get, or more topical, tesla model x. Tesla and porsche don't sell many to begin with and even fewer buyers read cr and even fewer of those bother to return a survey. So let's say cr gets 3 tesla x surveys (unlikely) and 1 owner had a horrible experience for whatever reason. Even if the other two had no issues, it's going to have a bad rating.
cliff notes: cr survey is worthless, and worse than worhless because people actually rely on it for buying decisions.
https://www.statista.com/statistics/...es-since-1951/
cr says they have 500,000 responses to surveys on vehicles from 2000-2018.
that's .4% of vehicles sold (peanuts) who also responded to a cr survey. I maintain cr's subscriber base is an inheritantly biased and unrepresentative sample... call it the 'middle class sensible consumer' demographic. I believe many will respond with a virtue signalling survey to show how smart they were in purchasing what they did, or trashing a purchase to make sure no other 'sensible consumer' makes the same mistake.
i also imagine the sample size of loads of vehicles they 'rate' are minuscule, and not meaningful. Imagine how many, i dunno, porsche 911 turbo reviews they get, or more topical, tesla model x. Tesla and porsche don't sell many to begin with and even fewer buyers read cr and even fewer of those bother to return a survey. So let's say cr gets 3 tesla x surveys (unlikely) and 1 owner had a horrible experience for whatever reason. Even if the other two had no issues, it's going to have a bad rating.
cliff notes: cr survey is worthless, and worse than worhless because people actually rely on it for buying decisions.
CR has different type of surveys, we are talking about Dependability survey which actually pretty much matches other non CR surveys around the world for car dependability.
Nothing is perfect but it is as close as we get.
Lexus Fanatic
Joined: Jan 2005
Posts: 20,237
Likes: 274
This dawned on me the other day. CR factors in all kinds of things that (in my opinion) have nothing to do with reliability into their scores. Why not look at the combo of their "owner satisfaction" plus something like a "net promoter score" which shows how likely a person is to buy again and become an advocate for the brand?
In essence, CR has brands at the bottom of their "reliability" list that rank at the top of "owner satisfaction" and net promoter score. These things contradict each other - and personally, I believe CR uses flawed methodology to determine reliability. You couldn't possibly have an "unreliable" car with high scores on "owner satisfaction."
In essence, CR has brands at the bottom of their "reliability" list that rank at the top of "owner satisfaction" and net promoter score. These things contradict each other - and personally, I believe CR uses flawed methodology to determine reliability. You couldn't possibly have an "unreliable" car with high scores on "owner satisfaction."
Lead Lap
Joined: May 2010
Posts: 3,991
Likes: 1,616
From: San Diego, CA
Actually it does make a difference. It would be considered a limitation of such a study. The fact that CR does not open their data for relevant criticism is the problem with their data.
Any study that is done with data will have limitations or drawbacks, these do not make the finding invalid per say. They actually can in fact make the findings or claims more valid or reliable.
If you're in the latter group, your opinion about something you don't have any experience of isn't exactly valuable or helpful.
CR are transparent about what they measure and ask. The results are shared at an aggregate level. This makes sense, as individual level responses reflect one person's experience. It's only in the aggregate that trends can be identified or predicted.
And that's all CR does - it predicts based on data received from actual owners. CR doesn't take advertising, so it already has an independence advantage compared to the enthusiast press that does accept paid advertising and has far greater conflicts of interests as a consequence.
