Topic: Autos.ca Comparison Tests Steering Committee

[Northernridge]

No video. Would rather read and look at great pictures. More, better pictures that compliment the text.

[Weels]

No video. Would rather read and look at great pictures. More, better pictures that compliment the text.

For video, i think it depends on what's being tested.  For the 11 car compact comparo, would video interest me?  Not really.
But for more 'exotic' tests, i'd love to see short 2 minute-ish videos to give you a sense of what's it's like to be inside one.

[Northernridge]

Video would have to be good. Inter web is bunged up with shite video. So that means track footage or really well done review footage. Don't need talking head footage. Autos.ca would have to have car expertise AND video production expertise (and the budget).

Hard to pull off. The do it yourself approach will fail.

[JacobBlack]

How hard can this be?
Just rate each vehicle based out of five like all the test drives you are doing right now.

Assuming that 2.5 is average or the low mark and 5 being the top mark.
Basically just so that we as the audience can separate them into tiers.

so who rates each car? the author for that write-up? then we'd have cars in 7th with higher scores than cars in 4th depending on writer's subjective opinion of assigned car... it has to be a collective score, and we're talking about adding work here, which ain't free... but yeah, that star rating system is the obvious answer - just have to figure out how to translate our score sheet averages into that system, although because if its more limited broader categories, might still have a lower-scoring car receiving higher ratings.

I thought everyone would rate each car and you came out with an average star value out of five

And this way you could separate to individual results as requested above...if you wanted.

That is way, way too simplistic. Our scoring is much more in_depth than that. All you'd get  is  8 cars scored 3.15 and one 3.7.

[jyarkony]

How hard can this be?
Just rate each vehicle based out of five like all the test drives you are doing right now.

Assuming that 2.5 is average or the low mark and 5 being the top mark.
Basically just so that we as the audience can separate them into tiers.

so who rates each car? the author for that write-up? then we'd have cars in 7th with higher scores than cars in 4th depending on writer's subjective opinion of assigned car... it has to be a collective score, and we're talking about adding work here, which ain't free... but yeah, that star rating system is the obvious answer - just have to figure out how to translate our score sheet averages into that system, although because if its more limited broader categories, might still have a lower-scoring car receiving higher ratings.

I thought everyone would rate each car and you came out with an average star value out of five

And this way you could separate to individual results as requested above...if you wanted.

so another separate rating sheet of those broader categories that we do publish... but because of the broader categories, we still might end up with ratings that conflict with the results of our very detailed and specific scoresheet that we feel reflects the priorities and needs of the consumer in that market....

[Black Hatch]

How hard can this be?
Just rate each vehicle based out of five like all the test drives you are doing right now.

Assuming that 2.5 is average or the low mark and 5 being the top mark.
Basically just so that we as the audience can separate them into tiers.

so who rates each car? the author for that write-up? then we'd have cars in 7th with higher scores than cars in 4th depending on writer's subjective opinion of assigned car... it has to be a collective score, and we're talking about adding work here, which ain't free... but yeah, that star rating system is the obvious answer - just have to figure out how to translate our score sheet averages into that system, although because if its more limited broader categories, might still have a lower-scoring car receiving higher ratings.

I thought everyone would rate each car and you came out with an average star value out of five

And this way you could separate to individual results as requested above...if you wanted.

That is way, way too simplistic. Our scoring is much more in_depth than that. All you'd get  is  8 cars scored 3.15 and one 3.7.

Overall I can see scoring for vehicles almost the same but individual areas like performance, comfort would probably be different on the difference cars no?

I think it's fine to see 8 cars at 3.15 and 1 car at 3.7.
You still have one higher than the rest and the others are close.
You can still rank them based on your more elaborate "internal" scores.

[Snowman]

Keep It Simple.

[MKII]

that we feel reflects the priorities and needs of the consumer in that market....

IMO this statement you wrote sums up why your comparison result is nothing more then very very very subjective opinion.

It is quite obvious each manufacturers vehicle is purchased by a consumer who has particular priorities & needs.
It is hardly relevant for your testers to assume the roll of deciding what each of those consumers priorities & needs are and use
it in your test data to justify  your end results.

[SaskSpecV]

I'm not interested in score sheets. Unless you guys are attaching instrumentation, everything is subjective anyhow and that's perfectly fine because the reviews are the impressions of the authors which is what I'm looking for. I'm also interested in who says what, don't obscure individual opinion, emphasize it…especially where the internal opinion differs…or state when it's unanimous for that matter.

For comparos I need to know WHY you like one car over another, not just that you think it's better. My priorities may differ from your own so I can live you drawing a different conclusion than me, but it helps to know why…or all I can do is disagree and not understand your conclusion.

The most valuable thing you guys can offer is relative comparison, you get to drive all the cars. I don't need you to tell me what I think of a car but it's very helpful (and interesting) to see what you have to say about a car relative to its competitors, previous models and the many others you have driven. I can not replicate that.

I also appreciate personal impressions much more than describing spec sheets and option lists, I can look those up. I want to read your impressions.

That summarizes it for me.  A scoring scheme is fine, but because all the numbers are arbitrary (like every other car comparison), it doesn't mean much.  It's the qualitative differences between vehicles, not the quantitative differences.  But I would like to see how each reviewer ranked the competitors - i.e., is the new CTS really polarizing, with 3 1st place votes and 3 5th place votes?  While the E-class got 6 3rd place votes?

Just the fact that autos.ca is soliciting suggestions for this is awesome.  Sure don't see that anywhere else!

[JacobBlack]

I think this whole debate shows one thing very clearly:
A single scoring system and scorecard on its own is virtually meaningless.
What is meaningful, are write ups which explain why each vehicle fell where it did overall.
I'm a little disappointed that most of the information people say they want, like "why did the Jetta beat the X" is actually in the article. We even mention which categories and segments each car did well and badly in.
So really, a simple average of each segment would probably be satisfying, and I think that's achievable. Still sad though that much of the info people say they want, actually exists already.

[Northernridge]

I think this whole debate shows one thing very clearly:
A single scoring system and scorecard on its own is virtually meaningless.
What is meaningful, are write ups which explain why each vehicle fell where it did overall.
I'm a little disappointed that most of the information people say they want, like "why did the Jetta beat the X" is actually in the article. We even mention which categories and segments each car did well and badly in.
So really, a simple average of each segment would probably be satisfying, and I think that's achievable. Still sad though that much of the info people say they want, actually exists already.

Don't think people meant it as a criticism but rather an expression of what they value...which means that you are delivering. I understood the original question as an inquiry into what we want, not what's wrong with the article(s). Maybe you should Google, How to accept a compliment and not think everything is a criticism.    That's a criticism. 

[Snowman]

I think this whole debate shows one thing very clearly:
A single scoring system and scorecard on its own is virtually meaningless.
What is meaningful, are write ups which explain why each vehicle fell where it did overall.
I'm a little disappointed that most of the information people say they want, like "why did the Jetta beat the X" is actually in the article. We even mention which categories and segments each car did well and badly in.
So really, a simple average of each segment would probably be satisfying, and I think that's achievable. Still sad though that much of the info people say they want, actually exists already.

I disagree, its not meaningless. If the scoring system is properly set up and based on fact it can be be a good tool for buyers or readers in general to highlight strengths and weakness in an objective format. I'm not sating its the only method to rate a car but it is very helpful. If I see a high or low rating in a particular category I want to know why. That is useful information IMO. Tangible and intangible scoring both have a part in the evaluation process.

[wing]

Seems obvious.  The engineering minds want to see those numbers and care nothing about text.  While the rest of the world reads the text

[blotter]

Seems obvious.  The engineering minds want to see those numbers and care nothing about text.  While the rest of the world reads the text

there's a lot of true to this.   And some may simply want a tidy chart to reference

single scoring system and scorecard on its own is virtually meaningless.

 

Jacob, in a sense you're right.  However some people here are simply saying this has value to them.
let's be honest, we all know there isn't a single system that you won't get someone questioning, because the chart itself doesn't answer all the questions. 

I think the chart and more important the articles as Northernridge says play a roll together.
Nothing will replace a really good write up.  And you guys have been doing a great job. Why do you think so many of us are here.   

[tooscoops]

in sales, we looks at SUVPS... style, utility, value, performance, safety/security.... every buyer has a "hot button(s) that fits under one or two of those categories.

what if each car was given scores of 1-5 in each of those or something? doesn't even have to be directly related to the journo's rankings...  so a car that gets all 4's and 5's except in style will still be considered by many people who just don't car how the thing looks... even if it ranked 6th out of 11 or something.

[Noto]

A single scoring system and scorecard on its own is virtually meaningless.

I don't care for score cards, and I think you guys do an amazing job.  Video would be nice, but I'm on your side with the difficulty in so doing, so I wouldn't expect it (unless we bring Mike back?)

The constructive criticism I can request here is as follows:
Since you get press cars, and all are different trims, etc., I'd like to see you comparing the car assuming a level playing field.  The big price/value weighting does tend to hurt cars that come in as more expensive.  You did do this to a degree, I just would like to see a bit more and across all vehicles.  Is the Dart expensive at all trims, or is it just expensive as is?  Would it be so bad in Ralliart trim instead of Limited?

Maybe specify what your chosen trim would be and at what price.  If the Forte's only real benefit is value, and the SX is the only trim with great value (because lower trims don't include ventilated seats, etc), then would you recommend the SX but not the LX?  If the Navi screen on the Mazda 3 is so good, would you highly recommend the GT as worth the extra $?  You did this for the Note - specifying that the Tech package was only $800 or so, which is a 'steal' in the automotive world. 

It just seems like a shame that the cars' fates are dependent on the trim given, but not based on available trim.  I wonder if you had a Forte LX whether you'd like it at all, or if you had a Civic EX whether it'd be relevant at all?  What about the base Corolla with the 4AT?

That's all to say that if you could recommend the trim you think is the best value, scoring notwithstanding, it would be a good piece of info for would-be consumers.

Also see my comment here:
http://www.autos.ca/forum/index.php/topic,85989.msg942453.html#msg942453

[jyarkony]

No video. Would rather read and look at great pictures. More, better pictures that compliment the text.

how would you rate the photos in the Compact Car Comparison?

[Northernridge]

No video. Would rather read and look at great pictures. More, better pictures that compliment the text.

how would you rate the photos in the Compact Car Comparison?

I think the group shots are interesting (just looked at them again). Make a nice body of 'cover' or summary shots, was obviously a bunch of work driving them around into position.

It would be nice to see additional comparison photos…interior, trunk, engine, wheels, etc. Also a few images of standout features on some of the cars. Maybe a couple special shots of the winner.

Looking through the notes from your testers, maybe there are some clues there for some interesting shots to support the text.

I wonder if you could come up with a standard hit list for comparisons and then add a few special shots based on key observations from the reviewers.

Edit. OK, tracked done all the individual car photos taken during the comparison…pretty effective and thorough actually. But, I had to hunt them down in the photos section of the website, only the group shots were in the article. So, maybe there is something to be done in making access to the images a little easier and obvious.

[wing]

I still think photos are a huge difference at autos.ca   A lot of other sites just provide the press photos or a couple of exterior shots and done.  We offer many more detailed shots of interiors in general which is what most people want to see.

Not that they can't be improved just that we already do this well in terms of content.   Winter certainly makes it interesting.... I "love" taking photos of cars at -40

[jyarkony]

Edit. OK, tracked done all the individual car photos taken during the comparison…pretty effective and thorough actually. But, I had to hunt them down in the photos section of the website, only the group shots were in the article. So, maybe there is something to be done in making access to the images a little easier and obvious.

Huh?

You mean you don't see the exterior walkaround shots and various interior shots laid out in the article,
 which can be clicked on to see the enlarged photo?