Julian Edgar, 50, has been writing about car modification and automotive technology for nearly 25 years. He has owned cars with two, three, four, five, six and eight cylinders; single turbo, twin turbo, supercharged, diesel and hybrid electric drivelines. He lists his transport interests as turbocharging, aerodynamics, suspension design and human-powered vehicles.
New car tests
Let’s talk about new car tests.
When I first started writing them, about ten years ago, I said to myself that I was not – simply not – going to write wishy washy, shades-of-grey tests.
If the car has good points, I would emphasise them. If the car had bad points, I would emphasise them. I would not do as so many others do, and that is to use trendy, chatty writing to disguise that in fact little was actually being said about what made a car good or bad.
The results of the approach that I have taken has been threefold.
One: some manufacturers no longer lend us cars to test, something I have covered at length previously.
Two: in discussion groups and in emails I cop a lot of criticism – I’m a dickhead, on the payroll of a company, don’t know what I am talking about, don’t understand the concept of the car, and so on.
Three: I am happy with the work I have done, knowing that I have never kowtowed to vested interests to write a report that pulls punches or says nothing much.
I think that one very important approach that readers should take when looking at new car tests is to simply count the good and bad points mentioned by the writer. Those are the points that are stated as being unambiguously good, or unambiguously bad – not the throwaway “this might be bad but hey, you need to take this into account…”.
For example, do the ‘points count’ with this one of my tests – I count 37 good and bad points made about the car… 14 good and 23 bad points.
Here’s another. On this one I count 38 points – 19 good and 19 bad.
(At the time of writing these are my two most recent tests – I haven’t specially picked them out!)
Now clearly this sort of quantification does not give the overall view of the car – the 19 ‘good points’ might be a lot stronger than the 19 ‘bad points’, for example.
But what counting points does is show how fair honest and upfront the writer is being.
A particular type of writer can easily type a 1000 word test and say very little about a car; these tests tend to be greeted favourably by advertisers, car companies and the public.
Why? Because no one is offended.
Libel laws prevent me from picking out specific examples, but go to any major newspaper website, print out a test and do what I have just done – count those points that describe good and bad aspects of the car. Remember, if something is said and is then immediately qualified with words like “although” and “but you need to remember”, “but then” it is not unambiguous.
I have just done the ‘points count’ on the first test I found. I could find only 15 points (8 good and 7 bad).
On the next “test” I couldn’t find one point, good or bad!
But how could that be so?
One approach that many journalists take in road tests is to write a description of the features of the car. No value judgements are made, so no good or bad points need be mentioned. This wonderfully avoids any negativity from readers, editors, advertisers and car companies.
And the next test I look at random? A much better test with no less than 54 points, including a stunning 42 bad points. An excellent test in fact.
Clearly I am not saying here whether it’s important that you or I agree with the good and bad points; what I am saying is that a road test that makes only a few unambiguous points (or even none at all!) is just complete garbage.
Many, many road tests say very little. The first step is always to assess how many points are actually made.
The next step, of course, is to assess how justifiable those points are…
on April 3rd, 2008 at 8:07 am
With regards to Manufacturers not
allowing reviewers to review new vehicles; It makes me wonder what they’re trying to hide? It’d be nice to look up a list somewhere, of all the new vehicles that have details of which ones have been refused reviews and why.
My 2c worth
on April 3rd, 2008 at 8:09 am
Good one Julian.
I have used many of your tests as reference when considering potential future cars. When I made my last new car purchase the front-runner was the Volvo V50 wagon. I read your test thoroughly and then tried to gloss over the bad points, which largely related to poor interior space and practicality. No other reviews were so harsh as they largely ignored the car other than from the driver’s seat. In the end after looking at all the potential competition, back to back at the Melb Motorshow, I had to admit that you were spot on. I got a Saab 9-3 Sportcombi instead which was not as cutting edge, design and dynamics wise but has the space, safety, economy and praciticality that a young family of four actually need for day-to-day living.
For my money, your reviews are always well considered, as unpalatable as the truth might be. Eg. your review of the latest Saab 9-5 is gutting for a Saab owner but I can’t really fault it.
on April 3rd, 2008 at 8:39 am
Unfortunately this isn’t simply a failing of car reviews, most reviews suffer from it, and is really just a product of most people being too lazy to exercise any form of critical thinking and resent anyone who challenges their preconceived beliefs.
“In short, what people think they want is news, but what they really crave is olds.” – “The Truth” by Terry Pratchett.
on April 3rd, 2008 at 8:47 am
A thought provoking read and I found myself agreeing strongly with your comments. Though at the end of the article I found myself questioning just how impartial this site is as I came across an advertisment forn a new car, in this instance a Honda Accord. Coincidence?
on April 3rd, 2008 at 8:58 am
Rob, in both AutoSpeed and the blog, I have literally no idea of what ads will appear. It is nothing to do with me at all. Rather ironically, after our last two Honda new car tests, I am actually wondering if Honda will continue to lend us cars – we’ll see.
on April 3rd, 2008 at 2:18 pm
Hello Julian, would you like a soap box to stand on? Geee… I now have no illusion as to how much better than others you think of your journalistic ethos.
on April 3rd, 2008 at 2:29 pm
Bryce:
I gave you an impartial way of assessing new car test content – count the unambiguous points that the writer makes about the car.
You can either argue that such an approach is invalid (ie you’d rather as few points as possible are made) or do as I suggest.
Since I believe that a lot of clear points should be made, self-evidently I try to achieve that in my own tests.
on April 3rd, 2008 at 7:06 pm
Having just started to review products myself (not car related) I now understand just how much pressure there is to write ‘good’ reviews, after all, it is the interest of EVERYONE who benefits from the sale of the product that you do… advertising execs, publishers, the company themselves and every PR person along the way. I consider your reviews go-to material when anyone asks me about particular new car. As far as manufacturers not giving you cars, pretty silly move. You could at least do something worthy of a ban like roll/crash a few 02 sti’s (i remember that review was not popular!)
on April 4th, 2008 at 5:45 am
I think the Autospeed reviews are a breath of fresh air. While I may not agree with some of Mr Edgar’s comments (but we are probably in a different demographic) at least it is clear he is trying to live with the cars, and comment on what they are really like to own. Here in the US at least, most reviews concentrate on how a car looks and what frills it comes with. Hardly any space is used to describe what they are like to drive, take shopping, accommdate passengers, or figure out how to change the damn radio station and climate control!
on April 4th, 2008 at 10:02 am
Working with quite a few journos in my profession I often ask them about the ‘truth’ behind what they write. The ‘honest’ opinions they have of some cars are sometimes quite alarming when compared to the published story.
Whilst I can understand WHY they gloss over the bad points and concentrate on the good points – (it’s not easy to keep your publisher / editor happy and keep a weekly pay packet when you are pissing off manufacturers who threaten their publications with advertising and vehicle supply withdrawal), it does make it hard for the unwary potential buyers to get an honest opinion of a vehicle (or any product for that matter)
I have experienced this first hand – I was contacted by one manufacturer a number of years ago for giving my honest opinion of a new vehicle I was evaluating.
I was also told that they would no longer provide us with vehicles due to my comments (similar along the lines that Autospeed was).
At first I was quite worried that there would be backlash from my seniors, but thankfully in the end, they backed me and my honest opinions.
Good on you Julian, Whilst it may not win you any friends at some manufacturers or their media/public affairs managers – it does give people an open and honest apprasial and earn you the respect from everyone that wants to know the truth.
on April 4th, 2008 at 10:22 am
While the praise of my tests is appreciated, the main point of my blog is that many new car tests contain only few unambiguous judgements.
In other words, many new car tests say nothing – either positive or negative – about the cars that are being tested. Counting the non-qualified points is an easy way of seeing this.
on April 4th, 2008 at 1:53 pm
Or how about this Julian !? Seeing a [xxxxxx*] motor journalist, cover a specific euro car manufacturer on a long continual basis, all good write-ups, then your heart sinks when you see first hand, aforesaid journo driving around town in the respective motor companies vehicle, which btw, on his salary, Rupert doesn’t pay enough for. Then hearing the angst as other Euro dealers notice aswell.
Talk to a newsagent, they’ll tell you car mag’s are dying, just like newspapers.
It’s a turf war of open/hidden allegiances versus credibility, which only websites/bloggers can show any independence, without content being controlled, overt or via any other influence.
It was the ‘trendy/chatty writing that put me off car mags. You can’t do a technical based review intertwined with blokey descriptions that entertains only 15 year old’s.
And you comment about newspaper’s padding articles, sheesh, how many time’s my eye’s have glazed over. Journo’s covering the options available, without using a ‘break out box’ instead. It’s been getting worse & it’s deliberate.
Never understood why Aussie newspaper’s have motor journo’s, they never get enough space to write a decent review, all seems like an anachronism to me.
[*edited for legal reasons – JE]
on April 5th, 2008 at 10:54 am
It helps to know the writers quals and experience so one can evaluate the worth of their comments -eg your mechanical engineering studies back up your comments on design fundamentals but demographically we differ. But hey, for we car enthusiasts its not all quantitative stuff like point scores because the qualitative stuff really matters. Quantitatively the woman next door has 2 legs, two arms, two… just like Elle McPherson but qualitatively there is just no comparison! We also have our own biases (or carefully honed opinions based on years of experience) so if you value small, light weight, economy focused type vehicles then you will have trouble looking positively at my prefered larger, heavier built performance biased rwd sedans. Emotionally I love the sound of a nice V8 but of course logically few need them – I’ve just bought an XR6 Falcon instead of an SV8 Commodore because the head (just) won over the heart. Of course I really want a benchmark car like a BMW 5 series/E class Merc for the superior engineering but $$$…Additionally I’ve never wanted a station waggon because I want the luggage separate from the people and I value safety and solidity more despite the inherent weight penalty (I rolled my old Volvo in my youth and escaped unhurt thanks to good design and the heavy stuff secure in the boot). I prefer rwd dynamics so no fwd is on my shopping list (but I enjoyed other aspects of my Magna Sports). So you and I would attach different values to different qualities of a vehicle – I like the mechanical simplicity of rwd but was tempted with AWD Subaru or Magna until ownership costs ruled them out; points for design and roadholding but points against for turning circle and fuel use and long term repairs. In short, road testers must state their own biases as well as stating the facts because we all have our own weightings; a positive for you may be a negative for me on the very same point like compactness verses underbonnet service accessability…and so on. Bob
on April 5th, 2008 at 11:05 am
I’d like to say that I always look forward to reading the new car reviews on Autospeed. The most refreshing aspect is that the car is actually driven and used from the position of someone in the real world. A family is usually crammed into the car, including child booster seats, and usually a long drive is taken. I appreciate this, as this tells me what the car will be like if I took it home and had to live with it.
I find that the comments made are usually accurate, and Julian displays knowledge and most importantly, an interest in cars. Here is someone who genuinely makes an effort to get to know the car and gives his opinion and justifications for that.
In comparison, my local paper quite often publishes a collection of shallow and/or inaccurate comments that have been mixed with the manufacturers marketing material. I find that type of dribble offensive to the reader as it is clear they have no clear interest in cars and often mix up the manufacturer and model names.
Keep up the good work!
on April 6th, 2008 at 12:52 am
“If the car has good points, I would emphasise them. If the car had bad points, I would emphasise them.”
This isn’t what journalists do: a report. It is what cartoonists do: a caricature.
on April 6th, 2008 at 4:13 pm
I have owned 4 cars that Autospeed has done review about: Toyota Soarer V8, Toyota Avalon, Subaru Legacy GTB, Subaru Impreza V6 STi. Julian Edgar’s review seemed 100% correct so far. Therefore I have no doubt at his other reviews and will continue to purchase cars in the future base on his reviews.
on April 6th, 2008 at 7:42 pm
It’s refreshing to read Mr Edgar’s unbiased reviews. The number of reviewers from other car sites that pander to manufacturers is sickening to say the least.
The newspaper journos don’t do much better, and in many cases you can see that they don’t want to offend the manufacturer. They often disguise obvious poor qualities and sub-standard design by using words like “value for money” and “a likeable quirk”. What rubbish.
One look at another online blog, it’s clear to see even in the blog comments the blatant “advertising” by either delusional vocal fans of said marque, or by disguised robot minions from the marques themselves.
Once again, Julian, kudos on standing your ground.
on April 8th, 2008 at 7:26 pm
I don’t even count the good points, I just count the bad points. If a given reviewer reviews 10 different cars (or movies, or restaurants), and most of them have serious criticisms and negative points, then I’ll take what is said seriously.
If on the other hand, ever single review could have been written by the respective companies marketing department, then I’ll dismiss it all as rubbish.
Some reviewers will throw an occaisional bone by giving a harsh review to a Trabant or something, and then return to rewritten press releases, but anyone with a history of saying bad things about different companies products I take seriously. ESPECIALLY if they say that “XXXXX is really good.”
on April 9th, 2008 at 4:52 am
Even from the remote perspective of the US, I have found Autospeed reviews to be useful. Evaluating “real world” driving dynamics actually means more than hot laps around a track with a trained driver. The lap times are intriguing, of course, but knowing how the car drives, feels, responds to normal cornering, passing, cruising is much more useful. I am also delighted to see attention paid to outward visibility – I have test driven a few cars that were visually interesting, with good performance, but A and C pillar blindspots that made it feel like navigating out of a cave. Technical reporting is valuable for some comparisons, but at its very base, a “review” is an expression of opinion – “I liked xxx, and I did not like xxx”. Much the same with movies, theater, music, food, etc – just that the $$ outlay and time involved is quite substantially less than what most of us invest in our vehicles. Even in the few cases where I have not agreed with Autospeed, the perspective offered has been valuable – another factor for consideration. Thank you!
on April 9th, 2008 at 4:44 pm
Currently, I’m looking at purchasing a 4L BA or BA MKII Fairmont Ghia (dependant on how far I can bid the dealer down)… and I can totally see where Julian is coming from.
I’ve searched the net high and low, and apart from the autospeed review on the BA Fairmont (Not Ghia, and its a 5.4), there are no ‘reviews’. Sure, i found a handful of carpoint-esque ‘reviews’… well calling them reviews would be a very loose use of the term review… but anyway, I found a handful, and they simply outline the general points of the car with no critique or judgement given.
I applaud Julian for his honest and often critical view of cars when testing – he seems to actually treat the car as if it was going to be the new addition to his family.
on April 9th, 2008 at 4:59 pm
I should point out that some of the tests people are citing were not written by me, but in fact were done by Michael Knowling when he was a staff member.
In fact, Ford don’t lend me press cars, a situation that started after I wrote this column: http://www.autospeed.com/cms/A_1191/article.html
(Perhaps things have changed; I haven’t bothered asking in recent times.)
on August 11th, 2009 at 8:41 am
If on the other hand, ever single review could have been written by the respective companies marketing department, then I’ll dismiss it all as rubbish.