The Experts on Expert Advice

One thing that gets a little wearying in political discussions online is argument from authority. If you’re unfamiliar with the term, that’s when someone avoids an argument by pointing to an expert who supports their opinion. So, for example, when you say something disparaging about Obamacare’s finances, people will frequently point you to some CBO estimates that say those finances are sound. They themselves do not know anything about how the numbers add up, nor do they frankly probably really care – they just know that someone who knows more about the subject than you do disagrees with you and are happy to tell you so to avoid having to respond to your criticisms.

Arguments from authority are a tricky thing, because if the authorty in question really is an expert on something – like the CBO in the aforementioned example – AND that authority knows more about it than you do, AND there is no significant disagreement among relevant authorities about the point under dispute, then it’s reasonable to take that authority’s word on the subject. It’s that last point that most people on the internet seem to miss. If authorities on the subject themselves have not settled on a consensus as a group, then of course people are free to shop for authorities that say what they want to hear, and the argument becomes suspect.

That is transparently the case with the CBO argument. It’s true enough that the CBO has signed off on certain estimations of cost for Obamacare that its supporters find comforting, and it’s true enough that they are professionally engaged to estimate goverment spending costs where I am not. All else equal, that means I should defer to their judgement. But all else is not equal in this case: there are plenty of think tanks that are also professionally engaged to estimate government spending costs that have good reason to doubt the CBO’s estimation.

And of course there’s the additional factor that everything that I’ve just outlined assumes that all parties are (a) acting in good faith and (b) not simply mistaken. As we all have had reason to experience, experts frequently either do not act in good faith (experts have biases of their own) or else are simply mistaken (experts are human too, you know).

All of this is a roundabout way of saying that just being recognized as an expert on something does not mean you are absolved from having to prove your case. There is good reason to privilege expert opinions about matters, but that is not the same thing as saying that we should follow them blindly. Experts have a responsibility to be able to explain their opinions on matters involving the public in terms the public can understand so that the public can, with a reasonable amount of effort, check on their reasoning.

I mention this because I read a study recently which suggests that people are actually pretty reasonable in their deference to experts. The academic title is How Expert Advice Influences Decision Making – and while it’s primarily an fMRI survey (that is, it’s trying to figure out which parts of the brain are involved in deciding whether to adjust an opinion based on expert advice), it has some interesting things to say about the process of thought that goes into deciding whether to accept an expert’s advice as well.

The authors assume, and find some evidence to support the assumption, that there are two layers to this process: (1) whether the advice comes from an expert and (2) how much the expert’s advice differs from the recipient of the advice’s starting position. As for (1), people are, perhaps unsurprisingly, significantly more willing to accept advice from experts – but, it seems, not that much more willing in raw terms. The researchers use a term called Weight of Advice, which they calculate as the value of the expert’s opinion (their experiment is done with prices, so this is actually quantifiable). Weight of Advice is the percentage of the difference between the advice they were given and their initial opinion that they were willing to adjust their opinion by. So, since the study is done about rent prices, if the subject initially estimates the monthly price of the apartment at $1000 and an expert tells them it’s really $2000, and they adjust their estimate to $1500, the Weight of Advice score is 0.5, since they adjusted by $500 for a bit of advice that told them to adjust $1000 – i.e. they adjusted 50% as much as they were told to.

(Now – as an aside – I should say here that there’s another measure in the study called "opinion difference" that seems confounded with the WOA measure. In fact, I’m not really clear on what the purpose of this measure was. It’s the difference between the initial guess and the advisor’s opinion, and the researchers group their results according to whether there was a "high" or "low" opinion difference. Well, since Weight of Advice subsumes this information, I think the study would have been clearer without these groupings. This is especially true given that I can’t see an interaction of any kind with the "opinion difference" category. All the action seems to be between whether the advisor is "expert" or "novice." So, the next paragraph glosses over this aspect. I come back to it later.)

Well, Weight of Average is uniformly higher for expert advice over novice advice, indicating that people are willing to adjust their opinions further if told to do so by an expert. In terms of numbers, it looks like they’re willing to give about 65-70% on the advice of an expert, but only around 40-50% on the advice of a novice. So people listen to experts more often and more closely than they listen to novices.

But the takeaway point for me is that expert advice is not sufficient to induce agreement. People are more deferential to experts, but they are nothing like completely deferential to experts. That is, they seem to know that experts are not always right.

Another point the paper attempts to make, but makes somewhat less successfully (at least, in my opinion, at my current level of understanding of the paper – which I freely admit I haven’t read too closely), is that people are more likely to adjust to opinions that are already close to their own. So, there’s some kind of a "breaking point" beyond which if an expert’s advice differs that much from your own, you simply disregard the expert. And again, that seems both right and healthy, since at the end of the day we all ultimately answer only to ourselves, and since at the end of the day our opinions about things are rarely about narrowly-defined things. You might flip your opinion completely on a highly-specific Physics question, say, if told to do so by an expert. But for something like housing prices, where the answer is well known to be a moving target defined by a range of complex factors, you know that an expert can be and frequently is catastrophically wrong about things, and so you only listen within a zone that doesn’t impact too much about what you’ve learned about the world independently.

For me, there are two related takehome points. First, that people have a healthy skepticism of expert advice in general. They are willing to follow expert advice to a point, but not if it requires them to take a leap of faith. The expert advice still has to broadly "make sense." Second, that lack of expertise doesn’t seem to disqualify anyone’s advice, it just blunts the impact. People are people-pleasers, in other words, and they generally try to conform to opinions they are given, they just do it less the less qualified you are to give said opinion.

In terms of political arguments, the consequences seem to be these. First, that there is little point in arguing from authority. People do not generally flip their opinions in the presence of an expert. They may adjust them slightly, but if you’re hoping to say "We need to spend X because Krugman," you’re probably wasting your time. Moreover, you’re almost certainly being hypocritical, since it’s virtually guaranteed that you yourself have, at some point in the recent past, failed to "appropriately" adjust your opinion in response to being presented with an expert’s pronouncement. Second, that if you want to persuade people, you have to nudge them toward your position. Talking someone into slightly lower tax increases is probably doable by presenting some economic surveys that show that the taxes your opponent is proposing are too high. Getting someone who doesn’t believe that the earth is warming to agree to carbon taxes ain’t gonna happen by simply pointing to a climate study or two.

All of which means that being persuasive is hard. People come with a lot of inertia on hand – and that’s a good thing, really, since it cuts down on volatility. So embrace it. Stop waving CBO studies in my face as a defense mechanism when I tell you that I’m skeptical of Obamacare. Chances are, you haven’t read the study, you haven’t made a good-faith effort to seek out opposing studies, and you wouldn’t be completely persuaded were a totally opposing study to come your way anyway. Experts are not get out of jail free cards, and you know this because you yourself have as good as never flipped your opinion in the presence of expert advice you didn’t want to hear.

Leave a Reply

Your email address will not be published. Required fields are marked *

You may use these HTML tags and attributes: <a href="" title=""> <abbr title=""> <acronym title=""> <b> <blockquote cite=""> <cite> <code> <del datetime=""> <em> <i> <q cite=""> <strike> <strong>