The University of Chicago Law School Faculty Blog

Two Answers for Judge Posner

I said that I thought the Justices in the gun use case could have passed polygraph tests in which they said their semantic arguments were a substantial basis for their decisions.  In reply, Judge Posner has asked me two questions.  Here are my answers.

1.  What is my basis for thinking this?  Conversations with judges.  For anyone who does empirical studies of judicial decisions, and finds evidence of what seems like ideology at work in them, it is common enough to be told by a judge something like this: "that's interesting, but it doesn't square with my experience -- when I decide cases, I really feel that I'm making decisions based on legal materials, not policy preferences."  I have been told that sort of thing a number of times in conversations about these same issues, and I don't think the judges who have said it are being disingenuous.  (Sometimes objections like these are made public, as by Harry Edwards of the D.C. Circuit.  I don't agree with him, but I don't think he is being disingenuous, either.) 

The implication of the question, and of Judge Posner's earlier remarks, is that he doubts the polygraph test would be passed.  I would be interested in hearing speculation about this from the other judicial participants in this panel (but I sense a danger of circularity here!). I think polygraph tests of this kind are an exciting and amusing frontier in empirical legal scholarship.  If Judge Posner can persuade his colleagues to take them, I will start hunting up grant money. 

2.  Suppose they did pass the test; what would that show?  It would show that (a) they thought those semantic arguments were important, and didn't intend them as a smoke screen; and that (b) they thought that by relying on semantic arguments, they were deciding the case on grounds independent of their policy preferences.  Point (a) would fortify our claim that it's worthwhile to study how semantic arguments work, as we have been doing; for they are serving some function (but what?) in judicial decisionmaking.  As for point (b), despite the polygraph results I would not conclude that the semantic grounds really were the basis for their decisions in any deep sense.  On the contrary, I would point to our studies and suggest that the semantic arguments probably felt attractive because they squared with the judges' policy preferences.  So perhaps the semantic arguments are best viewed an unintentional or unconscious smoke screen (or maybe as a way to reduce cognitive dissonance, to use Judge Posner's phrase).

And so I would say:  beware semantic arguments; they sound unconnected from policy preferences, but easily get entwined with them and express them even if you don't realize it (arguments about language can feel very convincing, like they have nothing to do with preferences, but that feeling is unreliable).  If you really do want to argue about semantics and not policy, we have found a method that might make this somewhat more likely to work, though the method may or may not be attractive in any given case for reasons we talk about in our papers (Judge Posner would probably never like this approach).  If it turns out that you actually want to argue about policy and not semantics, then it would be better to stop pretending that semantics are the issue.  (Here I think Judge Posner would agree.)

Two Questions for Professor Farnsworth--Posner

In his latest post, Professor Farnsworth says: "I expect that the Justices who decided the gun use case could have passed a polygraph test in which they said the semantic arguments were a substantial basis for their decisions." Two questions: 1. What does he base this expectation on? 2. What would their passing a polygraph test show? That semantic arguments were a substantial basis for their decisions, or that (to avoid cognitive dissonance) they merely thought they were?

Ambiguity: Further reply to Posner

Like Judge Posner, I doubt that semantic arguments do the work that they appear to do in judicial opinions.  But unlike him, I don't quite think the issue is a lack of candor.  I expect that the Justices who decided the gun use case could have passed a polygraph test in which they said the semantic arguments were a substantial basis for their decisions.  But sometimes, and probably here, the legal arguments in a case are pliable enough to inadvertently express policy preferences rather than constrain them, even if the holders of the preferences do not understand this to be happening.  Our studies show this effect in a more controlled and stylized environment.  People told to put their preferences aside and just worry about semantics end up taking semantic positions that advance their preferences.

I suspect that Judge Posner makes it a question of candor because he himself has so little use for arguments about semantics (so it would show a lack of candor for him to make them).  But I think many other judges have not yet arrived at his conclusions, and that they take conventional legal arguments, including arguments about semantics, more seriously -- even if, as we mean to show, those arguments often end up carrying the water for policy preferences after all. 

We do suggest a way to make judgments about language that are less likely to be infected by policy preferences.  If nobody cares about doing that, and all the talk to the contrary in judicial opinions is a smoke screen, then this part of our study, like much other legal scholarship, would indeed be a waste of time.  But we think some judges still like the idea of doing it.  Judge Easterbrook says he is among them; perhaps Judge Posner isn't.  That's an interesting dispute (if it really is a dispute), but our project is not meant to try to settle it.

Further Reply to Farnsworth

I take the key passage in Professor Farnsworth's reply to be the following: "Maybe he thinks these passages are insignificant; it's just stuff that judges are required to say by custom, but that has nothing to do with their actual reasons for decision.  There may be something to that, but we think the materials that judges use (and feel obliged to use) to build their arguments can have some effect on what they actually do decide, so we aren't ready to treat these sorts of debates as though they don't exist" (emphasis added). I take it that by "some effect" he means a substantial effect, as otherwise the semantic arguments would not be interesting. My question is: what is the evidence that arguments about dictionary meanings and other semantic issues have a substantial effect on case outcomes?

The Supreme Court is a political court operating in a goldfish bowl; why should we expect candor in its opinions, any more than we expect candor in a Presidential address? There used to be candid Justices--Holmes and Jackson spring to mind--but who are the candid Justices today? And how often does one find semantic nitpicking in outstanding lower-court judges like Hand and Friendly?

Ambiguity: Response to Professor Eskridge

Other studies find that judges' votes are correlated with political views.  Indeed, even in the same case judges from across the aisle vote in different ways.  Those are observational studies, so suffer from issues with internal validity.  Our study has greater internal validity because it is randomized, but suffers from external validity.  We study students not judges.  We don't provide as much context or room for reasoning.  Yet our study finds similar results.  When there is evidence of political influence on one side and the other, it takes a great deal of faith to suppose that judges are fully insulated from their own political preferences.  The commentators provide no evidence on which to base that faith.

Our remaining claim is that the ordinary readers framing helps mitigate the effect of preferences.  Unfortunately, there is no observation data on judges corroborating our findings here.  Perhaps the data from Brudney and Ditslear would help.  In any case, our data suggest that the ordinary readers framing could have a potential debiasing effect.  This should be explored in future work.  Ideal would be a randomized study of judges.  

And again, judges are not the only individuals who bear responsibility for interpreting statutes.  Indeed, I would conjecture that only a small fraction of behavior is influenced by how judges read a statute.  Lawyers working for clients and the government must read statutes and tell clients or government employees how they must behave.  Prosecutors must read statutes and decide which individuals to prosecute.  All without judicial guidance.   In some cases these lawyers will use outside information, such as legislative history.  But in many cases they will base their decisions on the text of a statute and a quick judgment.  Our study may be more predictive of what influences lawyers in this context. 

One final point.  Perhaps it would be easier to absorb our findings if the experiment were viewed not as an attempt to see how judges vote but rather as a cognitive psychology experiment.  Social scientists have run numerous experiments on students revealing that they have trouble dealing with low probability events, or cannot understand fractions as well as odds, or are averse to ambiguity, or suffer endowment effects. Numerous legal scholars have cited these studies as suggesting that judges may suffer similar foibles. (And judges have surely wondered whether these behavioral findings apply to litigants before them.)   It can always be objected that the researcher did not study the target population to which scholars want to extrapolate their findings.  But the response is that the studies shed light on psychology and this psychology is not specific to one profession or another.  

From this perspective, perhaps it would have been better if, instead of showing students statutory text, we had simply showed them some other writing, such as a newspaper story, and asked them how they interpret that story.  I predict we would have found that preferences affect how people read the story.  But surely the fact that we actually used statutory text should not make such a cognitive psychology experiment less persuasive.  On the contrary, we show how the problem of separating preferences from judgments can affect a person's reading of the same sorts of texts that judges read.  We  don't claim that judges make their ultimate decisions about cases in the same way our respondents answered their survey questions; but we  think it's plausible that judges struggle with the same basic problem  our respondents did, and with the same lack of success, at the stage  of a case when they are trying to make the same kinds of judgments that our respondents were.

Ambiguity: Comment from Judge Williams

I understand that the judge confronted with decent briefs in a moderately close case doesn't ask the question of how "ordinary readers" would interpret the statute.  My point is simply that I don't see the study as offering evidence on the marginal gain (in reducing judicial non-neutrality) to be had from judges' asking themselves that question."

--Judge Williams

[Anup Malani is posting this comment on behalf of Judge Williams.  Judge William's is in Moscow and does not have reliable access to Typepad.]

Ambuguity: Eskridge Comment on Posner and Response to Malani's Reply to Eskridge

I have not been following the exchange very carefully but have read the authors' replies to Dick Posner and to me. I want to amplify my earlier point, that studies of law student responses to abstract problems about ambiguity have limited (perhaps no) value in understanding how judges, administrators, or even practicing lawyers process issues of plain meaning, ambiguity, and statutory interpretation.

Limitation #1. As Aristotle said (paraphrased for the current debate), statutory meaning is driven by context--including the facts of the case, the policy and legislative background, the surrounding text and the whole act, precedent and other prior interpretations, the constitutional terrain, etc. Simple hypotheticals have limited utility in telling us much about how judges et al. evaluate statutory texts--but perhaps the more telling point is that judges et al. have a lot more experience with legal materials than first-year law students do.

Limitation #2. Judges, administrators, and (usually) lawyers are publicly accountable for their interpretations of statutory text. That the law students answered the questions anonymously (I assume that was the case) or surely with no public accountability makes their task much different from that of the judge who realizes that her interpretation can be criticized by the losing party, lampooned by commentators, or reversed on appeal. Do these features of professional practice suggest that judges will be more disciplined than law students? I'd assume so. (This calls forth my biggest surprise about the study we are discussing: without accountability, I am surprised, and impressed, that the law students were not more influenced by their views of the merits. Good for them.)

Limitation #3. Judges, administrators, and attorneys also usually provide reasons for their decisions. I wonder whether the requirement of reasons might exercise some discipline on respondents, independent of the accountability point.

Ambiguity: Farnsworth reply to Posner reply to Farnsworth reply

 

Judge Posner disputes that judges argue much about the "normal" usage of words; when they argue about words, there is always a "complex background understanding" at work of the motives of the drafter and the function of the interpreter.  And he doesn't see any place for cognitive hazards to enter into the process of interpretation, either. 

 

So let's take as an example a case that I've mentioned in a previous reply:  Smith v. United States.  That's the one where the defendant tried to trade his gun for a bag of cocaine, and the question was whether he had "used" the gun in relation to a drug trafficking offense.  The case was decided 5-4 in favor of the government.  The two sets of Justices seemed much concerned with the exact point that my co-authors and I have been studying:  how to read the language of the statute.  Not how to read it in view of complex background understandings, but how to read it as a piece of English.  The majority opinion, like so many opinions, says that "When a word is not defined by statute, we normally construe it in accord with its ordinary or natural meaning."  They went through dictionary definitions and examples of how the word "use" is used.  The analysis then sounded like this:

 

"It is one thing to say that the ordinary meaning of 'uses a firearm' includes using a firearm as a weapon, since that is the intended purpose of a firearm and the example of 'use' that most immediately comes to mind. But it is quite another to conclude that, as a result, the phrase also excludes any other use. Certainly that conclusion does not follow from the phrase 'uses . . . a firearm' itself.  As the dictionary definitions and experience make clear, one can use a firearm in a number of ways. That one example of 'use' is the first to come to mind when the phrase 'uses . . . a firearm' is uttered does not preclude us from recognizing that there are other "uses" that qualify as well. In this case, it is both reasonable and normal to say that petitioner 'used' his MAC-10 in his drug trafficking offense by trading it for cocaine; the dissent does not contend otherwise."

 

And then the dissent argued back about what an ordinary person would think it means to "use a firearm."  These discussions didn't owe much that I can see to the complex background understandings Judge Posner talks about.  (Other parts of the opinions might.)  Our claim is that in arguments like this, there is a risk -- the "cognitive hazard" mentioned before -- that the person makings claims of the kind just shown will be giving effect to policy preferences without realizing it.  The interpreter claims not to be making a policy judgment, and thinks no such judgment is being made, but it is.

 

I'm not sure which part of this seems objectionable to Judge Posner. Maybe he thinks Smith is atypical.  I don't; lots of Supreme Court and appellate opinions—though not, as a rule, opinions from Judge Posner—contain passages like these.  Maybe he thinks these passages are insignificant; it's just stuff that judges are required to say by custom, but that has nothing to do with their actual reasons for decision.  There may be something to that, but we think the materials that judges use (and feel obliged to use) to build their arguments can have some effect on what they actually do decide, so we aren't ready to treat these sorts of debates as though they don't exist.  Maybe he thinks—I'm reasonably sure he must think—that the passage quoted a moment ago, like many passages from Supreme Court opinions, is vacuous nonsense that shouldn't be trusted.  Again, that may be.  Indeed, I'd say our claims and evidence can provide some additional support for that view, so I'm surprised he doesn't welcome them.

Ambiguity: Posner Reply to Farnsworth Reply

It may seem hard to quarrel with Professor Farnsworth's anodyne statement: "Obviously judges know a lot more than our respondents did, as we acknowledged early.  The fact remains that judges often are confronted with a basic cognitive task similar to the one that faced the takers of our surveys:  they have to figure out what a piece of English text is best understood to mean.  Sometimes their impressions from the wording and punctuation are heavily influenced by things they know that our respondents didn't know.  Sometimes they aren't.  In the gun use case, a lot of the arguing was all about what a normal usage of the statute's words would mean; there wasn't very much reliance on knowledge that our respondents lacked.  In other cases the reliance on additional knowledge is greater, but the judges still argue about the simple English meaning of the sentence as one component of their analysis.  That is the component we are studying.  We suspect that the cognitive hazards we describe are common to judges just like anyone else."

But I will quarrel with it, particularly the notion of "cognitive hazards," and that judges do a lot of arguing over the "normal" usage of a statute's words. There is no interpretation of a document without a complex background understanding of the motives, intentions, and so forth of the drafter and also the function of the interpreter, which might be docile, obedient, but might instead be creative, and/or forgiving (I know what he's driving at, but he's expressed it badly--I will interpret according to what I think he intended). I don't know where "cognitive hazards" enter. If judges are literalists (a kind of "screw you" view of legislators: we know what you're driving at, but you said it wrong and we'll rub your noses in it by interpreting the statute to mean what it says rather than what it means), they will produce different interpretations from the loose constructionists, but the difference between the two groups has nothing to do with cognitive hazards. Judges make mistakes when they misapply their own interpretive approach, whatever it is.

I would be interested in Farnsworth's approach to the Raffles case. The contract was clear "on its face." Once the latent ambiguity is identified, interpretation becomes policy-saturated (who should bear the risk of a mistake in a contract). Since law is part of government, judges

Ambiguity—Reply to Judge Posner

We find Judge Posner's comment, like Judge Easterbrook's, agreeable enough.  We think the study of interpretation by non-judges still has value, and don't understand him to disagree.  Obviously judges know a lot more than our respondents did, as we acknowledged early.  The fact remains that judges often are confronted with a basic cognitive task similar to the one that faced the takers of our surveys:  they have to figure out what a piece of English text is best understood to mean.  Sometimes their impressions from the wording and punctuation are heavily influenced by things they know that our respondents didn't know.  Sometimes they aren't.  In the gun use case, a lot of the arguing was all about what a normal usage of the statute's words would mean; there wasn't very much reliance on knowledge that our respondents lacked.  In other cases the reliance on additional knowledge is greater, but the judges still argue about the simple English meaning of the sentence as one component of their analysis.  That is the component we are studying.  We suspect that the cognitive hazards we describe are common to judges just like anyone else. 

 

As for the general differences between the positions of our respondents and the positions of judges, in some respects doing experiments is like creating economic models.  Either way you have to simplify; either way you abstract certain features of the situation for rigorous study, and end up with a stylized result.  The experiment (or model) isn't the same as reality, but it may capture something important about the reality that is harder to see when all the complexity is left in the picture.  

 

In this case, the experiments suggest a cognitive issue in interpretation and a way to help overcome it.  In real judging, the issue and the corrective may both be less important than they were in our experiments, because (as we have said, and as others have said) judges operate in different and richer circumstances than our respondents.  But we think it is a mistake to too quickly assume judges are able to avoid the hazards that our experiments illustrate, whether because of their training or because they have contextual details to consider that our respondent lacked.  After all, those other materials that judges consider contain ambiguities of their own.  As I mentioned in reply to Professor Elhauge, perhaps—we might even say probably—those other things (ambiguous legislative history, ambiguous statutory context, etc.) may themselves be subject to interpretation, and perhaps those interpretations are subject to the same tendencies we have identified here.  We hypothesize that policy preferences color the handling of ambiguous legal materials of all kinds, and that our studies just show a simple example of it.  The argument to the contrary would be a lot stronger if judges, as a group, were famous for deciding hard statutory cases in ways that are hard to predict on the basis of their policy preferences.  They aren't.