Professor Anup Malani on how we know when we don't know
What are judges supposed to do when statutes are ambiguous? This question is the source of endless quantities of legal scholarship and is at the core of public debates over the proper role of judges. While the is obviously important, it is hard to see how anything really new could be said. Asking this question - how to deal with ambiguity - presupposes another, however: that ambiguity exists in the first place. How do we - or any given decisionmaker - know when a text is ambiguous? Relatively little academic work has been done on this question, which would seem to be at least as important.
Chicago's Anup Malani, along with co-authors Ward Farnsworth and Dustin Guzior (both at Boston University) attempts to fill this gap with his paper Ambiguity about Ambiguity: An Empirical Inquiry into Legal Interpretation, which was presented at this week's Works in Progress (WiP) talk at the law school. The paper is based on an empirical study of nearly 1,000 law students.
Prof. Malani starts with the observation that ambiguity has (at least) two possible meanings: a text could be ambiguous because the person reading it is uncertain about its meaning, or it could be called ambiguous because the person reading it thinks others would disagree about its meaning (even if the reader is not uncertain). Different people, therefore, might consider different texts to be ambiguous depending on which definition they are using.
In the study, students were given a fact pattern and a potentially ambiguous text (and, necessarily, pizza). For example:
A federal statute, 21 U.S.C. § 841(b), provides for a mandatory minimum sentence of five years for anyone who distributes more than one gram of a "mixture or substance containing a detectable amount of lysergic acid diethylamide (LSD)." The defendant was caught distributing LSD that had been dissolved and sprayed onto blotter paper. The weight of the LSD alone was 50 milligrams, well below the statutory threshold. But if the weight of the blotter paper was included, the total weight was five grams, well above the statutory threshold.
The question is whether, under § 841(b), the blotter paper should be included in deciding, for purposes of sentencing, the weight of the LSD the defendant distributed. Under the defendant's reading of the statute, the blotter paper should not be included in deciding the weight. Under the government's reading, it should be included in deciding the weight.
The students were then asked a few questions about their reaction to the passage. Not all students were asked all questions, but generally they were asked whether they found the statute to be ambiguous as applied to the facts, whether they found each side's reading to be "plausible", and whether "ordinary readers of English would disagree about which side's reading of the statute is better". By asking questions in this way, Prof. Malani sought to separate the "internal" view of ambiguity ("I think this text is unclear") from the "external" view ("I think other people will disagree on the meaning of this text"). The survey also asked which outcome each student would prefer as a matter of policy.
The survey results showed that the survey participants do consider the different types of ambiguity differently - students were more likely to believe that others would disagree about the meaning texts than to be unsure about their meaning themselves - that is, the students were more likely to find external than internal ambiguity. Even more interestingly, ideas about internal ambiguity but not external ambiguity depended on policy preferences. Survey participants who strongly preferred that one side or the other prevail were less likely to be uncertain about the text, but these preferences had no effect on what the participants thought others would believe. For the survey scenario above, results were as shown in this graph:
As the graph shows, those with strong policy preferences are more sure of their interpretation of the statute but no more sure that others will find it ambiguous. For this scenario, "respondents with strong policy preferences are 11-21% more likely to say the statute is clear than respondents with weak policy preferences".
This result may have significant implications. Lots of people are worried about how bias may affect decisionmakers' handling of cases with ambiguous texts, but bias seems to have an even deeper role - it influences which texts are treated as being ambiguous in the first place. The paper also shows that discussions about ambiguity in law should probably be more precise. Since people seem to come to different conclusions based on which meaning of ambiguity they are using, there may be a lot of miscommunication in these debates.
More broadly, it may be useful to focus decisionmakers on the external rather than internal meaning of ambiguity in order to excise their personal preferences from the decisionmaking process. Since estimations of external ambiguity are consistent across policy preferences, it would seem that forcing judges to "change their frame of reference" in this way would yield more consistent results. Prof. Malani and his co-authors do not suggest that judges should (or could) be issued such a command, and are content to note that "the external question . . . is a useful, though informal, 'debiasing' heuristic."
Commenters at the WiP noted possible problems or bias effects from some of the questions and fact patterns in the survey. For example, all of the fact patterns dealt with criminal cases - this might or might not be generalizable to other types of cases. Another commenter pointed out that supplying each sides' different readings of the statute might bias participants toward finding ambiguity. Commenters also noted that law students might or might not be a representative sample. Surveying judges and legislators would be ideal, though obviously more difficult.
Another commmenter noted that, like "ambiguity", "policy preferences" might have more than one meaning. Such preferences could be based on principles or on the participants. For example, some people might have a preference that defendants generally prevail, while others might have a preference that statutes be interpreted in a certain way (which might favor the government or defendants in different cases). These different ideas of preference will cloud the survey results unless the questions are carefully asked to separate the two.
Prof. Malani responded to these criticisms essentially by agreeing that they raise potential concerns - he intends to conduct a new, expanded round of surveys that should address these issues.
Other commenters asked deeper questions. For example, one noted that the critical issue is how policy preferences get "turned off" by asking the external ambiguity question. The paper does not answer this (or attempt to). Another commenter noted that a contrary reading of the implications of the results is possible - instead of believing that the judgment of those with strong policy preferences is clouded by bias when we see that they make different conclusions about ambiguity, isn't is equally plausible to conclude that they are right and the moderates are wrong? Maybe moderates just aren't paying attention?
In any case, Prof. Malani and his co-authors' work is a rare example of a new look at an important, basic question. I hope it gets the wide reading it deserves once the final version of the results is published. If it does, it should breathe some new life into stale debates over judicial behavior and open a whole new set of questions – policy, philosophical, even neurological – based on its conclusions.
As vagueness an ambiguity are being conflated in this paper, I would change the subject to "uncertainty about uncertainty in the meaning of texts".
Posted by: Dennis Tuchler | June 05, 2009 at 12:17 PM
The paper finds a correlation between strong views on whether a statute is unambiguous and strong views on policy. The paper suggests that participants' views on policy "appear to bias their judgments" regarding whether the statute is ambiguous. However, the paper contains no evidence of this. (The survey failed to ask participants what the better interpretation of the statute is.) It is possible that individuals with a strong pro-defendant view as a matter of policy may also hold a strong pro-government view as a matter of interpretation (or vice versa). The fact that strong views are correlated may simply indicate that some people are more confident than others.
Posted by: mm | June 16, 2009 at 04:00 PM