Student Blogger - Summer WIP: Posner, Landes, and Epstein on Supreme Court Questioning

The Posner/Landes machine rolled into the summer WIP Thursday, as they presented their latest project (also co-authored by Northwestern University's Lee Epstein, who was not present but whom both effusively praised as having compiled every judicial dataset conceivable to humanity): Inferring the Winning Party in the Supreme Court from the Pattern of Questioning at Oral Argument. Court observers such as Linda Greenhouse had long suspected there was an important relationship between the number of questions asked and the ultimate outcome of the case, but nobody had done the empirical work to figure out exactly what it was. Posner, Landes & Epstein looked at the effect the raw quantity of questions (and total words in questions) had on the probability a given side would win their case before the Court s using data tabulated from all arguments for all cases decided in the 1979 to 2007 period . They found a consistent correlation: Whether measured by total questions asked or total words in questions, more is less. The more queries a given side received by the justices, the less likely they were to emerge victorious. For example, petitioners win about 62 percent of the cases before the court but if the petitioner is asked fewer questions than the respondent, that probability increases to 71 percent. On the other hand, if the petitioner is asked more questions than the respondent, the probability that the respondent will win increases from 38 to 50 percent.

There are two models of judicial behavior, specifically, why judges bother to ask questions at all, that the authors seek to fit this data into. First is the "legalistic" paradigm, which posits that judges ask questions in order to better understand the legal issues at stake and figure out what the true legal right answer is. In this model, we should expect more questions to be directed to the losing side, as judges continue to probe unsatisfactory answers, creating a higher total number of questions. Or alternatively, judges question the side that they believe is weaker to begin with, in order to test their intuition and give their attorney a chance to allay the judges misgivings.

The second model is what the three call "realistic". Here, judges are using questions to strategically influence their colleagues (having already made up their mind themselves). Directing questions to the party they want to see lose gives them the opportunity to refute that side's best arguments and keep them on the defensive, making them appear weaker than they might otherwise seem. Because justices typically do not discuss cases before oral argument, and because even conference discussions are highly constrained and ritualized, questions at oral argument present a key opportunity to sway other justices to one's position. One faculty member suggested a novel way of testing this theory: breaking down the data into the eras of the various Chief Justices, noting that Chief Justice Rehnquist was notoriously more restrictive in what he allowed in conference than the more free-wheeling Chief Justice Burger. Alternatively, Judge Posner noted that there are substantial differences in the discussion norms amongst the various appellate circuits; differences which could manifest themselves in the questioning data if greater opportunity to influence judges after the oral argument reduces the need to engage in strategic questioning behavior.

The authors also looked at how each Justice voted and the questions he asked over the 2004–2007 terms. Here they also found that the relative number of questions a Justice asked the parties significantly predicted his vote. One intriguing exception to the rule was perhaps the most important current Justice on the court, the swing vote Justice Kennedy. He was the only justice for whom the  number of questions he asked the petitioner relative to the respondent did not bear a statistically significant correlation to the outcome of the case. Precisely because Justice Kennedy often represents the crucial vote, the authors hypothesized, he may be more conscientious about equally probing each side -- or he simply might not want to tip his hand too far in advance.

Comments

[Michael F. Martin]

For example, petitioners win about 62 percent of the cases before the court but if the petitioner is asked fewer questions than the respondent, that probability increases to 71 percent. On the other hand, if the petitioner is asked more questions than the respondent, the probability that the respondent will win increases from 38 to 50 percent.

What are the standard deviations?

[IB]

If this correlation is meaningful (i.e. more questions from a PANEL, less likely to win), then specific JUDGES on a panel that ask a party more questions (relative to their usual number of questions and/or relative to other judges on the panel) should be more likely to vote against the party. Have these findings been validated by running this test?

[Alvin]

What about Justice Thomas? I heard he rarely asks questions. It'd be interesting to know how he votes the few times he speaks up.

[David Schraub]

I unfortunately don't have the standard deviations. I do know that with the exception of Justice Kennedy (for whom there is no correlation in any direction) and Justice Thomas (who, as Alvin indicates, doesn't ask enough questions for him to be measured), the findings do hold solid for the individual justices.