The Politics of Testing

The latest humiliation in the world of standardized testing involves significant scoring errors on the October SAT.  Most of the errors appear to have been small but some may have been as large as 300 or 400 points (out of 2400 on what is now a three-part exam, with each part still bearing a maximum score of 800). It is the outrage that interests me, and the potential involvement of courts and legislatures.  Why is it that people hate these tests so much?  Standardized tests play an important and perhaps increasing role in our society.  Ideally they provise a common denominator in a world where different schools have very different curricula and grading patterns.  And there is something of a free market (but only something, because in some industries rankings make it difficult for schools to ignore the standardized test scores of their applicants) in that employers, graduate schools, and others need not use or put much weight on these exams.

But the interesting comparison is to "normal" grades.  I suspect that even the worst of the widely used standardized exams produces fewer errors, and certainly fewer significant errors, than my own grading of law school exams.  Most professors take grading pretty seriously, but it is easy to make mistakes in constructing questions and in grading them.  We expect lawsuits against instructors, TAs (which we do not use in law school but were wildly variable in my experience as an undergraduate and then as a teacher who employed teaching assistants in economics courses), and universities to fail completely.  And schools are quite careful to discourage review of grading patterns and performance.  As between high school grade point averages and SATs (and then college grades and LSATs) can anyone doubt where more errors are to be found?  And yet the anger, and potential lawsuits and legislation, is directed at the standardized exam.

An obvious explanation is that most students experience many grades from many sources (teachers), so that the errors might offset one another.  Most applicants take a standardized exam once or twice and so an error is more important, or at least seems so. I am not sure this is so.  the number of test-takers who experience an error of 50 or more points, however we define error, is very, very low, and a comparable combination of non-offsetting errors in grades may well be easier to obtain. Colleges and high schools may also have an incentive to monitor their own grades, and indeed they have an incentive to give high grades and to see their graduates "defeat" others in the quest for spots in elite colleges and graduate schools. Test makers might seem to have more market power.  In reality, the opposite may be true.  The assembler of the standardized test will drop a vendor who makes errors, and there is even some competition (as between the ACT and the SAT) amog test makers.

There are many other explanations for the quantity and quality of rage.  Parents and students think of the College Board as an entity, and the test as a single all-important measure, easily described as unfair when it dooms a test taker to a future that seems less bright than that which grades alone would have pointed towards.  Grades received in courses are seen as dispersed, and although there are complaints about individual teachers, to be sure, most people see that bad experiences will tend to average out.  But of course many colleges and professional schools are driven by rankings and other reasons to care about applicants' numbers.  A GPA is a single number, as is the standardized exam that usually accompanies it in a single breath.

I doubt there is much of a legal issue here.  Most courts will want to stay away from the job of policing the College Board, much as they stay away from reviewing the work of teachers and TAs.  Moreover, test-makers are fairly good at public relations.  Even as I blog, they are assuring everyone that no one will be hurt by the error; fees will be returned; college admissions offices are reviewing affected applications and in some cases reaching out to the injured and offering them admission or financial aid where none was offered before. No one seems to notice that each piece of generosity or repair disadvantages some other applicant in a world with scarce resources.

In the long run, the way to keep the customers happy might be to increase transaction costs and divide the test into many parts and give but one part at a sitting.  The single standardized score would be the sum of five scores, say, each earned at a different sitting.  Complaints about testing conditions and unfairnesses would diminish, as they do for the components of GPAs. The testing industry would be happy, test takers' expectations would be adjusted slowly but surely, and many of the complaints would lose their intensity.

Comments

[Kimball Corson]

I don't insist people dress well. Now, being almost totally free, I often don't. I just say excellent dress matters and New Yorkers know that much better than people in LA do, just as the artistic and cultural scenes between the two cities are miles apart (except for the LA Philharmonic under your new conductor). That movies produced on the West Coast, which too often show things being blown up, are so popular is a measure, I submit, of how genuinely bored (and boring) most viewers are, from a lack of other well-developed interests to pursue. (The occasional movie I do see is typically a good foreign film.) But back to our subject: if, as hiring partner, you or anyone else had your choice between two good and otherwise substantially equal candidates for an associates position, but one dressed very well and acted accordingly and the other did not, the choice would be pretty obvious. I really do believe dress affects behavior and there are supporting studies on that point. Manners and social skills, like dress, matter a great deal to good lawyers and their clients. Even accepting variations in esthetical taste, there really is such a thing a “class,” in the context of dress, I believe. Sorry, I guess we really do disagree here.

[Kimball Corson]

Some afterthoughts:

1) You make my case on LA's vestial “tastelessness” by explaining dress-down business casual is becoming increasingly widespread in LA.

2) I don't think good dress should be a mandatory criterion for admission committees or hiring partners, only one to be recognized and given some weight when it appears – that is human enough.

3) In this highly competitive world, it seems imprudent to throw away any potential competitive advantages.

4) All this said, I did represent some high-tech executives who abhorred good dress and I would dress down for them when appropriate. Also, I once had a jury trial in cow-poke country and showed up for jury selection in cowboy boots (suitable scuffed) and a tan corduroy shooting jacket (which I never buttoned). When my opposing counsel showed up in a three-piece, cheap suit, I smiled, knowing a comparative advantage when I see one.

Again, dress matters, even where people, as in LA, struggle to make that not so. Again, the reason is dress affects behavior.

[The Law Fairy]

Kimball, I don't understand your beef with LA. What makes you say people in LA have no sense of what's appropriate? Have you ever lived here?

[Kimball Corson]

Law Fairy,

I did not say those in LA have no sense of what is appropriate. I say they are comparatively tasteless and unaculturated to the arts compared to New Yorkers. While I have not lived in LA (and would not), I have travelled there and had legal business there and in San Franciso many times. By the way, those in LA fair less well too in these regards than those in San Franciso, I think. That too says something.

[The Law Fairy]

Okay, Kimball, but what's your barometer for taste and culture? What makes New York (or you) the arbiter of good taste, other than your own predilection (what I would call an East Coast bias)? I've traveled to New York and lived in Chicago (which, I assume, also fares better than LA under your measure of "taste") and can't fathom ever wanting to live in New York rather than here. For one thing, I appreciate how Angelinos (outside of Hollywood and Beverly Hills, that is) don't place an unbalanced focus on something as minor as what people wear.

[MMJ]

Kimball, you seem to have shifted from an inadequate $700 suit to dressing down when appropriate. Would it not be easier to just say "dress appropriately"?

[Kimball Corson]

Possibly, but people's concepts of "appropriately" vary so. In any event, I felt compelled to dress well while practicing law. But now that I am retired and sailing the world on my own for the next couple of decades (should I live so long), I am beholden to no one and much less interested in what other people think of me, so I largely dress for my own comfort, the climate and my own convenience, though I do have good clothes on board. "Appropriately" is so conditional.

[Kimball Corson]

Law Fairy writes,

“Okay, Kimball, but what's your barometer for taste and culture? What makes New York (or you) the arbiter of good taste? . . . [I prefer people who do not have] an unbalanced focus on something as minor as what people wear."

I respond:

That is a hard question to answer. My son has put it to me unidimensionally another way: “Why is what you consider good music, really good music?”

In areas of the world where all or most dimensions of living are consciously viewed as having the potential for artistic or aesthetic expression or at least a savoy faire aspect, art and culture tend to flourish and grow and life is commensurately richer, more engaging and more interesting. Consensual arbiters of taste can then emerge called critics and they are listened to thoughtfully and sometimes challenged because people are attuned to care about things and pay them all attention. Taste then matters.

Where too many people think too little really matters or is important, and don’t give thought to much or these things, life is lacking and the people themselves are less acculturated and less interesting. That all aspects of living should be well considered means dress as well. It is not a question of unbalanced focus on something as minor as what people wear; it is a question of a full life with all aspects well and interestingly considered.

Life is as rich as we make it.

[The Law Fairy]

Kimball, I understand your broader point, which seems to be that the potential for self-expression and creativity is everywhere. But I still don't understand how people are less interesting or less acculturated (do you mean cultured? Because certainly people living in Los Angeles are acculturated to Los Angeles culture) just because they choose to express themselves in ways that are different from how others express themselves.

If I take two hours to get ready for work, taking my time to carefully lay out my clothes, painstakingly styling my hair and makeup, I will be expressing better "taste" per your rubric, but I will have less time to express myself in other ways, as through my work, or perhaps through painting or creative writing. Would you admit, at least, that appearance *can* be given too much import? Or would you honestly prefer that your date keep you waiting an extra hour, lest you think her tasteless for not dressing herself up sufficiently?

[Kimball Corson]

Law Fairy writes,

“I understand your broader point, which seems to be that the potential for self-expression and creativity is everywhere. But I still don't understand how people are less interesting or . . . cultured . . . just because they choose to express themselves in ways that are different from how others express themselves.”

I respond:

It is a way of looking at life. The cultured (you’re right) think in esthetical terms and find avenues of realization and expression in much or all they see and do. They are responsive to each other and critics and the informed in these regards. They are attuned and considered in most all quarters. They think about these things that make up their lives and given them meaning and interesting aspects. Those who don’t have this mindset are less interesting and cultured because they ignore too much, there is too much they do not consider well and they are attuned to too little. They don’t care or understand. Thought about such things and their esthetical aspects is lacking. So it is not the case often that such people just choose to express themselves differently. More typically, they are just fundamentally lacking in what I describe and not considerately focused, and that makes them less cultured and interesting.

Law Fairy again:

“If I take two hours to get ready for work, taking my time to carefully lay out my clothes, painstakingly styling my hair and makeup, I will be expressing better ‘taste’ per your rubric, but I will have less time to express myself in other ways, as through my work, or perhaps through painting or creative writing.”

I respond:

There are always trade-offs, I admit. I am not suggesting a perfect, time-consuming get up for every occasion, only a considered and esthetically interesting one. When the mind-set I suggest is present, it can become second nature, natural and not time consuming. The point is life has many aspects and we wish to give as many of them as possible worthy meaning and interest for us and others in order to make life generally more interesting and richer for us all. Balance is part of this adventure as well. Some people who are excellent at it make it seem effortless, but that is a learned skill and it is by no means effortless.

Law Fairy continues:

Would you admit, at least, that appearance “can” be given too much import? Or would you honestly prefer that your date keep you waiting an extra hour, lest you think her tasteless for not dressing herself up sufficiently?

I respond:

Appearance can of course be given too much importance, especially when it interferes with other things. Too often, though, those who do that, for want of savvy and an esthetical sense, very often wind up looking terrible, but quite perfectly so. Someone with a good esthetical sense and dress savvy can often be ready to go very quickly and faster than someone lacking those qualities who struggles with his or her appearance. Again, the trick is balance, within the framework I posit, and I have known people who do that spectacularly well. They are a pleasure to watch live.

[The Law Fairy]

Kimball, I feel like on some level we have basic agreement -- I wholeheartedly agree with you that people who take the time to think about things and give thought to expressing themselves are more interesting, and I suppose one could call them more "cultured" as well, in that they've matured to a point where they recognize the beauty to be found almost anywhere.

But I'm still not seeing how this makes one particular manner of dress superior to another. Indeed, I would venture that given your definition, which pays meaningful homage to self-reflection, would deem "more cultured" a hippie living on a commune than a rich sixteen-year-old whose entire wardrobe is supplied by Burberry. I still don't think you've adequately explained the connection between creativity/self-reflection/maturity and expensive, snappy dressing.

[Kimball Corson]

Law Fairy writes:

“But I'm still not seeing how this makes one particular manner of dress superior to another. Indeed, I would venture that given your definition, which pays meaningful homage to self-reflection, would deem "more cultured" a hippie living on a commune than a rich sixteen-year-old whose entire wardrobe is supplied by Burberry. I still don't think you've adequately explained the connection between creativity/self-reflection/maturity and expensive, snappy dressing.”

I respond:

Fair enough. A good question. I respond with a question: which style of dress is more considered and thought out, in a cultured, good esthetical way with a sense of what critics (within the framework I have described) would consider fashionable or at least interesting or perhaps tasteful – thrown-on-without-much-thought hippy attire and business casual, on one hand, or interesting, considered dressing, on the other?

Hippy attire (often dirty) and typical business casual reflects a quick, common denominator, largely thoughtless approach to dressing that seriously discounts dressing as a potentially meaningful experience for the dresser and others. It brushes aside thought about clothes and looks to being dressed more like others in the putative group – sometimes a sort of quick brand identification, if you will. It is not so much an issue of self-reflection that should be involved as clothes reflection, as an esthetical aspect of a life experience. It becomes in good taste when it is very well considered, within the framework I have described. It is not just or predominately a question of money (which surely helps), but more a reflection of the thoughts and aesthetic behind the dress.

I had a particular girlfriend too many years ago who had an incredibly good knack for it. She could reach into her, her mother’s and her grandmother’s closets and come out smashing every time and always appropriate for the occasion. She was an artist by mind-set and trade. Not all of us have that innate capacity and that is why we need thoughtful and good critics to help our thinking along. Such a critic would not likely consider hippy attire or business casual very interesting, largely because it reflects the rejection of dress as being worthy of serious consideration and is too dismissive of too much.

The situation is similar to the consideration of good music. Those who are prodigies, critics and well-educated musically do not gravitate toward the simple, less thoughtful and less well-crafted musical structures of rap (rhythm talk, as I call it – no melody, no harmony, no counterpoint, etc.) or other music popularly in vogue. The overwhelming tendency in taste of such people is toward the more interesting, thoughtful and well-structured music that has a higher order aesthetic and that has become classical for those reasons.

I hope I make some sense here. Esthetical taste is hard to describe, but like pornography for Justice Stewart, I believe it was, it is rather easy to recognize.

[The Law Fairy]

Kimball, I see what you're saying but I think you're unfairly dismissive of people who opt not (or can't afford!) to dress extremely "well" everyday. I put to you that hippies do in fact put careful consideration into their clothing -- their clothing, dirty or dissheveled as it may appear to you -- is their expression of their rejection of the dominant culture. They look at people you call cultured and better-dressed (and therefore more interesting) and they see cookie-cutter minions for The Man. In many cases, I suspect they're right. Their clothing choices may look unconsidered to *you* simply because you would dress that way only if you were being careless about your appearance. But for them, it is the result of their consideration. Are you saying, then, that rejection of the dominant culture is not a legitimate expression of taste? I'm sure you'd agree with me that this line of thought leads down an unpleasant road.

As for business casual, I'm not sure why you're so dismissive about it. I think carefully each day about what I wear. I'm grateful that I don't have to dress extremely "up" every day because if I did, that would be more expensive. My firm's business casual policy frees up more of my money for things like paying off debt. Certainly, I could choose to dress "better" for work than I'm required to -- but does the fact that I wear business casual everyday make me a less interesting person in your view? If not, how does this square with your alignment of one particular form of taste and general elightenment?

[Kimball Corson]

I am not dismissive of such people. I only find them uninteresting in that regard and often correlatively many other regards as well. But this is not always true and I am always prepared to look past dress for more where I suspect it exists. However, life is short and we must choose our friends considerately.

Similarly, if hippies wish to proclaim themselves as such vestially, I have no problem with that, except do not expect me to particularly appreciate their cultural views any more than they appreciate or know mine. Rejection of dominate or any specific culture is certainly permissible and may of course have vestial expression as well. Part of what helps distinguish good taste, as I have tried to explain it, is the lack of it in many quarters. It stands in contrast, not only in this regard, but from other considered tastes as well. Differences here enrich us. Uniformity is not usually very tasteful, for the reasons I have implied. There is little interesting in uniformity, even counterclutural uniformity.

As for business casual, I am no more dismissive of it than are those who dress that way dismissive of thoughtful, interesting or caring dress. It is business casual that is dismissive of what I suggest because too often it is uniform, uninteresting and brings too little to the table. This does not need to be so, but it just very often is.

Also, to be clear, I am not necessarily talking about dress that is expensive, hyper carefully doned or even maticulous as you seem to imply. I am taking about dress that is esthetically intersting or engaging in some interesting manner or has some aspect to it that is tastefully thoughtful. Too often, as I have explained, dress in bad taste can be very expensive and time consuming to done. Hollywood shows us that in spades. Taste and money are not correlatively locked. You may very well be tastefully dressed in business casual, but perhaps not. I do not know.

Hope this helps.

[Kimball Corson]

Law Fairy,

Where did you go? To change clothes?

[Kimball Corson]

To change the subject and to get back to testing, are the top tier law schools relying excessively on LSATs to cut applicates and keep their numbers up, even if it means that truly better applicants are rejected?

To be sure there is the US News & World Report competition and also there is too much gradeflation in regard to transcripts, but is this sensible where LSATs are so poor at predicting first, law school sucess and second, professional success?

Are we being held hostage by the numbers competition?