Is the ABA violating its own protocol for auditing law school employment survey results?

In my last two posts, I criticized ABA managing directors Barry Currier and William Adams for asserting, on the basis of highly deferential preliminary audits of the annual employment survey results from 10 randomly selected law schools, that law schools were accurately reporting their placement results. See here and here. 

These posts did not receive the customary number of page views or comments. The reason, I strongly suspect, is that I am one of the few persons, other than sub-150 LSAT-scoring 21-year olds, who was sufficiently naive to believe that the ABA audit of would amount to something more than a public relations ploy. Law schools scam up their placement numbers without consequences. The sun also rises. What else is new?

Nonetheless, I am going to flog this hobbyhorse onwards for the length of at least one more boring post.

Recall that five out of the 10 law schools selected for a random annual audit did not comply with the ABA's preliminary stage requirement that no more than 5% of their employment files be deficient (i.e. incomplete, inaccurate, or misleading). Accordingly, Adams informed the ABA Council of the Section of Legal Education by memo that a three-member ABA committee would decide, as a matter of discretion, which of the five schools would be subject to Level 2 Review (A Level 2 Review involves independent verification of some of the reported data, unlike a first-stage audit, which basically just involves a review of employment files for completeness). 　

In his memo, Adams stated that "If more than 5% of a school’s files are found to be deficient, a Level 2 Review. . .  may be ordered. . . . A committee of three persons, including me, has been appointed by the Data Policy Collections Committee to determine which of the schools with discrepancies should get elevated levels of review. . . After this further review of the schools with a compliance rate below 95%, the review committee will determine which of these schools may warrant a Level 2 Review." (emphasis added)

What is the problem here? The problem is that the ABA's "Protocol for Reviewing Law Graduate Employment Data, and Statement of Procedures for Collecting, Maintaining, and Reporting Law Graduate Employment Data" (hereafter: "Protocol") states that "If more than five percent (5%) of the Files are found to be deficient, the ABA will then proceed to a Level 2 Review." (emphasis added).  

Why then is there a need for Adams and the other two members of the review committee to deliberate and determine which of the five law school violators may be subjected to Level 2 review? Under the Protocol, they all should.  You see how the Protocol's "will then proceed" has turned into the memo's far more equivocal "may be ordered" and "which. . . may warrant"?  I think that every lawyer appreciates the often night and day difference between a mandatory procedure and a discretionary one, and Currier and Adams surely do. 

As a further confidence-builder, the members of this great triumvirate have been appointed by a larger ABA committee known as the Data Policy & Collections Committee (DPCC). The DPCC consists of two law school deans, two associate law school deans, four law school professors, one director of law school admissions (at bottom-tier New England Law), and one dean emeritus. So the three member review committee was specially selected by a group of persons who have a vested interest in the public perception that law schools are modeling the values of professionalism and are accurately representing that a JD usually leads to a good job.

Consider the following significant defects in the ABA Protocol and in the way the audit has been managed by Currier and Adams: (1) Only 10  out of the 205 accredited law schools, were selected for a random audit, a disturbingly small pool that Currier has no intention of expanding. (2) The first stage of an audit consists "solely" of a review of the employment files themselves and does not involve independent verification or confirmation of reported data. (3) The first stage of an audit accords the school a presumption that its employment files are complete, accurate and not misleading, absent credible evidence to the contrary. (4) Under the Protocol, up to 5% of a school’s employment files may be found to be incomplete, inaccurate or misleading without triggering additional review. (5) According to Currier, the names of the five schools that did not comply with designated minimum standards will not be publicly revealed.

Now consider the following praiseworthy aspects of the ABA Protocol, those that may genuinely protect consumers by deterring or exposing scammed-up employment survey results: (1) According to the Protocol, if more than 5% of a school’s files are found to be incomplete, inaccurate, or misleading, that school will be subjected to independent verification of some of its reported data, aka Level 2 review. (2) Sorry, this list only runs to one item. And yet it is this verification provision that has been quietly diluted. 

It is distressing that the ABA is, seemingly, unwilling to enforce its own standards. Are law school employment survey results anything more than fantasy fiction in highly granular numeric form?  Who knows?  The law school scam abides.