This year, the ABA finally began auditing some of the data in the annual law school ten-months-after-graduation employment surveys. Law schools have been required to conduct these surveys since 2012, following revelations that they had been routinely and brazenly misrepresenting their graduates' employment and salary outcomes in promotional materials and to US News. The ABA audits are obviously a welcome development, though it is remarkable that auditing has only been initiated this year, the fourth year of the mandatory survey. At least the audits have been conducted by an independent firm, the Berkeley Research Group (BRG).
Of the ten law schools that were randomly selected annually for an audit (i.e. a "Random School Review"), five presented the auditors with deficient employment files in sufficient number to violate the requirements set forth by the ABA, and four of these five schools were significantly or flagrantly noncompliant. Moreover, of these four schools, one school apparently tried to create supporting documentation after it had been selected for an audit. So did one other school that was not selected for a full-blown audit, but only asked to submit a couple of files pursuant to "Random Graduate Review." (See Section B)
A. The ABA Protocol for Reviewing Employment Data
This section summarizes the key features of the ABA's 11-page "Protocol for Reviewing Law Graduate Employment Data, and Statement of Procedures for Collecting, Maintaining, and Reporting Law Graduate Employment Data." (hereafter: "protocol"). If too boring to read in its entirely, just note points 2 and 4.
1. Schools act at their own risk when reporting data that have not been adequately confirmed or when supporting documentation is lost or not maintained.
1. Schools act at their own risk when reporting data that have not been adequately confirmed or when supporting documentation is lost or not maintained.
2. A law school employment data audit has three levels, of which only the first level is mandatory. At Level 1, "Documentation included in the Files will be presumed to be complete, accurate, and not misleading in the absence of credible evidence to the contrary."
3. Only at Level 2 and Level 3 is there verification or independent confirmation of submitted data. A Level 2 audit consists of verification of employment outcome data for 20% of the class. At Level 3, a law school must hire an independent firm to review and confirm all of the school’s reported employment data.
4. An audit may proceed to Level 2 if "more than 5%" of the files are found to be incomplete, inaccurate, or misleading.
5. Not every ten-month-out JD fills out the written placement survey. Law schools are permitted to obtain employment data from these grads by calling them up and asking them (i.e. "indirectly completed employment surveys"). Failing that, they are allowed to guess the employment status of non-responsive grads from comments related to employment status that the grad posted on social networking sites or elsewhere (i.e. "employment information from sources other than the graduate").
6. Information gleaned from "telephone or other oral communications," must be committed to writing as soon as it is obtained and before the reporting deadline. Documentation of oral communications must include the date of the communication and the parties involved in the communication. (Obviously, in appropriate recognition of the potential for fraud as to alleged data obtained in this manner).
7. As noted, ten law schools are randomly selected each year for an audit of their employment result data. Other schools may be subject to an audit if they are red-flagged for credible reports of significant inconsistencies or inaccuracies in their data reporting. Schools that are not selected for a full-blown audit may be asked to provide a very small number of randomly selected files for "Random Graduate Review." If any of the files submitted by a law school pursuant to Random Graduate Review is deficient, the law school is subject to an audit-- starting at Level 1, and if warranted, proceeding to Level 2 and Level 3, as described above.
B. ABA Memo: The Preliminary Findings from the Initial Stage of Employment Data Auditing and Review.
On August 5, 2016, Bill Adams, Deputy Managing Director of the ABA Council of the Section of Legal Education and Admissions to the Bar (and former Dean of bottom-of-the-barrel for-profit Western State College of Law) sent the Council a memo entitled "Employment Data Review Update," which contained preliminary findings from BRG's initial review of law school employment data. According to Adams's memo:
- "Five of the 10 schools randomly selected had compliance rates of 95% or above. Another school is at 94.8% and two are at approximately 86%. The remaining two schools had file compliance rates in the mid-50’s. BRG is conducting follow-up discussions with the schools that had more than 5% of their files to determine whether some of the files that appear to be deficient are actually deficient. One of these schools also appears to have created its supporting documentation after it was notified that it had been selected for a file review. After this further review of the schools with a compliance rate below 95%, the review committee will determine which of these schools may warrant a Level 2 Review.""
- "For the Random Graduate Review, 382 files were selected from 156 schools. Of the 156 schools that had files randomly selected, 16 appeared to have a potential problem with missing items or supporting documentation for an item. Of these 16, 8 had minor issues regarding documentation of an item and probably will not warrant further review. . . .The remaining 8 may have issues requiring heightened review, but BRG is engaged in discussions with the schools to seek clarification about documentation or ambiguities in the file. In regard to one of these [other] 8 schools, it appears to have created its documentation for its files after it was notified that its files would be audited."
C. Analysis
In the one-paragraph long summary section of his memo, Adams puts an unwarranted happy spin on the preliminary results. He states that "[t]he good news is that the overwhelming majority of schools subjected to the data review have both accurately reported employment results and provided credible documentary support of what they have reported" and he goes on to commiserate with the law schools over the "complex" documentation requirements imposed upon them.
Adams's attitude is especially troublesome in that he is one of a three-member committee that will determine which of the schools with discrepancies will get elevated levels of review. (In fairness to Adams, he at least recognizes that "the two schools that appear to have created their documentation after the fact raise more serious problems").
Adams's attitude is especially troublesome in that he is one of a three-member committee that will determine which of the schools with discrepancies will get elevated levels of review. (In fairness to Adams, he at least recognizes that "the two schools that appear to have created their documentation after the fact raise more serious problems").
Adams's "overwhelming majority" comment is obviously inaccurate as to the 10 randomly audited schools. As to these schools, only five out of 10, a decided non-majority, achieved compliance rates of 95% or above, as the protocol requires. I would not be a stickler about the school that provided 94.8% file compliance, but four schools fell drastically short of the 95% requirement.
Though enjoying a generous presumption that its files are not incomplete, inaccurate, or misleading, two of the ten schools selected for a random audit nonetheless provided incomplete, inaccurate, or misleading documentation as to 14% of their files, or nearly triple what the protocol allows. Two other schools provided incomplete, inaccurate, or misleading documentation as to almost half of their employment files, or almost ten times what the protocol allows.
Suppose that these results are representative of all 205 accredited law schools. That would mean that 82 law schools (40%) would be unable to provide the requisite percentage of non-deficient files and that 41 of these 82 (20%) are sitting on so many incomplete, inaccurate, or misleading files that their reported employment data must border on fiction. These are shocking results. The ABA's protocol for reviewing law graduate employment data should be immediately amended to require a random audit of far more than 10 schools per year, and to require verification of reported data at the initial review stage.
Otherwise, we will be left to wonder whether a school's employment survey results are really just lies in highly granular form, a whole bunch of numbers that represent not actual jobs for recent grads so much as a school's success in hoodwinking the ABA and its prospective students.
Otherwise, we will be left to wonder whether a school's employment survey results are really just lies in highly granular form, a whole bunch of numbers that represent not actual jobs for recent grads so much as a school's success in hoodwinking the ABA and its prospective students.
Whenever the pigs enact "change" or authorize a study, they usually give themselves and their fellow swindlers extra time to get in compliance. Just look at Congre$$ and their worthless-ass reports, printed months after hearings long-since forgotten. What an "honorable profession," huh?!?!
ReplyDeleteThe brazen sloppiness of record keeping is readily apparent, thanks to audits such as these. The Law Schools have largely been making stuff up concerning employment results for decades. Full stop.
ReplyDeleteBut, the blame as always falls on the student/mark, who does not engage in false reporting or document production after-the-fact. These are your "ethical" law schools, folks.
Some schools report that less than half of graduates get real lawyer jobs. If they have to lie to get the numbers up to 30 or 40%, what the heck are the real numbers?
ReplyDeleteThe audit should not be random, for plainly some schools fall under much greater suspicion than others. I'm not particularly concerned about the accuracy of Harvard's data; I do, however, suspect the scores of Cooleys.
ReplyDeleteImagine if the IRS selected taxpayers to audit totally randomly and audited a bunch of $40k households with no passive income while the audit rate for complex or suspicious returns fell like a rock.
DeleteYou have got to go further out than first year jobs to get the dirty data on Harvard. It is dirty and it is swept under the rug.
DeleteFor goodness' sake. Attorneys have to register with the state bar and provide employment information, and, if unemployed at a law firm, at least have to provide a current address. How hard can these graduates be to track down?
ReplyDeleteOh yeah, 1/2 aren't registered with the bar because they couldn't pass the bar exam. Whoops.
No way. Administrators aren't nasty. They would never Rig things. Believe me.
ReplyDeleteThe 10-month employment surveys need to be conducted by an entity other than the law schools, because the law schools have every incentive to fudge the data (and some have even been caught doing so in the past). Some states require law students to register with the bar on commencing law studies. Perhaps this should be a requirement in all states, with the progress of each student tracked by the state each year through law school and employment outcomes/bar pass information after law school. Then, each state's report would require an audit by an independent auditor and that auditor's opinion letter as to the reliability of the data.
ReplyDeleteBanks must present audited financials to their regulators; public companies must present audited financials to the SEC; retail tenants who pay rent based on a percentage of their sales must present audited financials to their landlords. If your business wants to borrow money, the bank will require audited financials.
Independent accountants and auditors are the gold standard for trust. Most of us learned this in law school of all places!
The ABA random audit of just a few law schools, inadequate though it is, still turned up bad numbers! Wonder what would the FULL DATA SET look like!!
Oh would I love to see a few investigative news reporters, armed with a forensic accounting team, take a DEEP DIVE into all of the law school employment outcomes data, and present an in-depth report on what they find!
Just impose a minimum LSAT score, such as 156 (with a possible exemption for, say, 3% of the class). That simple measure would eliminate a multitude of sins—not to mention a multitude of law schools.
DeleteGood suggestions, but there also needs to be a limit on law school spots so it is some function of jobs - like 115% of the first year FT JD required placement. Supply and demand don't work otherwise.
DeleteIf you look at the placement of graduates from top law schools several years out, there is a further problem of too many first year spots for lawyers to work a career. The longitudinal demand for lawyers who are experienced should be taken into account in setting maximum law school enrollment.
Setting the LSAT score high enough should do the trick. Only some 23,000 people get 156 or better, and many of them won't study law.
DeleteEliminating the easy Federal loan money would end the lawyer mill pretty quickly. The "standards" argument really plays on the presumption that the loan gravy train does not end.
DeleteThis tells us that the 88% FT JD-req'd placement figures disclosed for most top 8 law schools may be more like 83% placement in such jobs. You may have a good 8 to10% or so of the class at the top 8 law schools not placed in jobs that it makes sense to go to law school for.
ReplyDeleteCompare the T8 placement rate to that for MDs graduated from US med schools into residency - 93.6% out the gate and 96.5% including those that had to reapply.
Without even getting to up or out policies, a top 8 law school degree today presents a much higher risk of unsatisfactory first year job placement than an MD degree from the lowest ranked or an unranked US med school.
Thank the ABA for not attempting to limit the supply of law graduates to some function of the demand.
What the ABA sees as opportunities for bright young lawyers is really structural unemployment and underemployment coupled with a quarter million dollar cost for law school, even at the top, without prospects of a job that pays enough to pay off that cost of attending law school.
At the bottom law schools, maybe even most of the bottom ranked half, this audit tells us that they have no reason to exist because their placement is so poor.
10:30, that is spot on. It is worth going further with their analysis and comparing foreign medical schools to the bottom feeding law schools. There are foreign medical schools that are producing better results than the bottom feeding law schools. Overall, 53.9% of U.S. citizen graduates of international medical schools applying for a residency matched in 2016. There are many law schools that wish that 50% of their graduates obtained a full time, long term, JD required job. And those residencies pay salaries in the $50k range, with health insurance, life insurance, disability insurance, paid vacation for a few weeks per year, and other perks like free food at the hospital cafeteria while working. Shit law and document review do not come close to offering those kind of benefits. Many of the foreign graduates are unable to obtain competitive residencies, like neurosurgery, etc. Most end up in primary care residencies like internal medicine or family practice. So when they complete their 3 year residency, they will earn a salary comparable to a big law associate. But many of them do go onto fellowships after residency, and go into the higher paying specialties like cardiology or gastroenterology.
DeleteThe comparison between passage rates on the professional exams is also shocking. In 2015, 72% of students of foreign medical schools passed the USMLE step 1 exam. This year, we are seeing state bar passage rates plummet well below 72%. Arizona’s bar passage rate was barely above 50%. Pennsylvania’s bar passage rate dropped to 69%.
I would also add, because of the shortage of doctors, U.S. medical schools and residency programs hire foreign educated doctors to teach. Personally, I have been taught by a graduate of a foreign medical school. They are an incredible physician and teacher. They were much more impressive than the Harvard educated buffoons that played hide the ball in law school. Looking back, that was always one of the insulting aspects of going to a lower ranked law school. They are not interested in hiring their own graduates. They only want to hire graduates of the elite schools with minimal practice experience at a big law firm. But doctors can obtain academic jobs teaching, even if they are a foreign graduate.
I've had to point this out many, many times, but I'll do it again. The ABA used to be a lot stricter but then some school that wanted to get rich off a law school and got rejected went whining to the USDOJ anti-trust people who went after the ABA and they had to sign a consent decree. Sure, sure it has expired but were they to go back to employing real standards they'd end up in the same place. Law schools are accredited by the ABA and medical schools by the AMA, but here's where they differ.
Delete1. Residencies are funded by the gubmint. There are already not quite enough to go around so churning out more new graduates will accomplish nothing. Granted, those Caribbean schools show there are a lot of wannabes out there, but because the gubmint controls how many people can get licensed the AMA cannot be accused of stifling its own potential competition, which is exactly what USDOJ accused the ABA of doing, and unlike the feds they don't have an anti-trust exemption.
2. Law schools used to be cash cows. To some degree some no doubt still are for their universities, while most are now just a gravy train for their faculties. But they do lend an aura of prestige even if they just break even. Med schools on the other hand tend to be money losers so absent a huge endowment there isn't much motivation for anyone to open one other than prestige without breaking even on the deal. There are schools with ten figure endowments that don't have a medical school.
3. The scholar who cannot get into med school can always try dental, podiatry or optometry school and still get to put "Dr." in front of his or her name. No analogous situation exists in the legal field, it's all or nothing.
I have no axe to grind for the ABA, I hold that organization in total contempt, but the truth is the truth and in this case they are getting a very bad rap. The ultimate solution is to give acredidting to an independent body that cannot be accused of anti-competitive practices
Nobody forced the ABA to pack its board with law-skule scamsters.
DeleteNobody is preventing the ABA from enforcing its pitifully poor standards.
Nobody is forcing the ABA to stay in the business of accrediting law schools.
Nobody is stopping the ABA from setting more meaningful standards.
12:21- You are completely incorrect that any attempt by the ABA to limit the supply of law schools or law student places in law school today will meet with antitrust challenges. The issue of lawyer supply and demand was not raised at the time of the antitrust decree. Surely the issue of loading up people with huge educational debt and terrible employment outcomes is not a goal of the antitrust laws.
DeleteJust like the for-profit colleges have recently been forced out of the federal loan system, it is easy for the government to do the same thing to underperforming law schools.
The ABA as accreditor is REQUIRED to take into account employment outcomes, so they surely could limit the number of law schools and law school places based on the demand for lawyers.
What is egregious about the antitrust argument which the ABA uses to keep accrediting law schools in the face of a profession where there are not even jobs for 50% of all graduates of accredited law schools is that the ABA has not even tried to publicly analyze the legal issues they face as accreditor. Why not convene a committee on the antitrust issue? Why not write a report? Why not ask the Justice Department if the rule is "i pledge allegiance to the United States of antitrust" or whether the Justice Department seriously believes an accreditor has to allow an unlimited number of professional school slots to open in every professional field, and antitrust actually is the only relevant law.
As far as I know, I have not and will not pledge allegiance to the United States of antitrust. This "Antitrust Uber Alles" attitude of the ABA might as well be the Nazi party speaking because the ABA's taking this antitrust doctrine to an extreme level of enforcement has ruined countless American lives in the name of a doctrine that has no place in the current severely oversaturated market for lawyers. By enforcing the antitrust doctrine to an extreme, the ABA is simply ruining lives.
The ABA is tone deaf to the economic and personal disaster of severe lawyer oversupply and should be removed as accreditor.
12:21, you do make some accurate points. The government limits the number residencies. But U.S. medical schools are not prohibited from churning out more graduates. U.S. medical schools choose to limit the number of grads so that the vast majority of them can obtain residencies. That’s why you are seeing foreign graduates fill the remaining residency slots. The medical schools aren’t churning out more grads and telling them if they can’t practice medicine, they can get an MD advantage job.
DeleteAnd law grads are not the victims here because of antitrust problems and the economics of legal education. Law schools lied for years about employment outcomes, long before the great recession. Students relied on the fraudulent stats and enrolled at these schools. And because students believed the stats, the law schools drove up tuition. The law deans would tell prospective students not to worry about paying back the $100k+ student loans to attend, because every student got a job and private practice salaries averaged close to $100k.
Law schools were also running other schemes, like conditional scholarships with section stacking. By stacking all of the conditional scholarship students into a section graded on a strict curve, the law schools were guaranteed to have a group of students that would lose their scholarships and become paying customers. No other professional schools ran schemes to take away scholarships.
These schools also knew that legal employers were only interested in hiring a handful of their grads. The schools had to find a way to identify these grads for the employers. So the schools graded courses on strict curves and the professors played hide the ball in class using the Socratic method. Medical schools grade students using a pass/fail system and actually teach the students how to practice medicine. Business schools and other professional schools actually teach their students instead of hiding the ball.
Additionally, the law professors typically have very little legal experience and do not practice law. They are chosen because they went to an elite law school and this pedigree somehow improves the prestige of the school. But these professors do not provide any benefit to the student. The professors claim they need exorbitant salaries because they gave up million dollar a year careers, and they need light workloads to research and write their law review articles. Legal practitioners have complained that these articles are worthless. Compare that to medical schools, where the MDs teach, see patients, and do research. The experience and expertise of professors at medical schools benefits the students.
All of this has been exposed by the law school scam blogs. And this resulted in a plunge in applicants. Now the law schools admit large numbers of students at risk of failing the bar for the sole purpose of collecting student loan dollars. Law schools have operated for years as a moneymaking scam for the benefit of the administrators and professors. The schools have shown that they do not give one damn about their students.
The answer is very simple: end federal loan money from flowing into underperforming institutions. The ABA can accredit whatever it wants, but if you end the duty-free student loan gravy, you end the scam. If some rich putz wants to go to an overpriced trash pit, then follow up with a few short stints in unpaid internships or shitlaw before taking over the family business, that's fine with me.
DeleteBut... the Democratic Party isn't going to let that happen because you know, racism and shit, i.e. Protecting the real reach and privileged.
I'm even fine with getting rid of all barriers to practice. Let's call the bluff. Every time someone raises the issue of standards, the accusation of racism and sexism flys left and right. It's better to just let the whole thing come down quickly and with minimal pain to future participants than letting the thing die a slow, rotting, and cancerous death with a generation of debt slaves and bilked tax payers as collateral damage.
What is worse about the law school complex is placement statistics based on jobs that are not career-track jobs. Placement is dependent on a class of associates or clerks being replaced by each younger law school class. That the only reason why there is anything close to 50% placement rate of law graduates.
ReplyDeleteAfter those short term jobs end, many elite lawyers are left without full-time permanent JD-required jobs.
The short-term horizon of the jobs that law schools rely on for placement is unlike most other industries or professions, unless you are talking about the military, elite consulting firms or tenure track professors. What is unlike the tenure track for professors- you have 10 or 12 hires in law for every partnership that is awarded. Furthermore, the partnerships are anything but secure, many last only a few years, with some former big and mid-law partners unemployed or underemployed long term.
Big firm associate jobs generally force the lawyer to terminate employment after a few years and clerkships by definition last only a few years.
The placement numbers published by the ABA at their best present a misleading, rosy picture of lawyer employment. The scam is lack of disclosure of placement several years out of law school. For white men, the numbers several years out are mixed. For women and minorities especially the numbers are bad.
Only 82 schools are sitting on fictitious results? What's wrong with these people! As the football coaches say, "If you ain't cheatin', you ain't trying."
ReplyDeleteScam on.