A piece of software called ChatGPT passed four out of four exams at the law school of the U of Minnesota. Most of the questions were multiple choice (a format that was not used at Old Guy's élite law school), but many required essays, and on some of the latter ChatGPT scored in the top half of the class.
The authors of the article cited above suggest that ChatGPT may be useful to practicing lawyers. Without wishing to boast, Old Guy admits that he has a reputation for writing excellent briefs (some lawyers call them the best in the jurisdiction), and he would never use a tool of this kind. It is evident from the article, and quite predictable, that the software just looked through material on the Internet and pieced it together into text, without understanding what was asked. Although the text was perfectly grammatical (which is more than can be said for that of many students), it gave no real analysis: it would often cite an authority on a point of law, but it would not analyze the applicability of the authority to the case or even offer clear conclusions or advice. (There appears to be a mode in which it gives definitive or at least assertive answers; that was disabled for the purposes of this experiment.) Old Guy will continue to write his briefs the old-fashioned way—by himself.
(It must be said that useful software packages known as expert systems have been around for many decades. An expert system asks questions of the user in an attempt to solve problems. These have been developed to assist physicians with obscure diagnoses. Conceivably they could help lawyers to explore options for litigation. Again, Old Guy relies on his own abilities. He can sometimes come up with a creative solution that would escape 99%+ of lawyers and that would probably elude software as well.)
The authors also suggest that students are likely to use tools such as ChatGPT, either to prepare for exams or to cheat on them. A student running short of time, for instance, could submit a question to ChatGPT rather than giving no answer at all. That would clearly constitute cheating (unless, improbably, the professor invited computer-generated answers). More profitably, the student might generate a first draft with ChatGPT, improve it, and submit the result as the student's own work. That too would be cheating. Or a student preparing for an exam could run sample questions through ChatGPT and rely on the output for guidance. That sounds like skating on thin ice. But ChatGPT can apparently outperform many toileteers already.
More interesting than the possibility of using software to generate briefs and such is the possibility of turning some legal work over to computers, thereby casting many lawyers out of a job. Software for writing wills has been around since the 1980s (taking the place of forms that used to be sold), and much other legal work is so formulaic that it may soon be automated. For clarity, Old Guy does not recommend that anyone produce a will with software or forms, although even a naïvely generated will may be better than none at all (yes, I said may!). But is the day far off when an expert system will be able to interview a testator and generate a respectable will—or a referral to a lawyer when the expert system cannot handle some difficulty (assets outside the country, complicated trusts, sophisticated estate planning)? Old Guy uses a template—his own—for the purpose, although it invariably requires alteration rather than mere filling in of blanks. Although Old Guy does not write many wills, he prides himself on producing superior ones. For uncomplicated situations, maybe the task could be automated.
Last month a company announced that its software was going to be used experimentally to defend a traffic ticket: it would listen in court and tell the accused (through an earphone) what to say, and the company would pay any fine. Perhaps some common defences could be handled effectively by software, and the options for some tickets are likely to be so limited that the approach might work some of the time. But effective computer-generated cross-examinations and closing arguments are probably a long way off.
Old Guy's work, in the main, is along the lines of briefs for court more than basic wills or defense to traffic tickets. He doesn't expect to be displaced by a computer. But some bottom-end legal work—filling out routine forms, calculating support in a divorce, document review, soon enough perhaps even traffic court—may soon be automated if it hasn't been already, and that will reliably reduce the number of openings for lawyers. Über-toilet law schools will suffer the most: after all, their graduates are the most likely to do bottom-end work as it is, and they cannot expect to migrate into something like appellate litigation that will be a human endeavor for some time to come.
