Cutting through the noise: The impact of GPT/large language models (and what it means for legal tech vendors)

A little knowledge can be a dangerous thing, and as the legal industry has become savvier about artificial intelligence, there is a temptation for some to write off OpenAI’s generative pre-trained transformer (GPT) technology as just another big fuss that will be talked about incessantly before things calm down and return to normal (hashtag #bringbackboring). On the other end of the spectrum, there are those forecasting the end of lawyers, again.  

In the interest of cutting through some of the noisier noise, I sought a few answers from AI expert and law professor Daniel Katz, who teaches a variety of courses at Illinois Tech Chicago Kent College, and is director of The Law Lab at Illinois Tech. He co-founded legal analytics company LexPredict, which was acquired by Elevate in 2018. The conversation happened at the end of January, and I’ll come back to reflect on some of the developments that have happened in just a few short, crazy weeks since then, and how that fits in with what Katz predicted. Spoiler alert, Katz says that for legal tech companies that are based on a Word plug-in, he would be very concerned, but more on that in a minute. 

Is GPT the real deal?

You may or may not be aware that in December, Katz and fellow legal scholar Michael Bommarito put GPT-3.5 through the US Bar Exam, which ordinarily requires at least seven years of post-secondary education, including three years at an accredited law school and months of exam-specific preparation. Approximately one in five test takers still score under the pass rate on their first try. GPT-3 achieved an overall score of 50.3% (significantly above the 25% baseline guessing rate) and a pass rate for Evidence and Torts. Katz and Bommarito’s conclusion is that in the near future,

Read the rest