At some point computers will be able to provide better, cheaper, and faster legal advice than humans. No human can fit all of the law in their head, and don't always offer the 100% accurate advice. Not everyone can afford a lawyer.

Planes with autopilots can fly cheaper and "better" than human pilots. We still have human pilots.

I want a lawyer who can do their work more effectively because they have assistance from LLM-powered tools.

I might turn to LLM-only assistance for very low-stakes legal questions, but I'd much rather have an LLM-enhanced professional for the stuff that matters.

On the other hand, would you rather have a math professor calculate the square root of a large number for you, or would you use a calculator?

I'd rather have a math professor teach me why I'm doing a square root of a large number. Same thing applies to lawyers. All this talk about automating away complex professions is just that. LLMs are tools people use, not lawyers, doctors or professors.

> No human can fit all of the law in their head,

Good thing there is no need to!

This service may have been better with a higher context window but with the required accuracy of legal document writing the inaccuracy of the RAG systems are too high.

Also, people have actually used it in practice and it didn’t go that well. So human in the loop systems in practice should have users finding corrections but won’t occur when you release the product.

https://qz.com/chat-gpt-open-ai-legal-cases-1851214411

The systems described in the article don’t sound like RAG at all.

RAG systems have a much lower propensity to hallucinate, and generate verifiable citations from the source material.

Though I think they’re better as research aides than “write the final product for me.”

My number one request is still: "please rewrite this legal answer in simple language with short sentences." For this, it is amazing (as long as I proofread the result). For actual answers, eh...

I assume you're in the legal profession? Do you find, as you proofread, that you want to insert caveats and qualifiers into the "simple" language and end up with something like the original legalese?

Legal language is what it is in large part because simple short sentences are too imprecise to express the detail needed.

Indeed, a corrolary in computer science is the the reasoning behind why using the Go programming language is, in general, a major mistake.

Simple vocabularies, while attractive will inevitably fail to properly describe systems of any sufficient complexity without long chains of hard to follow low-level logic.

Any system of sufficient complexity necessitates the use of more complex vocabulary to capture all the nuances of the system. See: Legalese, medical jargon, chemical naming schemes, the existence of mathematics itself, etc..