site banner

Friday Fun Thread for July 4, 2025

Be advised: this thread is not for serious in-depth discussion of weighty topics (we have a link for that), this thread is not for anything Culture War related. This thread is for Fun. You got jokes? Share 'em. You got silly questions? Ask 'em.

1
Jump in the discussion.

No email address required.

Tooting my own horn. December 1, 2022 I predicted:

My honest bet is that any student currently in their first year of Law School will be unable to compete with AI legal services by the time they graduate. Certainly not on cost. The AI didn't incur 5-6 figure loans for it's legal training.

Put another way, the AI will be as competent/capable as a first-year associate at a law firm inside 3 years.

This was before GPT4 was on the scene. Reiterated it 3 months ago

And then today I read this nice little headline:

Artificial Intelligence is now an A+ law student, study finds

If they can stop the damn thing from hallucinating caselaw and statutes, it might already be there.

But, let me admit, that if we don't see downward pressure on first-year wages or staffing reductions this year, I missed the meatiest part of the prediction.

There's the counter-argument that AI lawyers will actually stimulate demand for attorneys by making contracts way more complex. I don't buy it, but I see it.

If they can stop the damn thing from hallucinating caselaw and statutes, it might already be there.

Sure, but hasn't that always been the challenge? This feels like it boils down to "if they can fix the problems, it'll be great", which is true but applies to everything.

Well, they've gotten better and better over time. I've been using LLMs before they were cool, and we've probably seen between 1-2 OOM reduction in hallucination rates. The bigger they get, the lower the rate. It's not like humans are immune to mistakes, misremembering, or even plain making shit up.

In fact, some recent studies (on now outdated models like Claude 3.6) found zero hallucinations at all in tasks like medical transcription and summarization.

It's a solvable problem, be it through human oversight or the use of other parallel models to check results.

My point is not that the problems are unsolvable (jury's out on that), it's that "this will be good if we can fix the problems" isn't a very meaningful statement. Everything is good if you can fix the problems with it!

I expect that when people usually say that, they're implicitly stating strong belief that the problems are both solvable and being solved. Not that this necessarily means that such claims are true..