This weekly roundup thread is intended for all culture war posts. 'Culture war' is vaguely defined, but it basically means controversial issues that fall along set tribal lines. Arguments over culture war issues generate a lot of heat and little light, and few deeply entrenched people ever change their minds. This thread is for voicing opinions and analyzing the state of the discussion while trying to optimize for light over heat.
Optimistically, we think that engaging with people you disagree with is worth your time, and so is being nice! Pessimistically, there are many dynamics that can lead discussions on Culture War topics to become unproductive. There's a human tendency to divide along tribal lines, praising your ingroup and vilifying your outgroup - and if you think you find it easy to criticize your ingroup, then it may be that your outgroup is not who you think it is. Extremists with opposing positions can feed off each other, highlighting each other's worst points to justify their own angry rhetoric, which becomes in turn a new example of bad behavior for the other side to highlight.
We would like to avoid these negative dynamics. Accordingly, we ask that you do not use this thread for waging the Culture War. Examples of waging the Culture War:
-
Shaming.
-
Attempting to 'build consensus' or enforce ideological conformity.
-
Making sweeping generalizations to vilify a group you dislike.
-
Recruiting for a cause.
-
Posting links that could be summarized as 'Boo outgroup!' Basically, if your content is 'Can you believe what Those People did this week?' then you should either refrain from posting, or do some very patient work to contextualize and/or steel-man the relevant viewpoint.
In general, you should argue to understand, not to win. This thread is not territory to be claimed by one group or another; indeed, the aim is to have many different viewpoints represented here. Thus, we also ask that you follow some guidelines:
-
Speak plainly. Avoid sarcasm and mockery. When disagreeing with someone, state your objections explicitly.
-
Be as precise and charitable as you can. Don't paraphrase unflatteringly.
-
Don't imply that someone said something they did not say, even if you think it follows from what they said.
-
Write like everyone is reading and you want them to be included in the discussion.
On an ad hoc basis, the mods will try to compile a list of the best posts/comments from the previous week, posted in Quality Contribution threads and archived at /r/TheThread. You may nominate a comment for this list by clicking on 'report' at the bottom of the post and typing 'Actually a quality contribution' as the report reason.
Jump in the discussion.
No email address required.
Notes -
Yet More ChatGPT Stuff
Let me begin by saying up front that this message may read a bit oddly because I'm trying to keep some information out of it in hopes of keeping what remains of my fragile veneer of online anonymity.
Okay, background on me that is relevant here, and stabs my aforementioned fragile veneer of online anonymity in the back with a steak knife. I've just finished my 1L year at a top-50 law school in the United States. It was challenging, but not as challenging as a lot of law students like to say. Bitching and complaining are two of a law student's favorite things to do, but speaking as someone who spent a few years in the workforce before coming to law school I can say with no small degree of certainty I absolutely prefer law school to a 9-5.
But anyway. For anyone not familiar, law school, much like an undergraduate institution, runs on the semester system. My fall semester concluded in December of last year. Before finals season we all got an email from the Dean of Students promising fire and brimstone if we even dreamed about cheating on an exam. These warnings were in the usual fashion. No phones, no internet unless permitted (to some professors "open book open note" means your book, your notes, to others it means "Google? Sure why not."). The usual. Given the average educational level of the Motte I'm sure most of you received these emails or something almost identical twice a year for many years. My spring semester ended in April, and once again we were given the fire-and-brimstone email, this time with a twist. Among the most absolutely verbotten things that we must never-ever-ever do was access ChatGPT during an exam. Now I have to admit, this was something of a surprise. I suppose it shouldn't have been, but a surprise none the less.
Then one of my professors said something interesting. He couldn't give us permission to access ChatGPT, but in his opinion the ban was absolutely useless for his class. He said he'd played around with it, and was completely convinced it could not pass one of his exams. Certainly not. I actually quite like this professor, he's an engaging speaker, clearly passionate, appreciates intelligent disagreement, and is just a very kind person. I genuinely think he went and tested it, and decided it couldn't pass. I don't think he was being a blow-hard, that's just not the kind of person he is, at least in my judgment.
Now, the format of a law school exam, for those who are unfamiliar, generally follows the same basic model. You are given a fact pattern that varies in complexity from the fairly straightforward, to the reasonably realistic, to the completely outlandish. You are then asked to analyze it. Sometimes it's an extremely broad question like:
Which in the case of Torts is a notoriously painful proposition. Other times, you're given a slightly more narrow question like:
Which I realize seems very similar, but when you have four pages of dense fact pattern and only 90 minutes in which to finish this section before you need to be moving on to the next, those seemingly minimal boundaries are very helpful. Then sometimes professors will give you a very narrow question.
Generally speaking, what your professors are looking for is for you to "issue spot." They're not really interested in your legal analysis, though of course it needs to be at least credible, and they're almost never looking at your grammar or spelling. What they want is for you to show that you're capable of spotting what a lawyer should (in theory) be able to spot. What parts of these facts match up to the law we've studied in this class? Emphasis on the "in this class" part, I've heard horror stories about students evaluating potential civil liability in criminal law exams. Which I'm sure they did an excellent job of, but again. Ninety minutes before you need to be moving on to the next section or you won't finish in time, and you're not getting any points for talking about tortious trespass when you should be talking about whether or not you can charge common law murder or manslaughter.
I'm getting to the ChatGPT stuff I promise.
Anyway, this professor gave us the background facts in advance. Why? Because there were more than twenty pages of them. Agonizingly detailed, dense, and utterly fascinating if you enjoyed the class like I did. Not the questions mind you, just the fact pattern. But given the fact pattern, you can generally get a sense of what the questions will look like. After all if your professor spent several pages talking about someone shooting someone else, you're probably not going to be asked to analyze the facts for potential burglary charges. So I read the facts, figured out roughly what my professor was going to ask, and then...
Went in and took the exam like a good noodle without trying to use ChatGPT.
What? I'm training to be a lawyer. We're supposed to be risk-adverse.
But after the exam, well things are different. I still had the fact pattern, I remembered roughly what the questions were, and it was no longer a violation of the honor code to use ChatGPT. I checked. Thoroughly. So I spent some time copy and pasting every word of that 20 page document into ChatGPT, and then asked it something fairly analogous to the first question on the exam.
It spat out junk. Made-up citations, misquotes, misunderstanding of the black letter law, but, in that pile of garbage, were a few nuggets of something that looked fairly similar (if you squinted and turned your head ninety degrees to the left) to what I'd written on the exam. Now, I'm not going to toot my own horn here. I'm no budding legal genius. I will never be on the Supreme Court, I probably won't be a judge, I doubt I'll make it on to law review. But I am confident that I am somewhat above median. Not far above median, but law school grades on a very strict curve. Professors are given an allotment of grades. Something like "you can give at most 5 As, 10 A minuses, and 15 B pluses, anything below that at your discretion." So if the top 15 students on the final exam (which is 90-95% of your grade) were five 99s, and ten 98s, then the 99s all get As, and the 98s all get A minuses. The poor bastard who only got a 97 gets a B plus. It is hard to achieve a high GPA in law school. Conversely, it is very hard to do worse than a B minus (predatory law schools excluded). Anyway the point is that I know that according to my (above the median) first semester GPA, I am above the median. Not brilliant, but top half of the class.
So I started poking at it. I fed it the actual citations it was trying to make up based on my class notes and outline (read: study guide - no idea why but in law school study guides are called outlines), informed it of previous court rulings and the actual holdings that were relevant to the analysis, and then asked it the same question again.
Suddenly it was spitting out a good answer. Not a great answer, it was still way too short on analysis, but it correctly identified sticking points of law, jurisdictional issues, and even (correctly) raised a statute I hadn't fed it, which was a surprise. It must have been part of it's training material. But the answer was still way too short. So I hit it with a stick and told it to try again and make it longer. Then I did that again, and again, and again. The hitting with a stick was really just telling it "write this again but longer, add more analysis, focus on the section about [whatever wasn't fleshed out enough]." Almost no effort on my part at all. Eventually I ended up with about five hundred words of actually pretty decent issue spotting and analysis.
Now, do I think that this was good enough to get an A? I doubt it. Good enough to get a median grade? A nice solid middle of the pack B? Yes. It could, I think, get a B, which is most definitely a passing grade.
The obvious caveats to all of this are manifold. I'm not actually a lawyer yet, so I have no idea how good my understanding of what good legal analysis looks like is. I also don't know how I did on my exam yet, so it's entirely possible I completely misunderstood an entire semester-long course that I sincerely enjoyed and am about to get the only C (you have to try to get lower than a C) in the whole section. I don't think that's likely (see supra "I am somewhat above median") but it is absolutely possible. This only keeps me up at night a little bit.
The further caveat is that law school is nothing like the practice of law, something that has been repeated ad nauseam by every lawyer I have ever met. So this is not me saying that ChatGPT is capable of performing as a lawyer. But it has taken the first step. Law school is supposed to teach you how to think like a lawyer, at least in theory. There's another theory that it's three years of extremely expensive hazing born out of nothing more than tradition, but let's assume for the moment that it actually does teach you how to think like a lawyer. ChatGPT is capable of, in a minimal sense, and with some poking and prodding, thinking like a lawyer.
Edit: apologies, I wrote this very early in the morning and forgot to include that I was using the free 3.5v, not 4.
PART 1/2
Chat GPT-4 gets 88th percentile on the LSAT vs 40th for Chat GPT-3
If you are using the free version it's the equivalent of a D student, pay $20 to get an A student.
Your professor used to be right when he said the cutting edge AI model wasn't that good for law, but AI moves so fast he is now wrong
OP is talking about a law school exam, not the LSAT, which is very, very different.
They are both used in the real world to evaluate legal ability (or potential) so such a drastic gain in ability between versions is worth knowing. I think it's almost certain he'd see improvement in his task using the latest version
More options
Context Copy link
More options
Context Copy link
More options
Context Copy link
More options
Context Copy link