I had my first "AI at work" experience the other day when I sat through a luncheon meeting presented by a rep for one of the big legal research companies. It was billed as a continuing education event but was really just a sales pitch for their AI products. The guy was able to cite two uses for AI in the legal field:
Legal research
Pulling information from documents (in this case up to 5000 pages)
That's all well and dandy, but I don't do either of those things very often. This wasn't presented as "the technology is quickly changing and you'll be able to do more in the future" as much as "this is all you can do within the bounds of ethics and without exposing yourself to a malpractice suit". The idea that law firms will consist of a few partners handling a suspiciously large number of cases by prompting AI to generate outputs is pure fantasy. The people who think that AI will take over everything do so on the assumption that all work boils down to a set of deliverables that simply need to be generated, when that's not the case. If I'm looking to generate deliverables, I can already have a paralegal do all the drafting and research and just put my name on it, because there's nothing that says you need a law license to do legal research or draft documents.
What the client is paying for is for someone to take responsibility for the case, and it would be irresponsible of me to "handle" a case about which I knew nothing. Most of my time is spent reviewing and analyzing facts. Sure, an AI may theoretically be possible that can determine what's relevant and formulate a strategy better than I can, but the AI is not going to be responsible for its output. I'm never going to trust AI with tasks I wouldn't trust to support staff, no matter how much I trust my support staff (and they're great, btw), because the client doesn't want to hear about how it's the paralegal's fault. If I allow AI to do all my work for me, and I go into negotiations missing something, that's a pretty big matzo ball hanging out there. It's not that I'm perfect, or even necessarily better than AI theoretically could be, but the client is ultimately trusting me to make the relevant decisions, and I can't make them without a thorough knowledge of the case. It's the same problem with autonomous vehicles. I said a decade ago that they would never catch on, not because of any technical limitation, but because auto manufacturers aren't going to take responsibility for them. We've already seen this with Tesla being very aggresive in their defense of lawsuits stemming from autopilot. I don't necessarily disagree with Tesla's stance on this as things stand now, but if a vehicle is truly autonomous then an accident isn't caused by negligence on the part of the driver but on products liability on the part of the seller and manufacturer. As long as auto makers take the stance that the owners of vehicles are ultimately responsible for them, true AVs will never exist.
The other big issue is data security. You can tell me all day long about how great Claude, or ChatGTP, or Gemini are, but in the legal world using any of these is a complete nonstarter. Any lawsuit is going to deal with confidential data, and some suits are going to deal with little but confidential data. At the very minimum, we need to use settlement histories to evaluate potential settlement value of a case. Google literally built its business around data harvesting, and the tech sector as a whole doesn't have a stellar reputation for protecting client data. Regardless of whatever "opt out" provisions are allegedly in place, no law firm in their right mind would take the risk of feeding reams of data into a chatbot if there's any risk whatsoever that that information will show up later in a chatbot response. And no, this isn't the same as companies feeding their proprietary code bases into chatbots; the code's confidential status is subject to the discretion of management. An attorney does not have the discretion to reveal confidential information, especially if that information will be harmful to the client in the wrong hands.
This is before you even get to the fact that the current technology is underwhelming even for legal research. It looks good in demos but as soon as you try to use it for anything it proves its inadequacy. For document summarizing, 5,000 pages sounds generous, but it's rare that I'm concerned about finding information in a single document. I said this in a comment last week, but the utility would be more like "search all the depositions we have on file and pull all the ones where a witness testified about X". Well, we have tens of thousands of depositions on file, most in PDF but some in a special format used for court transcriptions. Conservatively assuming 100 pages per depo and 140 words per page, that comes out to something like half a billion tokens of context required, before we even consider that PDFs take more tokens than plain text, and a lot more if they haven't been OCR'd (which most of these haven't). Even the document functions described by the sales rep weren't that good; the example he gave was that if you were searching medical records for mentions of cancer it could broaden the search to include mentions of specific cancers.
p
I had my first "AI at work" experience the other day when I sat through a luncheon meeting presented by a rep for one of the big legal research companies. It was billed as a continuing education event but was really just a sales pitch for their AI products. The guy was able to cite two uses for AI in the legal field:
That's all well and dandy, but I don't do either of those things very often. This wasn't presented as "the technology is quickly changing and you'll be able to do more in the future" as much as "this is all you can do within the bounds of ethics and without exposing yourself to a malpractice suit". The idea that law firms will consist of a few partners handling a suspiciously large number of cases by prompting AI to generate outputs is pure fantasy. The people who think that AI will take over everything do so on the assumption that all work boils down to a set of deliverables that simply need to be generated, when that's not the case. If I'm looking to generate deliverables, I can already have a paralegal do all the drafting and research and just put my name on it, because there's nothing that says you need a law license to do legal research or draft documents.
What the client is paying for is for someone to take responsibility for the case, and it would be irresponsible of me to "handle" a case about which I knew nothing. Most of my time is spent reviewing and analyzing facts. Sure, an AI may theoretically be possible that can determine what's relevant and formulate a strategy better than I can, but the AI is not going to be responsible for its output. I'm never going to trust AI with tasks I wouldn't trust to support staff, no matter how much I trust my support staff (and they're great, btw), because the client doesn't want to hear about how it's the paralegal's fault. If I allow AI to do all my work for me, and I go into negotiations missing something, that's a pretty big matzo ball hanging out there. It's not that I'm perfect, or even necessarily better than AI theoretically could be, but the client is ultimately trusting me to make the relevant decisions, and I can't make them without a thorough knowledge of the case. It's the same problem with autonomous vehicles. I said a decade ago that they would never catch on, not because of any technical limitation, but because auto manufacturers aren't going to take responsibility for them. We've already seen this with Tesla being very aggresive in their defense of lawsuits stemming from autopilot. I don't necessarily disagree with Tesla's stance on this as things stand now, but if a vehicle is truly autonomous then an accident isn't caused by negligence on the part of the driver but on products liability on the part of the seller and manufacturer. As long as auto makers take the stance that the owners of vehicles are ultimately responsible for them, true AVs will never exist.
The other big issue is data security. You can tell me all day long about how great Claude, or ChatGTP, or Gemini are, but in the legal world using any of these is a complete nonstarter. Any lawsuit is going to deal with confidential data, and some suits are going to deal with little but confidential data. At the very minimum, we need to use settlement histories to evaluate potential settlement value of a case. Google literally built its business around data harvesting, and the tech sector as a whole doesn't have a stellar reputation for protecting client data. Regardless of whatever "opt out" provisions are allegedly in place, no law firm in their right mind would take the risk of feeding reams of data into a chatbot if there's any risk whatsoever that that information will show up later in a chatbot response. And no, this isn't the same as companies feeding their proprietary code bases into chatbots; the code's confidential status is subject to the discretion of management. An attorney does not have the discretion to reveal confidential information, especially if that information will be harmful to the client in the wrong hands.
This is before you even get to the fact that the current technology is underwhelming even for legal research. It looks good in demos but as soon as you try to use it for anything it proves its inadequacy. For document summarizing, 5,000 pages sounds generous, but it's rare that I'm concerned about finding information in a single document. I said this in a comment last week, but the utility would be more like "search all the depositions we have on file and pull all the ones where a witness testified about X". Well, we have tens of thousands of depositions on file, most in PDF but some in a special format used for court transcriptions. Conservatively assuming 100 pages per depo and 140 words per page, that comes out to something like half a billion tokens of context required, before we even consider that PDFs take more tokens than plain text, and a lot more if they haven't been OCR'd (which most of these haven't). Even the document functions described by the sales rep weren't that good; the example he gave was that if you were searching medical records for mentions of cancer it could broaden the search to include mentions of specific cancers.
More options
Context Copy link