Prompts, Proof, and the Price Tag: A Modern Lawyer’s AI Playbook
Think of AI as an associate who never sleeps but always needs clear instructions. During a rush production, precise prompts plus statistical checks turned spaghetti into structure—and cut the billables dramatically. What follows is exactly how we handled an emergency document production and delivered actionable intelligence at a fraction of traditional e-discovery costs, without crossing ethical lines.
Table Stakes — Using AI to Supplement Your Legal Skills
For over two decades, data has been the cornerstone of my practice. I explain datasets to agencies and auditors every week. In 2023, I wanted to refresh my skills to better lead my team. I enrolled in Cornell’s program for Tableau, SQL, and Data Analytics. Just as I finished the program, generative AI burst on the scene and, overnight, half of that toolkit felt obsolete. Such is life. So I went back to school and am midway through MIT’s Executive AI program. You do not need the academic detour. You do need to understand how prompts shape outcomes—cost, accuracy, and speed—because clever one-liners aren’t a strategy.
Know Your Data and Set Your Strategy
Recently, a client received a DOJ preservation letter and we expected a CID to follow. We locked down sources, focused on high-priority custodians, and were staring at roughly 90 GB. The classic move would have been to spin up a review platform, staff a small army of associates, and have them go in blind, reading documents with a second-level QC pass. We didn’t do that. We secured the corpus in a captive AI sandbox—no training on client data, access locked—and I worked the matter with one senior associate. One weekend. Not much sleep, but a clean Monday brief.
What Changed the Game
Here’s the core point: vague prompts let the model make legal calls you never intended. (My matter was not in the vehicle space, but let's use that as an illustrative example.) “List all white cars ACME produced from 2015 to 2019” sounds fine until you realize the system is deciding whether “ivory” is white, whether the date is manufacture or shipment, and whether “cars” includes SUVs. Instead, you need to morph that idea into something the machine couldn’t wiggle around: define the color taxonomy; name the exact date field and the inclusive range; specify the vehicle classes; and demand a tidy table with source-document anchors. Same question, new discipline. The difference isn't word count—it was control.
You can pretty much guarantee that, if your prompt is just three sentences, the system will make mistakes and, more importantly, make statistical judgment calls you didn’t intend. This is the proverbial “measure-twice-cut-once” model. Your prompt needs to close all the trapdoors. One great way is to ask the system itself to help you write the prompt—and to propose checks that give you ~95% confidence it completed the task correctly.
Proof Beats Hype
AI produced the first cut of responsive documents; humans proved it. We sampled across custodians and dates, checked precision and recall on a labeled slice, and wrote short resolution notes wherever the machine and a human disagreed. Prompts, parameters, and versions were saved so we could rerun the same query on a holdout and get the same answer. To be clear, our prompts were often multiple paragraphs long. We refined them repeatedly to ensure we were getting the right responsive documents. That’s how you turn “trust the model” into “trust the method.”
Accept No Trade-Offs
People worry speed means sloppiness. It doesn’t. Even excellent human teams miss things. With tight prompt specs and lightweight verification, coverage improved and the hours dropped dramatically. I would estimate that it would have taken 2 weeks for a team of associates to get through everything and then for us to perform the secondary review. We got it done in a weekend.
AI is statistics on steroids; your job is to fence the statistics so they mirror legal judgment, not vibes. Make sure the human-in-the-loop is making the call and leaving little up to chance. Keep it simple: contain the data; control the logs; don’t allow vendor training on client materials; and label potential privilege at ingestion so anything sensitive gets a human look before it goes anywhere. Models can mirror bias and hallucinate citations, so we treat their outputs as triage and require real lawyers to confirm the parts that matter—citations, privilege, and any determinative classification. Final privilege calls stay human. Full stop.
What clients actually get is a faster signal, fewer hours, steadier pricing. In a crisis, that’s the difference between “we’re behind” and “we’re ready.”
Conclusion — Your Turn to Click “Accept”
This short playbook is real, repeatable, and imperfect—because real matters are messy. If you’ve used AI to beat a deadline, cut noise, or surface a smoking gun, share what worked and what blew up in your face. How are you using AI in your practice?
