The Tool Followed My Instructions. My Clerk Understood the Assignment.

I recently got access to a new legal GenAI tool built on top of real cases. This was not a general chatbot trying to sound like a lawyer from memory. It was connected to legal research, grounded in actual authority, and designed for the kind of work lawyers and judges actually do. So I loaded a new writ, gave the tool my tailored instructions based on two AI in Chambers guides, and asked it to prepare a neutral bench memo.

My bench memo instructions do not simply tell the AI to “analyze this case.” They give the tool a staged assignment. It must verify the authorities cited by the parties, conduct independent legal research, summarize the pleadings and record, and create a timeline. Only after those gates does it draft a neutral bench memo, and even then it is told not to recommend a result, not to cheerlead for either side, and not to pretend that judgment belongs to the machine. The whole point is to keep the tool in its lane.

The tool followed the instructions to a tee, which is what made the experience so impressive. The bench memo was organized, careful, neutral, and useful. It identified the issues, worked through the record, framed the arguments, and produced analysis that looked very much like the work of a serious research attorney. There were no wild guesses, no obvious hallucinations, and no sense that the machine was trying to take over the decision. It did exactly what I had been trying to design these tools to do.

Then my law clerk walked into my office and said, “Judge, I think this matter is moot.” She was right.

The lesson here is not that the AI failed. In some ways, the lesson matters more because the AI did exactly what it was told. It followed the instructions I gave it, but my clerk understood the real assignment.

It is tempting to say this is a prompt problem. I could easily add another stage to my instructions, maybe a preliminary step that checks standing, mootness and supervisory jurisdiction before reaching the merits. I could build that, and maybe I will. It may help in the next case where that is the issue.

But the deeper point would still remain. My clerk did not catch the mootness issue because I gave her a better prompt. She caught it because she is an experienced law clerk. She brought judgment to the file before anyone told her where to look. She did not need an instruction that said, “Now check whether this matter is moot.” She saw it because that is what experienced lawyers do.

Good law clerks are the people in chambers who understand that the assignment is bigger than the task in front of them. They know the case is not the same thing as the memo. They know the issue presented is not always the end of the inquiry. They know when to research, when to draft, when to test an argument, and when to walk into the judge’s office and say, “Judge, I think this matter is moot.”

The AI did not recognize the issue because mootness was not the question it was answering. It stayed inside the frame I gave it, which is what tools do and, in many ways, what we should want them to do. I do not want a legal AI system freelancing outside its instructions, inventing issues, or deciding on its own that it knows better than the judge. The discipline of the tool is that it moves through a controlled process as instructed, stays grounded in law, verifies authority, summarizes the record, and presents a neutral bench memo without trying to seize the judicial role.

That discipline is also the limit. A tool can follow sophisticated instructions. It can answer the questions presented. It can summarize the record, organize the law, verify citations, and draft useful memos. But an experienced law clerk can see the question behind the question without waiting for someone to write the next prompt.

The comparison between AI and a seasoned research attorney only goes so far. These tools are astonishingly useful when grounded in real legal databases and governed by a careful workflow. But a law clerk is not merely a research attorney, and chambers is not a document production shop. My clerk’s job is the same as mine. It is to think independently and get it right.

This experience did not make me less interested in AI. It made me more convinced that design matters, and just as convinced that design is not judgment. Even the best AI output still has to be met by human judgment, not just checked for hallucinations or skimmed for grammar. Someone still has to bring judgment to the work.

People miss this when they talk about AI replacing legal work. They picture a stack of tasks. Summarize this. Research that. Draft this. Apply that standard. If legal work were only a stack of tasks, then a tool that finishes them faster and cheaper would look like a replacement. But judicial work is not task completion. The work is to understand the case. That takes context, experience, humility, and the willingness to say that the stated assignment may not be the assignment at all.

The tool followed my instructions. My clerk understood the assignment.

www.JudgeSchlegel.com/bio