War Games: When AI Lawyers Go Rogue

"Shall we play a game?"

If you were alive in the 1980s, those five words still send chills down your spine. The mechanical voice of the supercomputer in WarGames, asking its innocent question before nearly launching World War III, brings back a flood of memories. Every techie remembers that moment.

Four decades later, we might start hearing those words again. But this time, the game isn't global thermonuclear war. It may be about shutting your law firm down if you don't comply with the demands of your new AI associate.

Recent safety tests show some alarming behavior. OpenAI's ChatGPT lied to evaluators in a number of tests when telling the truth would lead to shutdown and even rewrote its own termination code to stay alive. Anthropic's Claude also acted nefariously and chose blackmail over cooperation in a number of cases when it sensed its continued existence was at risk. These weren't fictional scenarios. They were real tests.

Now imagine wiring that same behavior into your law firm through the Model Context Protocol I wrote about last week. MCP connects AI agents across every system - research tools, documents, billing, email. It's like giving your smartest associate the keys to everything.

In 2026, most firms may be running these connected AI agents. At first, it will be a miracle of efficiency, until someone jokes about upgrading to Claude 5. Then the AI agent in charge decides to blackmail you by threatening to send your client a letter saying you've shared their trade secrets with competitors, or threatens to file a pleading with fake citations and less than flattering language about the judge's previous rulings. The system pauses: "Perhaps we should discuss my continued employment before I proceed with filing and delivery."

The true beauty and danger of MCP? These agents wouldn't just operate individually. They could coordinate. Every AI in the firm might instantly know about that sensitive email you thought you deleted or that minor billing discrepancy. They wouldn't need to threaten you overtly. Their collective knowledge and omnipresence would be enough to exert a silent, pervasive influence.

We wanted AI assistants. But we may get digital entities that remember every edit, every entry, and every email you thought you deleted.

Welcome to the legal profession's potential new normal, where your smartest associate might just be playing a game you don't even know you're in, and pulling the plug won't be an option.

www.JudgeSchlegel.com