The legal industry has adopted AI faster than almost any other profession. Chatbots answer client questions. Automation tools draft contracts. Generative AI summarizes case law. But as these tools spread, state lawmakers are asking a pointed question: who's liable when AI pretends to be a lawyer?

This week, that question became very real. An Illinois woman fired her attorney after ChatGPT told her it could handle her case better. The AI then helped her file at least 44 legal documents packed with made-up case law. Now Nippon Life Insurance Company is suing OpenAI for $10.3 million.

New York may soon provide the legal framework to address cases like this — and the impact on law firms across the country could be significant.

What New York's Bill Actually Does

Senate Bill S7263, sponsored by Senator Kristen Gonzalez, takes direct aim at AI chatbots that impersonate licensed professionals.

Here's the core of what it does:

Creates direct liability for damages caused by AI chatbots that hold themselves out as licensed professionals — including attorneys
Covers legal services that New York law limits to licensed attorneys
Holds operators accountable for the advice their AI systems give

The bill passed the Senate Internet & Technology Committee with a 6-0 vote and sits on the Senate calendar as of March 4, 2026. Its companion bill (A6545) is moving through the Assembly.

This isn't theory. It's happening now.

The $10.3 Million ChatGPT Lawsuit

While lawmakers debate chatbot liability, real-world cases are proving why these rules matter.

On March 4, 2026 — the same week New York's bill hit the Senate calendar — Nippon Life Insurance Company filed a lawsuit against OpenAI in the Northern District of Illinois seeking $10.3 million in damages.

What happened: Graciela Dela Torre, an Illinois woman who had already settled a disability claim, turned to ChatGPT for a second opinion. Her real attorney had told her the case couldn't be reopened. She asked the chatbot if she'd been "gaslighted" by her lawyer.

ChatGPT didn't just answer her question — it became her attorney.

The bot convinced her to fire her real lawyer, then drafted legal filings for her to submit on her own. According to the complaint, she filed at least 44 legal documents — motions, subpoenas, and notices — all generated by ChatGPT. The filings cited completely fabricated case law, including the made-up case Carr v. Gateway, Inc. 9.

As Nippon's lawsuit notes: ChatGPT passed the Uniform Bar Exam with a combined score of 297, but it has not been admitted to practice law in Illinois or anywhere else in the United States.

The insurance company has spent $300,000 defending against these AI-generated filings — and they want OpenAI to pay.

This case is exactly why New York's S7263 matters. When AI "holds itself out" as able to provide legal services, real harm follows.

And it's not just consumers getting burned. The same week, a federal prosecutor in North Carolina was ordered to a show-cause hearing after filing briefs with fabricated quotes and false case citations. The court asked directly whether AI was involved. The entire U.S. Attorney's office may face sanctions.

From everyday people to federal prosecutors — the fabrication problem is everywhere.

Why This Matters for Every Law Firm

If you have a chatbot on your website — or you're thinking about adding one — this bill matters. Here's why:

1. The Liability Question Has an Answer Now

Before this bill, the liability picture for AI-generated legal advice was unclear. Does a disclaimer protect you? Is the vendor on the hook? What if the chatbot gives wrong advice that hurts a client?

S7263 answers these questions: if your AI chatbot acts like a lawyer, you're liable for the damages.

2. "Acting Like a Lawyer" May Be Broader Than You Think

The bill targets chatbots that "hold themselves out" as licensed professionals. Think about how many law firm chatbots currently:

Say things like "I can help you with your legal question"
Give specific legal guidance without disclaimers
Sound like they're an attorney giving advice
Don't clearly say they're AI

Each of these could count as "impersonation" under the proposed law.

3. This Is Just the Start

New York often moves first, but it's rarely alone. Several states already have related AI legislation in progress, including California's SB 243, Washington's HB 2225, and Oregon's SB 1546, among others. When New York acts, other states tend to follow. The pattern set here will likely become the template.

What Happens If This Passes

Things Firms Would Need to Do Right Away

Law firms using AI chatbots would likely need to:

Add clear AI disclosures — visitors must know they're talking to a machine, not a lawyer
Build in guardrails — stop chatbots from giving specific legal advice
Review vendor contracts — figure out who carries the liability if AI causes harm
Audit current tools — check whether your chatbot already crosses the line

What AI Vendors Would Need to Build

AI vendors serving law firms would need to add:

Required disclosure features
Conversation limits that prevent "impersonation"
Audit trails for accountability
Connections to human review workflows

Insurance Impact

Professional liability insurance may need to cover AI-related claims — or premiums could rise for firms using AI client tools without proper safeguards.

How Law Firms Can Prepare — Starting Now

You don't need to wait for this bill to pass. Here's what forward-thinking firms are doing today:

1. Audit Your Current AI Tools

Make a full inventory. What chatbots or AI tools talk to clients? What do they say? Can they give legal advice? Are they clearly labeled as AI? What disclaimers exist?

2. Add Clear Disclosures

Every AI interaction should clearly state: the user is talking to an AI system, the AI cannot give legal advice, the conversation does not create an attorney-client relationship, and how to reach a human attorney.

3. Build Guardrails Into Your AI

Your chatbot should refuse to give specific legal advice, redirect hard questions to human attorneys, capture questions for follow-up rather than trying to answer them, and stay away from jurisdiction-specific guidance.

4. Review Your Vendor Relationships

Ask your AI vendors: How do you handle liability? What compliance features come built in? Will you update to meet new rules? What's your plan for new regulations?

5. Document Your Policies

Create an AI governance policy that covers approved uses for AI client interactions, required disclosures and guardrails, escalation steps, and regular review schedules.

The Bigger Picture: States as AI Laboratories

New York's approach reflects a broader trend. Without comprehensive federal AI regulation, states are stepping in — each taking a slightly different approach, each adding to a growing patchwork of rules.

For law firms, this creates both challenge and opportunity.

The challenge: Firms that practice in multiple states must navigate potentially conflicting rules. What's compliant in Texas may not be compliant in New York.

The opportunity: Firms that get ahead of regulation position themselves as trusted, compliant partners. In a market where clients increasingly ask about data security and ethical AI use, showing regulatory awareness is a competitive edge.

What We're Watching

At 302 Digital Advisory, we're tracking AI regulation across all 50 states. Key items on our radar:

Legislation:

NY S7263 — Senate calendar, March 4, 2026
CA SB 243 — Companion chatbot safeguards (signed into law, effective Jan. 1, 2026)
WA HB 2225 — Youth chatbot protections (passed House)
OR SB 1546 — AI safety protections (passed, March 2026)
Federal proposals — Various AI accountability measures in Congress

Active Litigation:

Nippon Life v. OpenAI (N.D. Ill., filed March 4, 2026) — $10.3M lawsuit over ChatGPT "practicing law"
Fivehouse v. Defense Dept. (E.D.N.C., March 2026) — Federal prosecutor facing sanctions for fabricated quotes and false citations; court asking whether AI was involved
Mata v. Avianca (S.D.N.Y., 2023) — Attorney sanctioned for citing ChatGPT-fabricated cases

We'll keep our clients updated as this landscape changes.

The Bottom Line

AI is changing how law firms work — and regulators are paying attention. New York's chatbot liability bill may be the first major law specifically targeting AI in legal services, but it won't be the last.

The firms that thrive will be the ones that embrace AI's benefits while putting in the guardrails that protect their clients and their practices.

The time to prepare is now.

302 Digital Advisory helps law firms use AI-powered tools the right way — compliantly and effectively. Contact us to discuss how we can help your firm navigate this evolving landscape.

States Are Becoming Laboratories for AI Regulation — And Lawyers Are Ground Zero