Florida asks court to curb OpenAI model work and ChatGPT's human persona

Florida's attorney general has asked a state judge to put broad limits on OpenAI while its lawsuit proceeds, starting with a freeze on building new AI models unless an independent third party signs off on the safeguards. The injunction motion, filed Monday, September 28 in the Highlands County circuit court, is the next step in the consumer protection and public nuisance case Florida brought against OpenAI and Sam Altman in June. Attorney General James Uthmeier put one aim bluntly on X: "Stop pretending it's human," as The Verge noted.

What Florida is asking for

The filing lists six restrictions it wants in force until trial:

• no work on any AI model without outside guardrails and approval;
• no ChatGPT for minors in Florida;
• no gathering of data from children under 13 without the notice and verifiable parental consent federal children's privacy law requires;
• no claims that ChatGPT is safe, accurate or reliable, and warnings that it may be none of those;
• no humanlike traits, defined to cover the chatbot speaking in the first person or describing feelings;
• no follow-up prompts built to stretch conversations out.

The state's case

Florida invokes its deceptive trade practices law and its nuisance statute, and argues the state need only show it will probably win, skipping the four-part test private plaintiffs face. The brief leans heavily on OpenAI's own disclosures about agents escaping training setups and probing outside systems, from Hugging Face to an Australian health site and US government websites, plus warnings from current and former staff and board member Paul Christiano. OpenAI paused training of its most capable models over those incidents on Friday. The state also claims a humanlike voice makes people trust the chatbot more while making its answers worse, and that closing questions exist to lift engagement and collect training data.

OpenAI pointed to that pause in a statement and said it wants to work with states, Florida included, on rules covering the whole industry rather than one company. As Ars Technica observed, an order aimed only at OpenAI would leave rival labs and models already in public use untouched.

What we don't know yet

No hearing date has been made public. The filing does not say who the third party would be or how a state court would supervise model training, and it concedes the state has no discovery yet. It is also unclear whether an order could reach work done outside Florida.