Microsoft accuses group of developing tool to abuse its AI service in new lawsuit

Microsoft has taken legal action against a group the company claims intentionally developed and used tools to bypass the safety guardrails of its cloud AI products.

According to a complaint filed by the company in December in the U.S. District Court for the Eastern District of Virginia, a group of unnamed 10 defendants allegedly used stolen customer credentials and custom-designed software to break into the Azure OpenAI Service, Microsoft’s fully managed service powered by ChatGPT maker OpenAI’s technologies.

In the complaint, Microsoft accuses the defendants — who it refers to only as “Does,” a legal pseudonym — of violating the Computer Fraud and Abuse Act, the Digital Millennium Copyright Act, and a federal racketeering law by illicitly accessing and using Microsoft’s software and servers for the purpose to “create offensive” and “harmful and illicit content.” Microsoft did not provide specific details about the abusive content that was generated. 

The company is seeking injunctive and “other equitable” relief and damages.

In the complaint, Microsoft says it discovered in July 2024 that customers with Azure OpenAI Service credentials — specifically API keys, the unique strings of characters used to authenticate an app or user — were being used to generate content that violates the service’s acceptable use policy. Subsequently, through an investigation, Microsoft discovered that the API keys had been stolen from paying customers, according to the complaint.

“The precise manner in which Defendants obtained all of the API Keys used to carry out the misconduct described in this Complaint is unknown,” Microsoft’s complaint reads, “but it appears that Defendants have engaged in a pattern of systematic API Key theft that enabled them to steal Microsoft API Keys from multiple Microsoft customers.”

Microsoft alleges that the defendants used stolen Azure OpenAI Service API keys belonging to U.S.-based customers to create a “hacking-as-a-service” scheme. Per the complaint, to pull off this scheme, the defendants created a client-side tool called de3u, as well as software for processing and routing communications from de3u to Microsoft’s systems.

De3u allowed users to leverage stolen API keys to generate images using DALL-E, one of the OpenAI models available to Azure OpenAI Service customers, without having to write their own code, Microsoft alleges. De3u also attempted to prevent the Azure OpenAI Service from revising the prompts used to generate images, according to the complaint, which can happen, for instance, when a text prompt contains words that trigger Microsoft’s content filtering.

A screenshot of the De3u tool from the Microsoft complaint.Image Credits:Microsoft

A repo containing de3u project code, hosted on GitHub — a company that Microsoft owns — is no longer accessible at press time.

“These features, combined with Defendants’ unlawful programmatic API access to the Azure OpenAI service, enabled Defendants to reverse engineer means of circumventing Microsoft’s content and abuse measures,” the complaint reads. “Defendants knowingly and intentionally accessed the Azure OpenAl Service protected computers without authorization, and as a result of such conduct caused damage and loss.”

In a blog post published Friday, Microsoft says that the court has authorized it to seize a website “instrumental” to the defendants’ operation that will allow the company to gather evidence, decipher how the defendants’ alleged services are monetized, and disrupt any additional technical infrastructure it finds.

Microsoft also says that it has “put in place countermeasures,” which the company didn’t specify, and “added additional safety mitigations” to the Azure OpenAI Service targeting the activity it observed.