Your AI chat history just passed through at least three companies you never agreed to trust. That is the real lesson of the Fireworks AI breach, and it should bother you more than the headline numbers.
Fireworks AI, a platform developers use to run open AI models, confirmed that an attacker breached its cloud provider and used that foothold to reach a Fireworks development system. From there, the intruder took customer billing information and some datasets. Fireworks claims tens of thousands of customers, including Harvey and Cursor. Only one affected customer has been confirmed so far: Notion.
Notion says the breach touched around 1% of its customers. Fireworks alerted it to an incident and later confirmed that Notion data was stolen. On November 3, Notion cut its connection to Fireworks and deleted the data stored in its systems. This week it confirmed the hackers took chat transcripts from Notion AI users of the "Insights" feature. Those transcripts include "any input to or output from the model that was written to the chat."
Think about what people type into an AI assistant inside a workspace tool. Meeting notes, draft contracts, salary questions, half-formed ideas. Notion says much of the stolen data is unattributable, meaning it can’t tell which customer a given transcript came from. That sounds reassuring. It isn’t, really. Anonymous text still leaks names, projects and company details, and the person reading it doesn’t need a user ID to figure out who wrote it.
Fireworks believes the attackers are financially motivated and wanted access to large language models, so they could sell tokens. Buyers get to burn through model usage without paying for their own compute. So the chat data may be a side effect of a hunt for free compute. But a side effect for the attacker is still a full exposure for the user. And once data is copied, nobody controls where it goes.
This fits a pattern. Most AI products are thin layers over a stack of third parties: a model host, a cloud provider, analytics, billing. Each link holds data, and each is a place for it to be stolen. A privacy policy from the brand on the front of the product tells you little about the security of the company three steps back. Customers rarely see that chain, and they can’t audit it.
Then there is the timing. This is Fireworks’ second breach in as many months, following an incident in October. Once is bad luck. Twice suggests the company’s security basics need a hard look, and customers should be asking pointed questions about what was fixed the first time.
If you use AI features in workplace tools, assume what you type could be stored by a vendor you’ve never heard of. Ask your provider which subprocessors see your prompts, how long they keep them, and whether retention can be turned off. Until vendors answer plainly, treat every chat box like a postcard.