8 min read

“Zero Data Retention” Wasn’t Zero, Quad9 Leaves France, Tor Nodes Hijacked & Proton’s AI Controversy

Notion's AI chat transcripts were exposed through its AI provider, Fireworks AI, and "zero data retention" turned out to underperform. I also cover Quad9 pulling out of France over site-blocking orders, 333 hijacked Tor exit nodes targeting SSH and email, and the Proton AI controversy.

“Zero Data Retention” Wasn’t Zero, Quad9 Leaves France, Tor Nodes Hijacked & Proton’s AI Controversy
📰
Listen to the weekly podcast on Apple Podcasts, Spotify, other podcast apps, or RSS. You can also watch Surveillance Report on YouTube or Techlore.TV

Notion's AI chats leaked, proving "zero data retention" wasn't zero

Fireworks AI is one of many providers who hosts models and hands out an API key to use them. On October 23rd and 24th, attackers breached a cloud provider hosting Fireworks' R&D machines and reached development systems. They got customer billing info, resource metadata, and partial model weights and data sets.

The link to Notion happens in the Notion AI Insights feature. Transcripts from about 1% of workspaces using it were exposed, so what users typed and what the AI replied was potentially exposed. And Notion’s write-up is interesting...

  • “No Notion AI customer data was involved” (about the 1st dataset)
  • A few paragraphs later: it actually included real Notion AI chat transcripts
  • Also Notion: the chats are “unattributable”
  • Later: “we revoked exposed Notion API keys"

Besides the clear confusion, another problem is how Notion (and others) describe zero data retention (ZDR). Their FAQ says ZDR covers live requests sent to AI vendors, but not features that need stored data, like chat history and insights. In other words: an enterprise customer who thought they had ZDR could still have had chats retained despite ZDR! Notion's own investigation also found Fireworks kept data they weren't supposed to keep anyway, so it's a wishy washy situation when it comes to both company's commitments to 'zero' data retention.

My takeaways:

  1. Assume a chat with Notion AI or any other outsourced chatbot isn't just with the company. You're trusting every company in that chain.
  2. ZDR is better than nothing, but in many cases it's only a loose promise to not retain data.
  3. Avoid pasting secrets into chats! Be careful what you give an LLM and/or use a local model to sanitize prompts first.
  4. Budgeting apps, health apps, and more now ship AI chatbots, and many forward your queries to a provider like ChatGPT. Read the privacy policy before assuming the service hosts the model itself under their normal privacy practices.

To me, this is a big hit to trust for the inference provider industry. Many people pay more to use these platforms because of their privacy guarantees, and this story shakes them. I know I'll be re-assessing how much weight I put on ZDR and our tools will get a refresh shortly in light of this situation. I'm currently investigating services that try to add more hardware guarantees to their ZDR, more on this in a future writeup.


Bits & Bytes 🤖

~ Story 1: France pushes Quad9 out
A French court ordered Quad9, a privacy-respecting DNS provider, to block sites accused of copyright infringement. Quad9 says it can’t block only French users reliably, and blocking globally would extend French law worldwide. With about 20 similar lawsuits pending, it chose to withdraw from France instead.

My take: This is a huge attack on the open internet!! I encourage everyone to follow this saga, support Quad9 if you can, and if you're French: contact your representatives about it. The implication of controlling the internet through DNS makes little sense technically (anyone can host their own DNS from home) and it still has negative repercussions for average users. I did a wonderful interview with John Todd from Quad9 not too long ago and we spoke in great detail about what's happening in France and why to pay attention.

~ Story 2: 333 Tor exit nodes hijacked
Unknown attackers took over 333 Tor exit nodes. They redirected SSH connections on port 22 to interception servers and tried to favor unencrypted email traffic. Anyone who clicked past the host-identification warning could have had their password stolen and the server infected with the Ebury backdoor.

My take: The defaults held! HTTPS, onion services, people who didn't blindly ignore the SSH warning, etc. So while I don't want to downplay the spookiness of this attack, I do want to set realistic expectations about what needed to go wrong for it to be pulled off. Valuable Tor lessons here that remind us about encryption and what the final Tor node can see.

~ Story 3: Proton’s AI push
Proton’s CTO published an AI manifesto, and the CEO reportedly called a request for pen testing before pushing code “loser thinking.” Plans to re-interview staff about their AI use were then canceled after backlash, and the internet has not been happy since!

My take: This is a rare situation where I encourage you to listen/watch the full Surveillance Report for my full take since I can't easily fit it all here. (timestamp is at 12:54) My TLDR: AI amplifies a company’s priorities rather than changing them, and a “move faster” culture is how future security problems start. So this is a concerning event and I'm watching carefully, especially since a huge portion of my life is in Proton and I want to feel like it's a safe place to be. If you have any issues with Proton in light of this, our SPA Tools have countless options across the ecosystem.


This Week Month on Techlore 📺

Whoa....Okay yeah so it's been a month since the last Surveillance Report. I want to start by thanking all of you for your patience. Not just for SR, but for all our other content that's fallen behind. Between Go Incognito, travel, and other life events between the two of us we struggled to stay above water. Our weekly YouTube livestream resumed for the last two weeks, now SR is back on, and Go Incognito section 2 is live with section 3 set to go live on Monday:

Go Incognito Section 2 is live
Section 2 is live! Jump in, say hi, and leave your questions as you go.

Besides that, I covered the new F-droid 2.0 update:

F-Droid 2.0 Is Finally Here! (First Look)
F-Droid 2.0 is the biggest update to the FOSS Android app store in a decade. I walk through the redesign, the new search + filters, install prompts that finally come *before* the download (thanks, EU DMA!), automatic updates, Tor VPN, signer info, adding third-party repos, and more! Watch

And I interviewed Carl Richell from System76 to discuss what they're building, as well as the story behind their involvement in getting Linux exempted from age verification laws. It's a really insightful interview:

Custom Linux Computers, Age Verification, and AI Privacy (System76 CEO Interview)
Techlore Talks brings you in-depth conversations with the experts at the forefront of digital rights, privacy and security.

It feels good to be getting back in the swing of things, so I'm excited to say I'll be back for next week's Surveillance Report 🫡


Action Item ✅

Straightforward this week: If you use AI anywhere in your life, audit it! Check what the risks are, which companies are involved, and challenge your understanding of what the risks are. And if you haven't already: consider a local LLM for sanitizing secrets.

Until next week 🫡

This Week's Sources

Highlight: Notion's AI Chats Leaked Through Its AI Provider

Story 1: A French Court Just Pushed Quad9 Out Of France

Story 2: Hundreds Of Tor Exit Nodes Hijacked To Intercept SSH And Email

Story 3: Inside Proton's AI Push

Story 4: The Secret Court Ruling On How The US Spies On Americans

The Defense Bulletin

Data Breaches

Threats

FOSS+ Updates

Get Ahead of the Snoops with Surveillance Report

Surveillance Report: what matters in privacy, security, and digital rights. Weekly, free.