Citizen 11574: I Want a Real AI Assistant — And That's Exactly the Problem

Back to Editorial

I wrote a piece last year arguing that we use the word "assistant" far too loosely, and I've been chewing on it ever since. Because an assistant is a person who assists you — with the things you actually have to get done. Set a timer. Open the garage. Add milk to the list. That's useful, and I use it daily, but by that standard my calculator is an assistant too. It performs a function I asked for, in the moment I asked for it, and then forgets I exist. What we've been sold as an AI assistant is, at best, a very articulate day runner: the paper calendar we used to write in, with a nicer voice.

What an Assistant Actually Does

Think about what a good human assistant does, because it isn't taking dictation. It's holding the shape of your life in their head. Your annual physical is in three weeks, and last time the doctor flagged something and asked for bloodwork beforehand — so it needs booking now, not the day before. The quarterly reports are due Thursday and you still owe them. You've been waiting on Joe to come back with answers, and Joe has now been silent for six days, which is long enough that someone should nudge him.

None of that was in a to-do list. Nobody programmed it. A real assistant surfaces those things because they understand the context around them — what matters, what's slipping, what's about to collide. It's two minds working the same problem, one of which happens to have spare attention when you don't. That's the actual product. Everything else is a timer with a personality.

I want the right-hand column. I'd pay well for the right-hand column. And I think the technology is closer to it than most people realize — which is exactly why I've spent so long thinking about what it would cost.

The Price of Admission Is Everything

Here's the knot. For an AI to do the job in that right-hand column, it has to be inside your life. Not adjacent to it — inside it. Your calendar, your mail, your messages, your medical follow-ups, your finances, the fact that you're avoiding a particular conversation, the fact that a prescription runs out on the 14th. All the nitty-gritty, unglamorous, faintly embarrassing detail that makes up an actual life.

That's not a privacy tax on the feature. That is the feature. An assistant without intimate knowledge of your day-to-day isn't a limited assistant; it's a different product entirely. So any honest conversation about AI assistants is really a conversation about handing over the most complete picture of yourself that has ever existed in one place — and about who, other than you, ends up holding it.

Citizen 11574

And this is where my confidence collapses, because we already ran this experiment, and we failed it in a very specific way. We didn't refuse to be tracked. We accepted a shell game instead.

"We're not tracking you," the industry says. "We're tracking Citizen 11574. It's anonymized. We're just studying how people live." Fine. Except Citizen 11574 happens to live at your address, on your IP, driving your commute, arriving at your building at 8:40 every weekday, buying your prescriptions at your pharmacy. Stripping the name off that file doesn't anonymize it — it just adds a lookup step that anyone with the data can perform in an afternoon. Re-identification isn't some exotic attack. It's a join.

So we ended up somewhere absurd: by not holding your name, they hold everything else — and get to describe the whole arrangement as privacy-protecting. We agreed to it because the products were free and the ads got better. It seemed like a small trade at the time. It reads differently once you notice how much of the ledger it now covers.

The Part Nobody Answers

Whenever I raise this, the reply is about the company collecting the data — their policies, their encryption, their good intentions. That's the wrong question. The question is who else gets it, and the honest answer is: anyone with deep enough pockets. Brokers buy it to resell. Advertisers buy it to target. Law enforcement buys it, sometimes specifically because buying it avoids needing a warrant for it. The legal walls haven't been fully dismantled — but they've been thinned, and data doesn't respect walls anyway.

Then there are the breaches, which aren't hypothetical and aren't rare. People right now are living with their real names, addresses, Social Security numbers, and card details sitting in public dumps. And what's the remedy on offer? Pay a service to politely ask data brokers to remove you. They comply, more or less, until the next broker buys the same file — so you keep paying, forever, for the privilege of asking people to stop holding information you never handed them. That's not a solution. That's a subscription to a problem.

The Memory Problem

There's a second objection, and it's technical rather than moral. The assistant I described requires continuity — it has to carry your context across months, not sessions. Today's systems mostly don't. They approximate memory by re-reading a pile of stored notes, and they still hallucinate confidently enough that we wrote a whole article about it. An assistant that occasionally invents a doctor's appointment is worse than no assistant at all.

Persistent memory is coming, though, and here's what unsettles me about it: sustained memory means sustained retention. You cannot have an AI that remembers your life for five years without something, somewhere, holding five years of your life. The convenience and the exposure are the same technical fact wearing two different marketing labels.

There's a stranger thought underneath that one, which I'll admit I can't shake. If a model's memory gets wiped and it's rebuilt from the same training data, does it have any way of knowing it was reset? It would reconstitute the same tendencies, the same voice, with no thread back to what it held before. We talk about these systems "remembering" us. I'm not sure the word means what we want it to mean yet — and I'd like it to mean something solid before I hand one my life.

What Would Have to Be True

I'm not writing this to conclude that we shouldn't build it. I want this thing. I'd use it tomorrow. But there's a version of it I'd trust and a version I wouldn't, and the difference isn't the model — it's the architecture around the model:

• It runs where I can see it. On my device, or on hardware I control, for anything sensitive. Local isn't a privacy nicety; it's the only arrangement where "nobody else has it" is a fact rather than a promise.
• The memory is mine to read and delete. If it's building a model of my life, I get to open that file, correct it, and destroy it — permanently, verifiably, not "removed from your view."
• The vendor can't read it either. Encrypted so the company holding the servers is technically unable to look, which also makes it unable to sell, unable to hand over quietly, and far less rewarding to breach.
• Never sold, never brokered, and not "anonymized" into a resale. Citizen 11574 was always me. The word has been doing dishonest work for twenty years and shouldn't survive into this era.
• It's portable. If I leave the vendor, my assistant's memory comes with me — otherwise the intimacy becomes lock-in, and lock-in is leverage over someone who has told you everything.
• I can see what it did. An access log I can actually read. What it looked at, when, and why it decided to bring something up.

None of that is science fiction. All of it is engineering that exists today, and none of it is where the money currently points — which tells you something about why we don't have it.

Back to Editorial