A Review of Claude:Twenty Mistakes, and a 5.0

I came at this sideways. Someone at work told me Claude could turn a stack of PDFs into a searchable knowledge base. I said it couldn't. I was right — it couldn't, not the way it was put to me — but finding out how wrong I was about everything around that took the next eight months and about seven hundred dollars.

This is the sixth and final review in a set I recorded over one weekend, and the only one where I have to tell you upfront that the thing being reviewed helped write it.

Why This Site Exists at All

Back around 2022 I was irritated. Phone makers were selling voice assistants as artificial intelligence, and they plainly weren't. Glorified message kickers. A voice memo with a nice personality. I wanted to write something like a letter from a consumer to the industry — this is what you're calling an assistant, and here is what an assistant would actually be.

Then I found a video claiming you could produce entire pieces of content with an AI writing tool, and I signed up for one. The reason I signed up was not curiosity. It was this:

If these things were going to take all the jobs, then one of the jobs they were coming for was mine. So I set myself a test, on the principle that has run this whole project: I can't criticise it if I haven't done it myself.

I built the first version of this site on a template in Pinegrow, with free stock images I cropped myself. The first article was mine — I wrote the beats, the tool filled them in. Then, to keep the project moving, I did something deliberate and slightly cynical: I asked it for a review of a service, five pros and five cons, and I published what came back without reading it. If it's so good, I thought, this doesn't need human intervention. Just put it up.

That was the experiment. Here's the result, and it's the reason every review on this site is being rewritten: can AI build a website? Yes. Can it build a website that makes money? In my experience, no. It generated the articles. They didn't drive traffic, and they didn't qualify for advertising. That's one case — mine — and I'd like to see other people publish theirs.

So this review sits at the far end of that arc. Same person, same site, opposite method: I do the reporting and the judging, and the assistant does the part I'm bad at.

$20 a Month — Which Is Half What I Paid

Twenty dollars a month is the advertised price. Here is what actually left my account between 27 February and 3 July 2026:

Exactly half of what I spent was not the subscription. It was five and ten dollars at a time, bought late at night, to finish something that was already in flight.

March was the worst of it — nine top-ups, $45, on top of the $20. That month cost $65, which is three and a quarter times the sticker price. And five of those purchases happened inside a single day: Sunday 29 March 2026, which I spent searching every file on my server for code I was afraid I had spread.

What twenty-five dollars in one weekend actually bought

The Friday before, we were chasing an unrelated problem with environment variables — which come up again later in this review, and have cost me more evenings than anything else on this site — when Claude noticed something in the page header that had no business being there, and said plainly that what it looked like it was doing could be bad.

The behaviour is what got me. The code was being injected into whatever page I opened, before I had even looked at it. Not a script on one site. Every site.

So I made the worst assumption available: my server was infected, and it had reached my work machine from there. If that were true then it was mine, and I would rather find the fix myself than let anyone else deal with the consequences of it. I emailed the IT team and the developers to get them ready in case it was real.

Then I went through the server file by file, with my host's technicians checking from their side and the security firm we pay running scans across the whole thing. That is what the top-ups bought — capacity to keep going, five times, on a Sunday.

Nothing. Not on the server, not on my Mac. It only ever appeared on the one machine. That machine was eventually wiped, and what surfaced was a leftover from an antivirus product installed years earlier on an old Windows build. Nobody — not our IT, not the host, not the firm we pay to catch exactly this — could tell me for certain what it was. I reset every password I own anyway.

Notice which way this one runs, though. Everywhere else in this review the failure mode is a confident claim about something it hadn't checked. Here it flagged something genuinely alarming that three sets of professionals then couldn't explain — and I'd spend that weekend again.

And one month the subscription didn't get paid

The renewal due on 27 April 2026 appears in my invoice list as $20, Refunded. It wasn't refunded — the payment failed. The charge went through on 3 May instead, and the billing date has run on the 3rd ever since.

It's in here because every review of every subscription is written as though twenty dollars is a rounding error, and for plenty of people it isn't. That's what this product costs when it meets a real budget in a bad month.

It's also a small factual note worth logging: a billing record describing a failed payment as a refund is the fourth history screen in this project that doesn't match what happened — alongside two tools hiding the year on old documents, and one reporting days longer than a day.

What work pays, and what that changes

A $100-a-month account started at the end of March and has run since — five months, $500. When that began, I stopped using my own account for work entirely. Both accounts are the same product at different tiers, so the $700 total is all one company.

On the larger plan I've hit the usage limit twice in four months, and both times I was close enough to the reset that I just waited, or went to lunch. That's an unexciting answer, which is exactly why it's worth printing.

The Night a Single Backtick Cost Me an Evening

I run Node.js applications on my own server. The environment variables — keys, passwords, connection details — go into the hosting control panel rather than sitting in a file on disk, which is the more careful arrangement of the two. This was the third or fourth application I'd set up that way.

Claude was confident, as it always is. Not a problem. You've done this before, we can do it again.

I generate long passwords on principle. The guidance said sixty characters minimum, so I went past it — password manager, all the bells and whistles. Pasted it in. It wouldn't connect.

Debugging turned up something strange: the application was failing before any variable loaded at all. So we searched outward. Server versions. Package versions. Connection strings. Permissions. Every layer either of us could think of, for over an hour, late on a weeknight.

Then I asked a question. "Special characters — do they make a difference?" I've been caught by that before, in systems that quietly disallow certain ones.

That was it. And this is the part worth printing:

The cause was one backtick. The panel loads those variables by writing a shell script that gets evaluated when the application starts, and a single unbalanced character aborts that evaluation — so every variable after it silently never loads. The application boots with them simply absent and raises no error of its own. The only trace is one line, in a log I hadn't thought to open:

And then the fix didn't work either. I restricted the password to the documented allowed characters — still failed. I replaced it entirely — the error disappeared from the log and it still didn't work. So removing the special characters didn't actually solve anything; the whole mechanism was broken regardless. It was late. We abandoned the panel and moved everything into a plain file read directly by the application, and it has run that way ever since.

That was pretty much the whole night.

Why this still isn't the worst kind of failure

The first time I hit this — same underlying problem, much earlier model — it went differently. That one ran in circles. Check this. No. Then it must be that. No. Then it must be this again. Round and round, never widening the search.

This time we went outward. Wrong for an hour, but wrong in a direction. Both are failures. Only one of them makes you want to throw the laptop across the room.

What I took away: throw everything at it. Could it be this? Could it be that? Ask the question that sounds too obvious to ask — it costs nothing, and it's the one that found the backtick.

Running in Circles at 10:30 on a Sunday

The usage window is five hours and it starts when you start. Begin in the evening and your reset lands somewhere around one in the morning.

There was a stretch — earlier Opus versions, and occasionally the one after — where it would get stuck. Not slow. Stuck, in a loop:

And the whole time, that burns the budget. It's 10:30 on a Sunday, I'm trying to finish so I can sleep, I've hit the limit, and the reset is at one in the morning.

I said this to it in the chat, in as many words: "Listen — are you doing this on purpose? Are you trying to get me to burn all my tokens so I have to re-up?"

It said it knew I was frustrated and that it would be frustrated too. It still couldn't do it. Excruciating is the word.

A good share of the top-ups in the table above came from that stretch. They weren't enthusiasm. I was buying capacity to climb out of holes the tool had dug.

It is markedly better now — that's a real change, not a mood. But I'm not going to pretend the earlier stretch didn't happen, and if you're weighing this against a rival, that's the failure mode to ask about. For what it's worth, the other coding model I use daily at work did not do this.

My mitigation, which works: run a second agent behind the first. Plan it, do the job, then send something else in to verify the work. It costs more and it catches things.

And here is the honest cancellation condition, since reviews rarely state one: I'd leave if I could no longer do my own side work on the $20 plan. I don't think I could keep pumping extra money in for a few more tokens just to finish something before one in the morning. If something else came along that was as useful and gave me more room, I'd consider it.

Twenty Mistakes

To be fair about this, I had Claude go back through its own working notes from the week we rebuilt this site and find every time it got something wrong. Not a summary — the list. Those mistakes were expensive, and what we paid in was time.

Nineteen documented errors between 26 July and 1 August 2026, grouped by type, because the pattern matters more than the count. The twentieth arrived later, and it's at the bottom.

1. Confident statements it hadn't checked — seven of them, and an eighth at the end

• 01
  Told me a gap in my billing history with another writing tool was a lapse followed by a discounted win-back, and built an argument on it. Nothing had happened. The payer simply changed.
• 02
  Told me my employer could see my writing statistics, because an admin console showed one member. It's my own account. Work just pays the invoice.
• 03
  Called a "fewer than 2 members" message a dashboard defect. It's a privacy floor — sensible design, working correctly.
• 04
  Decided that a document list showing no year meant the current year. I disproved that in one line: the same list showed months that hadn't happened yet.
• 05
  Assumed a server process was running one way when it was actually running under a supervisor, and wrote an entire control script on that assumption — one that would have been dangerous to run.
• 06
  Worried that an API request would be blocked by account permissions. My own account. No issue.
• 07
  Said a layout bug affected six files. It affected three.

Every one of those was a confident statement resting on something it hadn't verified. In four cases I corrected it from a receipt or a screenshot sitting on my own screen.

2. Code it wrote that was wrong — four

• 08
  The control script above, which would have killed a supervised process that gets resurrected within a second, then started a second copy fighting over the same port.
• 09
  A health check pointed at a public address that the server's own filtering answers with an error code — so a perfectly healthy service reports as down. That's a false alarm at three in the morning.
• 10
  A browser widget that worked exactly once and then hung forever, for two independent reasons: it re-used a token that had already been spent, and a guard it added deadlocked whenever a check never resolved. Hours.
• 11
  Two syntax errors in its own throwaway tooling. Minutes, but visible.

3. Edits that looked finished and weren't — four

• 12
  A full article rewrite left 161 lines of the old version sitting underneath the new one, because the cut stopped at the first closing tag it found.
• 13
  Replacing a sidebar block removed the opening half of a link and left the attributes dangling.
• 14
  One edit silently didn't save at all — an internal check failed before the write, and the page was reported to me as updated.
• 15
  An insert aimed at the wrong kind of element, put nothing useful anywhere, and left debris sitting after the closing tag of the document.

4. Instructions to me that didn't work — three

• 16
  Where to find a setting on my phone. Wrong twice before it was right.
• 17
  A command to run in a folder I wasn't in, which failed on the path.
• 18
  A command copied out of a file's own header comment, which assumed an environment I didn't have loaded and crashed on a missing built-in.

5. Sloppy verification — one

• 19
  Claimed a stylesheet was missing a rule and added a duplicate of one already there. It reverted it and told me, which is the right ending to a bad start.

6. And the twentieth, made while writing this page

• 20
  Drafting the money section above, it worked out that the five top-ups on 29 March 2026 must have been the day I launched this site — because eight pages carry that date in their code. It wrote that up as a finding: that's the launch, priced. It wasn't. That Sunday I was going through my server file by file, because of code in a page header that Claude itself had flagged two days earlier.

Two dates matching is not a cause. I caught it because I was there and it wasn't.

To its credit, it marked the guess as a guess and asked me to confirm before publishing, which is the only reason this is a paragraph rather than a false sentence sitting live on the internet. But it still wrote the confident version first — on the page whose entire argument is that confident sentences built on matching dates are the expensive kind of wrong.

If you want one example of what working with this thing is actually like, that's the one.

What the list actually says

Its own summary of the pattern, which I'm quoting because I can't improve on it: it is most dangerous when it's being helpful about something it hasn't verified.

Look at the shape of the list. The code bugs are ordinary and they get found, because code either runs or it doesn't. The expensive failures are the confident inferences, because they arrive sounding like findings.

And here's the thing I keep coming back to. Every single correction in this project came from a human going and looking at something — an invoice, a screenshot, a console — and never from either of us remembering harder. That's the real finding about working with an assistant, and it's why this list belongs in the review instead of a footnote.

Would I change anything, having seen it? No. This is exciting. It's something I'm passionate about.

It Argues With Me, and That's the Product

I used the other big assistant for three years before this, and the difference isn't raw capability. It's whether there's a conversation.

There, you ask for something and you get it. Do this. Okay. Do that. Okay. Ask how to do something and it tells you how, and that's where it stops. That's not an insult — it's a description, and for a great many jobs it's exactly what you want.

Claude answers, and then keeps going. Here's the answer — but honestly, if I knew more about what you're building, I'd say this isn't a connectivity problem at all. This is pointing at something else. Want me to check?

That sounds small. Across eight months it's the entire thing, and it's why I stopped opening the other tab.

One limit on that comparison, and I want to be straight about it. I've barely used the other company's dedicated coding tool, so what I'm comparing is Claude against a chat product, not against their best coding offering. I did use it once, to review a codebase. It consumed an entire week's credit allowance doing nothing but reading — not a day's, a week's. It found real problems, which I fixed. Maybe it was a restricted seat with a weaker model behind it; I'll never know, because I'm not going back. Credit where it's due: thank you, you did a good job, and I'm never using you again.

AI Hygiene: Make the Backup Yourself

Here's the thing nobody tells you, and I only learned it because these tools sometimes clip files while editing them.

Always make your own backup copy. You make it — not the agent.

If an agent must make one for you: have it verify the copy matches the original, then copy that backup to a different drive. And use Time Machine, or whatever the equivalent is on your machine. Do not trust an agent, frontier model or not, with the destruction of your folders.

This isn't about malice. It's about speed. Picture something processing a thousand times faster than you, writing its own tools to do the job, working at the pace your computer actually runs — on an operating system that already stutters and does odd things to you at human speed. Now imagine that happening a thousand times faster.

And there's a gap in the tools themselves that bothers me more than the clipping does. If an agent's own edits have truncated files across the last three sessions, and it has written that down, shouldn't that trigger a thought? This keeps happening. Let me make a copy before I start. It doesn't. You have to ask.

I'm not lecturing from a position of safety. I've lost whole projects because I wasn't paying attention. I deleted the wrong folder once, and when the box asked whether I was sure I wanted to permanently delete something that large, I clicked yes out of reflex — the way you dismiss any dialog that's got in your way. Then I sat looking at the place where the project used to be. That was terrifying.

What I check every single time

I'm relaxed about the contents of a working folder, because I've copied the whole folder before we start. I may lose a session's progress. I won't lose the project, and that security is worth more to me than any individual safeguard.

What I never skip: the article, before it publishes. I open the page in Pinegrow and look at it properly. And I check it on mobile — a step I had to consciously add, because I was so used to everything being fluid that I assumed it would just work.

Notice what isn't on that list. I don't re-read every line of generated code. The backup is what makes that acceptable. Verify the output; insure the process.

What This Actually Replaced

I ran a review site before this one, about vaping hardware. Physical products, which means buying the thing and living with it — a week at minimum, preferably two, ideally a month. That part hasn't changed and can't be automated. There are assumptions you make in the first hour that only get corrected in the second week.

What changed is everything after.

The old process went: Friday night after work, build the page framework. Find a photo, or shoot one on my phone and edit it to look less like a phone snap. Drop in a placeholder — empty space, some dead tags. Then sit and stare at the screen, trying to get the whole shape of the review straight in my head while simultaneously fixing text bleed and image borders.

Then Saturday, the writing. Which was the part I hated:

Write until frustrated. Lunch. Come back if the headspace was there. Reach a midpoint. Sunday: images, check the rest of the site, do something else, come back, finish. Then paste the lot into a word processor or a grammar checker, which would tell me a sentence was passive and decline to say what to do about it. How do I? What do you want me to say?

Best case, two and a quarter days. Worst case, about a week — which happened whenever I lost the shape of the thing mid-draft. I'd erase it, sit there, and try to find it again. I never stockpiled. Order one, live with it, review it, publish, then order the next.

I hated that process. The finish, when it was done, was great. But I hated the process.

This one took an eighty-two minute recording. The one before it was recorded between eight and quarter past nine one morning and was ready that afternoon.

Two caveats, because the comparison isn't clean. I never logged drafting and revision time separately — only recording time — so "same day" is accurate but the exact hours aren't on record. And hardware reviews aren't software reviews. Some of the saving is the subject matter, not the method.

The headline survives both. The step I hated — getting a fully-formed thought out through a keyboard, losing it, starting over — is gone. Not sped up. Removed. I talk for an hour and the shape survives the trip out of my head.

Why I'm happy to hand off the grammar

I am not, and have never been, a fan of grammar. Not because I dislike English — I love reading. And it's funny: I see these sentences and words and phrases all the time. But in my head I'm not looking at words. They become visual once you've read enough, and you stop paying attention to the mechanics.

That one observation explains nine years of running a grammar checker, and it explains this. The part that's invisible to me is exactly the part I want handed off.

And my apologies to the grammar people out there — the ones who genuinely enjoy that side of English. I'm glad you exist. If we were all like me, the language would be butchered into oblivion.

How to Use Any of These Without Getting Burned

This is the part I'd want a friend to read, and it applies to every one of these tools, not just this one.

Don't take health advice from AI. Don't take stock-market advice from AI, no matter what the confident young man in the video is showing you. Do not do it.

Researching a company before you put money into it is a perfectly good use — but don't rest on the one thing it told you. Go and look it up yourself. And here's the method I actually use:

Two more rules, both about you rather than the tool. Don't put personal information in. Not because of what these companies are, but because it's your personal information and you're typing it into an application connected to the internet — the same rule you'd apply anywhere else. And don't share chat sessions. That share button makes a public web page. In theory search engines don't crawl those. As we keep reading in the news, in practice it happens.

Who This Is For

It's for me, and I want to be precise about why. I write, I build websites, I work with servers, and I can read logic in languages I can't write. That combination is exactly the shape this tool is built around. When I say five out of five, what I mean is that it fits my use case almost perfectly — not that it's perfect.

If you don't code, I don't know. You may well find it much the same as any other frontier model. The everyday use case — the one for someone who isn't building things — hasn't been nailed down yet, by anyone. That's why the bottom row of my scorecard is a 2.5, and I'm not going to pretend otherwise just because the rest of it works for me.

Start on the free tier. They all have one, and for most people it's plenty — friends of mine use the free versions for writing help and questions and get real value from them. If you find yourself repeatedly bumping into the ceiling, the decision makes itself.

On the alternatives I've actually tried: I found one of them lacklustre enough that I'm still paying an introductory price I need to cancel, and another a hot mess — though I'll caveat that second one myself, because I haven't used it enough to give you a solid verdict. Which brings me to the thing that started this whole site.

I can't trust the reviews, so I make my own. That's the same instinct as I can't criticise it if I haven't done it myself, and it's why this site exists.