AI tool maintenance is not something you set up once and forget. I found that out last week, and it cost me three hours I did not plan on spending.
I use a CLI tool called RTK, short for Rust Token Killer. It sits between Claude Code and my terminal and rewrites common commands so they return less noise. Less noise means fewer tokens. Fewer tokens means I do not blow through my usage limit by lunchtime.
RTK just shipped a paid Pro tier. I read the Pro command list the way anyone does when a tool they like adds a paywall, half curious and half worried about what I might be missing. Then I did the thing I actually tell my clients to do. I went back and read the free tier commands too, the ones I had been running for months without a second look.
That is where I found rtk discovery.
What AI Tool Maintenance Actually Looks Like
rtk discovery is a free command. It scans your recent sessions and reports which commands slipped past the tool without getting optimized. I ran it expecting a clean report. I did not get one.
So I had Claude review the output with me, the way I review anything that looks off before I trust my own read on it. Claude found the actual bug.
Every command RTK is supposed to handle was rewriting correctly. git add became rtk git add. npm install became rtk npm install. The rewrite logic worked. But every single one of those commands was returning exit code 3, which means ask, instead of exit code 0, which means auto allow.
That is not a small difference. Exit code 0 means the rewritten command runs silently and RTK gets credit for the savings. Exit code 3 means Claude Code stops and asks me for permission every time. If you have ever hit deny out of habit, or a permission prompt got dismissed without you noticing, the raw command ran instead. RTK never touched it. It never got credit.
The Bug That Was Hiding in Plain Sight
That one detail explained a number I had been quietly annoyed by for weeks. My token savings sat at 0.7 percent, even though the hook was wired up correctly and every rewrite was technically firing. The tool was doing its job and then asking permission to have done it, which in practice meant it was barely doing its job at all. That is exactly the gap a real AI tool maintenance check is supposed to catch.
Some tools fail loud. You get an error, you fix it, you move on. Some tools fail quiet, and you only notice because a number looks wrong. And some tools fail so quiet that the only way you catch them is by reading documentation for a feature you already assumed you understood. RTK was the third kind.
I spent three hours fixing that bug and a few adjacent ones I found while I was in there. Three hours, on a free tool, installed through Homebrew, that I trusted because I set it up once and never saw an error.
Why Nothing Is Set It and Forget It
Here is the part that goes beyond one CLI tool. Nothing in your stack is set it and forget it. Not a free tool, not a paid one, not something you configured carefully yourself. Homebrew packages update. Defaults change. A permission model gets stricter in one release and looser in the next, and nobody sends you a memo. It is the same blind trust I have warned about with AI plugin lock-in. The tool keeps running, so you assume it is still doing what it did on day one.
I already run a support plan for clients who do not want to think about this on their WordPress sites. Plugin updates, security scans, backup verification, the boring work that keeps a site from becoming a 2am phone call. AI tooling deserves the same discipline. I write about keeping my coding context in git instead of trusting an app to remember it for me, and this is the same habit pointed at a different tool. You would not install a security plugin once and never check its logs again. You should not install an AI CLI tool once and assume it still works the way it did the day you installed it.
The Case for an AI Tool Maintenance Day
I am not telling you to audit every tool every day. That is not realistic and it is not necessary. Pick one day, monthly at minimum, weekly if the tool sits close to your daily workflow, and actually read what it is telling you. Run the discovery command if it has one. Read the changelog. Check the exit codes if you can get to them. The bugs that cost you the most are the ones quiet enough that you never think to look.
A well groomed token optimization setup could not fix my issue on its own. I still had to go looking for it. Set it and forget it was never a real strategy. AI tool maintenance is what replaces it, one afternoon at a time.
Leave a Reply