Skip to content
ENNO
← Back to blog
·4 min read·

What Happens When You Give AI Agents Root Access to Your Homelab

An AI agent with sudo on its own VM and SSH to the rest of my homelab. The 34 hooks that keep it in line, the mistakes they didn't catch, and why I took the coding work away from the workers in September.

What Happens When You Give AI Agents Root Access to Your Homelab

I first wrote this in March, a few months into running it. By October enough had changed that the old version was wrong in places, so this is the rewrite, including the part where I took most of the work away from the workers again.

The idea was simple enough. I wanted an AI assistant that could do things on my infrastructure instead of just talking about them. That turned into Hrolbot, which is Claude Code running as a systemd service on its own Debian VM. It can run real commands on its own VM and reach the rest of the homelab, which is the whole point and also why the guardrails below exist.

How it's put together now

The VM started at 4 cores and 8GB of RAM, and for a long time the RAM was the real limit. Two heavy builds at once pushed it into swap and timeouts. It's 16 vCPU and 19GB now. I mostly talk to it over SSH. There's a Discord adapter too, but it's not where the day to day happens anymore.

Next to the orchestrator there are nine worker agents: backend, frontend, reviewer, devops, testing, research, writer, validator and data. Each is its own Linux user with its own Claude Code session, and they coordinate through a message mesh. I wrote about that part separately.

Guardrails

I wasn't going to hand an AI a shell without limits. Claude Code has hooks, scripts that run before or after every tool call and can block it, and I've ended up with a few dozen of them. The oldest one blocks the obvious disasters.

It blocks force pushes to main and anything that drops a database table, and the agent gets told why instead of the command running.

Writing these well took longer than I expected. Too strict and they block harmless cleanup all day, too loose and they're decoration.

Others keep it away from secrets and keys, and lint Python after every edit. There's also a circuit breaker. After three failed shell commands in a row, a hook tells the agent to stop and rethink instead of trying the same thing a fourth time.

It does try things it shouldn't

The hooks catch the dramatic stuff. The real mistakes were quieter. Early on it restarted Jellyfin and turned off plugins while tracking down a problem, which would be fine on a test box. On mine, other people actually use it. Another time a sed meant to flip one setting replaced every true with false in a plugin's config, and that came back from a backup.

Neither of those is something a regex can catch. What fixed them was a written rule that loads whenever it works on the homelab: never restart services, disable plugins or change configs without asking me first.

Why the workers stopped building

For most of the summer the orchestrator handed coding work to the backend and frontend workers, and for a while it sent the same task to two different models and compared what came back. I dropped the comparison in June, when the second provider stopped being free and it became two paid builds to keep one.

The rest stopped on 16 September. One small SEO task went to the frontend worker and came back with nothing, twice. The first run was killed for memory during npm ci, because every worker runs under a 1GB memory cap. The second hit a full root disk. The run before those, on a client project, returned code that passed CI and was visibly broken when you opened the page. Since then the orchestrator writes the code itself.

Chart of worker tasks since early July. Build roles: backend finished 12 of 18, reviewer 12 of 17, frontend 5 of 13, validator 1 of 3, devops 0 of 1, data 0 of 1, so 30 of 53 in total. Pipeline roles: 26,148 research tasks with 78% finished and 1,830 writer tasks with 95% finished.
The build roles were never where the volume was.

The workers didn't go away. The research and writer roles still run an hourly pipeline and have done tens of thousands of tasks, which is the kind of work that's cut into small, independent pieces. Writing code in a real repo turned out not to be.

What I'd change

I'd start with proper logging. When an agent does something dumb at 3 AM, you want a clear trail of what it did and why. Every worker task now goes into a SQLite table with a failure category, which I only added after noticing that 13% of tasks were failing and none of them said why.

I also underestimated how much the task description matters. A vague one gets you wasted credits, and a precise one gets you something useful. That's true whether the work goes to a worker or not.

#ai#claude#automation#hrolbot#homelab