Backing up AI skills and agents with a care for security and privacy

Early on, I wanted to make sure that I had a good backup of all my skills and agents. So I started using a private GitHub repo that Claude automatically pushes to every time I update my configuration. I’d hate to rebuild them from scratch.

All good and well, but Claude and I both contribute to those files. And yes, I do have 2FA setup for my GitHub account, and my password is very strong. But still. In an age where we almost weekly get severe security reports and breaches, I try to keep any tokens and personal identification information (PII) off GitHub.

Details about people can live in my (local) notes, but the backup can only point to those details. I cannot rely on myself each time remembering this, so I added a check.

Before each commit, a hook scans the new lines for names of people, credentials and email addresses. It blocks the commit if it finds one, and if it isn’t sure, it blocks rather than skips. I keep an allow-list of product names and public figures that I cite, so those can be included in the repo.

Since I set it up, it has blocked a commit when the name of a colleague was introduced. She had shared an update to a similar skill we use (but with our own preferences), and when I asked Claude to update my own one it found it necessary to mention my colleague by name. No harm done (I’m even happy to name Andrea in my post as you really should follow her), but it shows how strict I want to be with privacy and what I back up online.

Next to that, it only looks at new lines the commit adds, so old text will never retrigger. I recently found it still wasn’t strict enough, so we tightened the rules again. We then re-ran it over 180 earlier commits, and it found nothing more. Finally, we deleted the old GitHub repo and created a new one to cut the exposure of the old commit history. That lowers the risk but doesn’t remove it: anything already cloned or cached is out of my reach, and if a real secret had been in there, I would rotate it rather than rely on the delete.

This is a safety net, not a guarantee. A hook that Claude can edit isn’t a security boundary. The check and its allow-list now live in a folder owned by root, and the repo’s git config is locked, so changing either takes my password. Claude runs as me, so this stops the agent from editing or skipping the check, not someone with my login. That said, Claude can still edit the settings file that triggers the backup, and the check reads its list of names from my notes, which Claude can also write to. For this reason, I’m also doing a manual security audit on a regular basis as well.

How are you setting up agents and making sure that it’s secure?

Comments

Leave a Reply

This site uses Akismet to reduce spam. Learn how your comment data is processed.