My Network Configs Back Themselves Up
Every network person has a story about a dead device and a config backup that turned out to be a year old. I decided my lab would never be that story, so I built the backups into the automation instead of leaving them as a chore.
The flow is simple. An Ansible playbook logs into the switch and the firewall, pulls the running config, encrypts it with ansible-vault, and commits the encrypted file to a private git repo. Semaphore runs the playbook on a schedule, pulling the playbook itself from git. Nothing is stored in cleartext anywhere, and every backup is a commit, so I get history for free. If I want to know what changed on the firewall last month, git diff answers it.
Inventory comes from NetBox instead of a hand-kept file. NetBox is my source of truth: devices, IPs, roles. Ansible’s NetBox inventory plugin reads it live, so when I add a device in NetBox it shows up in the automation without touching an inventory file. Connection settings hang off device roles, so a new access switch inherits everything the existing one uses.
Two gotchas worth passing on. Semaphore runs playbooks from the playbook directory, which quietly breaks the usual group_vars resolution. The fix is a symlink from the playbooks directory back to group_vars, and it has to be a relative symlink or it dies in the container. And Semaphore hands its environment values to Ansible as extra vars with lowercase names, which matters when your playbook expects an exact variable name.
The whole thing is the same principle as keeping a torque spec sheet with the toolbox. Recovering a firewall at 2am is not the time to find out the backup was stale, and it’s definitely not the time to be reading documentation about how you should have set this up.