Teaching Small Models to Work as a Team
After getting a local AI coding agent running in my home lab, the next experiment was obvious: instead of one model trying to do everything, build a team. I created several sub agents and gave them one boss. The orchestrator’s only job is to delegate. Under it sit a coder, a tester, and a designer, all running on a small 4B model on my RX 6600. Nothing leaves my network and it costs nothing to run.
The test project was a small text adventure game. A design doc holds the task backlog, top task first, one tiny change at a time. The orchestrator reads the backlog, hands the coder a single task, the tester runs the game and reports back, and the loop repeats. It sounds like overkill for a toy game, and it is. That’s why I picked it. If the pattern breaks, it breaks on something that doesn’t matter.
The biggest lesson was that small local models fall apart when you hand them big asks. Give one a whole feature and it wanders off and breaks things. Give it one tiny change with clear conventions to follow and it usually gets there. The second lesson was permissions. The agents that just need to look around the codebase are read-only. Only the coder can write, and the orchestrator can’t touch files at all. Cutting each agent’s tool access also cuts the token overhead that eats small models alive, same thing I ran into with my Ansible agents.
I also built in an escalation rule: if the coder fails the same task twice, it logs the problem to my shared memory system and the task gets handed up to a bigger cloud model. Same way a shop runs. The apprentice does the oil changes, and you don’t hand them a transmission rebuild just because they’re standing there.
The same pattern is now doing real work on my network automation. A read-only recon agent investigates how my Ansible repos are structured, then a writer agent drafts playbooks that match those conventions. The takeaway for me: orchestration isn’t about making one agent smarter. It’s about limiting what each agent can do so the whole team stays predictable.