For the last 6 months, I have been making full use of AGY, a CLI tool giving the ability of frontier AI models to work on your own computer, said very simply, although I think this is something already everybody knows. The thing I learned very quickly and that I would like everybody to immediately understand if they are starting out or just never thought about, is that you need an orchestrator. What is an orchestrator? An orchestrator has many different definitions especially as of right now: there are orchestrators made to simplify multi agent work, there are orchestrators that are just automations for the AI to avoid him making mistakes and so on.

The orchestrator I personally think is a must have for anybody the second they start using an AI, is an orchestrator that saves you from AI mistakes, saves you tokens, and lets you actually modify post and pre hooks of the AI giving you actual control of the model instead of being at its mercy. When you interact with a model through a standard CLI, you quickly realize how much waste and fragility is baked into the defaults. The model is polite, verbose, and context-blind. It re-reads files that haven’t changed, burns tokens re-parsing tool definitions on every turn, and easily gets stuck in sequential loops trying to diagnose an issue one micro-probe at a time. Left to its own devices, context explodes quadratically, cache misses become the baseline, and you remain at the mercy of non-deterministic behavior.

Seeing this, I just couldn’t accept this, I felt like most of everything I want it to do is spent in something that is not in the scope at all. So, I did a bit of research, thought about it a little and came out with my personal orchestrator, of which I will leave a link to my github because I cannot say it better than my readme already does. Anyway, this post is not so much about pushing my orchestrator, it is also only made to be working with AGY, but more so to tell all people getting into this kind of coding to absolutely give it a look, make their own research and implement one.

Secondly, what I started realizing as more and more personal projects were started, finished, and still work stable until now is how much freedom AI can give to normal, non coding expert people. Personally I never studied code, only enough that I know its basics, I know how to read it (more python than anything, but if I really have to I can get behind other languages too) but I could never sit down and start writing code. This was a limitation that was felt pre AI: I could download a repo of a tool I need, modify it up to the level my skills permitted to have it exactly how I want, but that was it. If the tool had quirks, or didn’t do what I actually needed it for, half of the work still fell on me. With AI, and good understanding of project architecture, this is not the case anymore, at all.

Before I continue, I want to make something very clear especially for people that are just starting out, you HAVE to understand what you want to do, how you want to do it, and how to implement it, AI isn’t a guessing machine. If, for example, I want to build an orchestrator for my AI, the wrong thing to do is open your CLI and ask the AI “Make me an orchestrator that saves on tokens and keeps memory of things”. If given a prompt like this, the AI will spit out something that will probably do the opposite of what you wanted. It is still up to the person to do research, understand what are the different methods, architectures, ways of doing it, and then give the AI the whole path, where the AI only puts the planks on a bridge already laid by coding.

Now, after this little ex cursus, AI isn’t a “know it all, fix it all” tool. It makes mistakes, and its up to you to mitigate them, or stop them completely. One way, is as said above, give full architecture, AI only implements it. The second way, is to make the AI leave documentation of every single function of whatever projects you are working on in organized MD files that you can open, and read through everything to understand exactly what it does, how it works, and compare it to the actual code. The amount of obsolete, unused cells of code I have found that the AI just forgot to delete after refactors of certain repositories is incredible. It’s not really the AI’s fault, it is his limitations, especially if you start new chats to clear context, the AI can’t be always sure if something is still needed or not, and codebases get more and more polluted.

Thirdly, don’t let AIs guess. If you are working on UI, describe to the AI ONCE what you want it to look at, possibly with a reference image. Before you make him build, tell him to give you a way to manually set UI after it finishes, or to generate organized code where each function is clearly labeled by what it controls, how to move what it controls and so on. AIs don’t have eyes, or feeling of order. Small reminder that I think is known by everybody, is to always use either github or Forgejo for every single one of your projects, as small and stupid as it could be. Reverting mistakes is a lot easier with them.

Finally, always think outside the box, and connect the dots. With experience a lot of bugs and glitches that happen are obvious with just a glance, telling an AI why something is broken is a lot more effective that just telling him “fix it, make no mistakes”.

The idea is that a tool is only as good as your knowledge on how to use it. AI is not inherently stupid, bad, evil or good, but the way you use it can make it either one of them, in a purely efficiency based scope, not a emotional one. For all the problems that AI can bring to us as a global society, it also has given a level of independence and freedom to everybody that was never seen before.

(For anybody interested, here’s the link to my personal orchestrator: https://github.com/jJohn313/agy-sovereign-orchestrator )