You Are Now a Manager of a Team of LLMs

Brush up on your people-management (aka LLM-management) skills

In an earlier article (Becoming an Expert LLM User is not "Prompt Engineering"), one of the subsections was “Be an Effective Manager of Your Team of LLMs”. That point was important enough that I felt the need to repost it as an entire article in its own right. I’m reproducing the section here, followed by some further commentary:

Be an Effective Manager of Your Team of LLMs

Every one of us now has a team of a dozen assistants—the models and modes I’ve listed [in this article]. And we need to learn to manage this team effectively: just as in real life, managing this team is a pain. Some of the models can get lazy unless you scold them (and some can even be bribed). All of them are capable of making mistakes, but in different areas, so you have to know which task to give whom, and how to check their work (even if you aren’t an expert in that area, and maybe you can get one of them to check the other’s work). Some are good at coding, some are good at strategy, some are good at dumb and repetitive tasks, some cost a lot less so you can overuse them while others are expensive consultants so you have to use them sparingly. Some of them need detailed and specific instructions, while others can take a high level goal and run with it. Many get tired after a long session and you have to give them a break and re-start later. Many of them forget their own capabilities and you have to remind them that they can do it. Seems hard to believe, but trust me, I’ve experienced each one of these.

Prompting is Managing

In Prompting is Managing, Venkatesh Rao writes:

The Prompting-Managing Impact Equivalence Principle
For today’s text generators, the cognitive effects of prompting an LLM are empirically indistinguishable from supervising a junior human.

and

As long as models write like competent interns, the mental load they lift—and the blind spots they introduce—match classic management psychology, not cognitive decline.

and to support the claim that prompting is managing, he cites this evidence from the real world:

• Creators shift effort from producing to verifying & stewarding (Microsoft–CMU CHI ’25 survey)

• 60 % of employees already treat AI as a coworker (BCG global survey (2022))

• HBR now touts “leading teams of humans and AI agents” (Harvard Business Review, 2025)

Across domains, people describe prompting in manager verbs: approve, merge, flag.

TMKK

So, what should you do?

If you’ve never been a manager before (and this is especially true for students, who may have never even seen a manager in action), start learning some of the basics of management. There is an entire industry of management books, so I will not provide specific recommendations.

If you’re already a manager, then great: you are in an excellent position to make maximum use of your new team of LLMs. But that means you have to learn the strengths and weaknesses of your new team, so you have to spend time experimenting with each of the different models.

Most importantly, you have to accept and internalise the fact that your role when using LLMs is that of a manager, not a student asking questions to a teacher, and not an internet surfer asking questions to Google search. There isn’t just ChatGPT. There’s a team of different LLMs that work for you.