In the early 2000s, my manager at the time handed me a Learning Tree catalog and asked whether any of the courses interested me because we had some training budget available. One of the courses I picked was about project management, not because I wanted to become a project manager, but because I wanted to better understand how projects are planned and managed.

The course instructor was a very entertaining and brilliant Scotsman. After all these years, I remember only two things from that course. The first was his introduction of the word “plonker”. It was not part of the syllabus, but the expression was thrown around so many times that it stuck with me, although I had to look it up to understand the term.

The answer is: “It depends”

The other was probably the most influential lesson I have carried throughout my career: for many questions, the most appropriate answer is, “It depends.” Most questions have no absolute answer. The best answer is usually nuanced and takes the context and specific circumstances into account. Most questions are not black and white, but people tend to oversimplify them or provide too little context for an appropriate answer.

Because it can, not because it should

In the age of AI, that lesson matters more than ever, because context is exactly what tells you whether a tool is the right one for the task at hand. But today’s story is not about “it depends”; it is about a related principle. In my work advising people on using GitHub Copilot (though Copilot itself is not relevant here, since this applies to all AI tools), I have noticed a pattern worth sharing: for some tasks, people tend to use AI tools because they can, not because they should.

A task better suited to a script

In 2024 (which, in AI years, seems like last century), I was helping a customer with Copilot. This was before Agent mode existed in Visual Studio Code, and the model being used at the time was most likely GPT-4o.

I was shown a case where Copilot was failing to perform a task reliably. It would correctly start streaming its output, but after a few minutes it would stop and return a message saying that it had exceeded the output token limit. GPT-4o has a maximum of 16,384 output tokens.

I started by trying to understand the task and the context of the prompt. The intention was to use a prompt to generate a set of tests from a template and a CSV file containing a long list of variables. Each row supplied the variables for one test, so applying the template across every row produced a very large amount of output.

Although the task was not overly complicated, the combination of the template and the number of rows in the CSV meant that the required output would exceed the token limit. The problem was not that Copilot or the model was incapable of performing the task; the required output was simply too large for the model to return in a single response at the time.

Although today’s models may be able to handle much larger versions of this task, that does not mean we should use them for it. This was a deterministic task that a simple script could perform several orders of magnitude faster than an LLM.

We ended up using Copilot to write the script. Writing it took less than five minutes, and the generation process could then be repeated in milliseconds.

Can it, or should it?

Since then, “it depends” has gained a runner-up. My second most common answer to “Can Copilot do X?” is:

Yes. But just because it can doesn’t mean it should.

The same question applies to any AI tool.

A practical rule

This principle should be applied when writing skills, instructions, and prompts, as well as when using AI tools directly. The rule is simple: if a task, or part of one, is deterministic and repeatable, make it executable instead of relying on AI reasoning, and use AI for the parts that require interpretation, ambiguity, or human judgment.

This has several benefits:

  • Running a script is usually faster than using AI reasoning.
  • A script is deterministic and repeatable, so the same input will produce the same output, making errors easier to reproduce, detect, and correct.
  • A script can be covered by automated tests, making its behavior easier to verify and changes safer to introduce.
  • An agent can execute the script, allowing it to delegate deterministic work instead of reasoning through every step.
  • Executing a script is usually cheaper than using AI reasoning, both in terms of time and money.

The takeaway

The point is not to avoid AI, but to use it where it adds value. Let AI help create or select the right deterministic tool, then let that tool perform the repeatable work. The important question is not only whether AI can do something, but whether it is the right way to do it.

And there is no need to choose sides here: writing scripts is something AI tools are genuinely good at. Use AI to write the deterministic script, then let the script do the deterministic work. A win-win.