Posted on — Leave a comment

Is prompt engineering real AI development, or just good documentation?

I’ve heard both answers from people I respect. One senior engineer told me prompt engineering is “writing a very long comment and hoping.” A teammate who ships LLM features for a living told me it’s the highest-leverage skill in AI development right now. After building a fair number of LLM-powered features, I think they’re both partly right, and the useful question is when each one is true.

The case for “it’s just documentation”

A good prompt looks a lot like good documentation. It states the goal, defines terms, gives examples, lists edge cases, and says what the output should look like. If you’ve ever written a clear README or onboarding doc, you already know most of the craft.

There’s also no compiler. You can’t unit test a sentence the way you test a function. The same prompt can give different results on different days, and “it works on my five examples” tells you almost nothing. If your prompt engineering is rewording until the demo looks good, then yes, that’s editing, not engineering.

The case for “it’s real AI development”

The moment you add evaluation, the picture changes. Real prompt engineering looks like this:

  • A test set of real inputs, including the ugly ones
  • A scoring method, whether human review, rules, or a model-as-judge
  • Versioned prompts, so you can see which change helped and which broke something
  • Regression checks every time the prompt or the underlying model changes
  • Decisions about where a prompt should stop and code, retrieval, or fine-tuning should start

That isn’t documentation. It’s an engineering loop that happens to use natural language as the interface. The prompt is a component with inputs, outputs, failure modes, and a maintenance cost.

Where I land

Here’s my rough test: If you can’t tell me whether your last prompt change made things better or worse, you’re doing documentation. If you can, with numbers, you’re doing development.

The word “prompt” isn’t what separates the two. Measurement is. A prompt without evals is a guess with good grammar. A prompt with evals, versioning, and clear ownership is a real part of your system.

Also Read: To become better at prompt engineering, learn how to think like a manager

What I’d tell a team starting out

  • Write the prompt like a spec first. Clarity is half the work.
  • Build 20-50 real test cases before you tune anything. Otherwise you’re optimising for your own taste.
  • Track every prompt change like a code change. If it’s not in version control, it doesn’t exist.
  • Know when to stop prompting. Some problems need retrieval, tools, or plain old code, not another paragraph of instructions.
  • Re-run your tests when the model updates. Prompts are tied to the model they were written for.

The uncomfortable part

If prompt engineering is “real” development, then a lot of AI projects have been shipping without anyone doing development. They have impressive demos and no tests.

If it’s “just documentation,” then the people getting reliable results are doing something else on top of it that we haven’t named yet. I suspect that’s the answer: the prompt is the visible tip, and the engineering is the eval and iteration process underneath.

Your turn

Where do you draw the line? Have you shipped a prompt-heavy feature to production, and what was the thing that actually made it reliable? I’d especially like to hear from people who think I’m wrong.

—

Editor’s note: e27 aims to foster thought leadership by publishing views from the community. You can also share your perspective by submitting an article, video, podcast, or infographic.

The views expressed in this article are those of the author and do not necessarily reflect the official policy or position of e27.

Join us on WhatsApp, Instagram, Facebook, X, and LinkedIn to stay connected.

The post Is prompt engineering real AI development, or just good documentation? appeared first on e27.

Leave a Reply

Your email address will not be published. Required fields are marked *