Prototype it with the tools you have

Exit

Lesson 3 · 20 min

Prototype it with the tools you have

By the end

A working first version of your tool, tested on five of your own examples, with the results in your sheet.

  1. 1Learn
  2. 2Try it
  3. 3Human check
  4. 4Quick check

Learn · 5 min

Build to learn, not to launch

A prototype is a rough, working version you build to learn, not to launch. Its job is to answer one question quickly: can this task be done well enough, with what you already have, to be worth a real trial?

You probably don’t need code for a first version. Three things you likely already have get you surprisingly far:

  • A custom assistant: a saved assistant with your own instructions and reference files, so every chat starts from the same rules. Most of the big assistants (ChatGPT, Claude, Gemini, Copilot) offer this, each under its own name; check whether your plan includes it.
  • A form (Google Forms or Microsoft Forms, for example), so the input arrives in the same shape every time, with the details the tool needs. If the tool keeps asking for the same missing detail, add that question to the form.
  • A spreadsheet, to hold your test set, each version’s outputs and your pass or fail marks.

The instructions are the tool

Write the instructions from the scope and test set you already have, clearly enough that a new hire could follow them, and keep facts out of them. Your price list, policies and service area go in reference files, short and current, so there’s one place to update.

Tell the assistant to use only facts from those files. A long, messy file makes it easier to miss the one line that matters, and an out-of-date price list produces confident, wrong answers.

In plain language, the instructions cover:

  • What it receives, and what to ask for if something’s missing
  • The steps, in order
  • The exact output format: headings, length, tone
  • What it must never do: your red lines, word for word
  • When to stop and write “Needs a person” instead of guessing

Test, change one thing, test again

Run your test examples one at a time, each in a fresh chat with any memory feature switched off, so earlier answers can’t leak into later ones. Use examples the instructions haven’t seen: if you pasted three into the instructions, test on the others. Paste each output into your sheet next to what good looks like, and mark it pass or fail yourself. Don’t ask the assistant to grade its own work.

When something fails, change one thing: a line in the instructions, a missing fact in a reference file, a question on the form. Save each version with a number (v1, v2) and rerun the whole test set, not just the example that failed. A fix for one case can quietly break another.

Expect the same input to get slightly different answers on different runs; that’s how these models work. Run the awkward examples twice. If the answers differ in ways that matter, the instructions aren’t specific enough yet.

Stay in the middle, and stop at good enough

In a prototype, you’re the connection between the pieces: you copy the form entry in and the draft out. That’s a feature. You see every output, and you learn where the tool is weak before anything runs on its own. Connecting the pieces with a tool like Zapier or Make can wait until the prototype has earned it; the Automator path shows how.

Stop building when the prototype clears the bar you set, and move to a pilot. Polishing a tool nobody has used on real work is an easy way to lose a month. A prototype can’t tell you whether people will use it on a busy Tuesday; that’s what the pilot is for.

Try it on your business

15 min

Write version 1 of your tool, set it up as a custom assistant, and run it on five examples from your test set.

  1. 1

    Fill in the brackets from your scope and test set, run the prompt, then read the instructions line by line and fix anything that isn’t how you work.

  2. 2

    Create a custom assistant, paste in the instructions (keep a copy marked v1) and add your reference files. If your plan doesn’t offer custom assistants, keep the instructions in a doc and paste them at the start of each chat.

  3. 3

    Run five examples from your test set, not the three you pasted, including at least one awkward one. Use a new chat for each, and switch off memory while you test if your assistant has it.

  4. 4

    Paste each output into the v1 result column and mark it pass or fail against the checks you wrote.

  5. 5

    Note the first thing to change for v2 in the Notes column. Change one thing at a time, then rerun all five.

Write the instructions for a custom AI assistant that does one job only: [THE TASK, FROM YOUR SCOPE].

My scope: [PASTE YOUR ONE-PAGE SCOPE].
Three real examples, names removed, with what a good result looks like for each: [PASTE 3 ROWS FROM YOUR TEST SET].
My red lines: [PASTE YOUR RED LINES].
Reference files it will have: [E.G. PRICE LIST, SERVICE AREA, RETURNS POLICY].

In plain language, cover: what it receives, and what to ask for if something is missing; the steps, in order; the exact output format; what it must never do, using my red lines word for word; and when to stop and write “Needs a person” instead of guessing. Tell it to use only facts from the reference files, and to say so when the answer isn’t in them. Number the sections and keep it under 300 words.

Human check: before you use it

Quick check · question 1 of 2No grades

Your prototype quoted a wrong price because the price list it was given is out of date. What’s the right fix?

Ask the Distillery

Answers come from The Distillery’s own pages, with sources.

If the Distillery doesn’t know, it says so and points you to a person.