Built a RAG assistant you can interrogate right now

How it happened

1. What it is

   It runs on the same infrastructure as the rest of the site. The corpus is twelve documents: nine case studies plus my background, my working method and my career history. I wrote or reviewed all of it. Anything confidential never goes in, which holds better than telling a model to keep quiet.

   

2. The architecture

   A Cloudflare Worker verifies the visitor, checks the input, and forwards the question to an n8n workflow. The routes are fixed: a directed acyclic graph (DAG), no loops, no tool selection at runtime. It is not agentic. Every question ends in one of five outcomes: an answer, a deflection, a redirect, a refusal, or an error.

3. Six stages, one measurable path

   1. 01Input guardrail — is this question safe and in scope?
   2. 02Intent classifier — what is being asked, in what language, and which route does it take?
   3. 03Query transformer — rewrite the question into something worth searching with
   4. 04Retrieval — vector search over the corpus in Supabase pgvector
   5. 05Answer generator — write the answer using only the passages that came back
   6. 06Output guardrail — read the draft before it ships

   

4. Decisions worth explaining

   1. 01
      Verification once per session, not once per question

      My first version verified each question separately, and it broke intermittently. Verification tokens are single-use and expire in minutes. Whether the second question worked depended on how long the visitor spent reading the first answer. I replaced it with a signed, time-limited clearance issued after one check. That single change cleared four bugs I had been treating as unrelated.

   2. 02
      Progress by polling, not streaming

      The pipeline emits about six events over seventeen seconds. Streaming would deliver each one a few hundred milliseconds sooner, which nobody can perceive. The elapsed counter animates in the browser anyway. Polling costs one endpoint and leaves no long-lived connections. Nothing to leak, to be suspended on mobile, or to be buffered by a corporate proxy. It was the duller option and the right one.

   3. 03
      Source links resolved from the retrieved records

      Early on, the model wrote out the titles and links it was citing. It eventually invented a plausible-looking one that led to a dead page. Now the model returns document identifiers. The system looks up the title and link from the records it retrieved. An identifier it invents matches nothing and gets dropped. A second check validates every link against the real routes before it reaches anyone.

   4. 04
      The question decides the language

      Someone reading the English site who writes in French gets a French answer. The site’s language setting only breaks ties on questions too short to classify.

   

5. What went wrong

   1. 01
      The first progress indicator was making its numbers up

      It showed a duration for each stage. Those durations were client-side estimates, and they came out identical on every run whatever actually happened. It looked good, which is roughly the problem. I took the numbers out. Then I built the version that reports each stage as the pipeline finishes it, with durations that were measured.

   2. 02
      An optimisation quietly starved the model

      I deduplicated retrieved passages by document, assuming repeated documents were redundant. They were different passages of the same article. Retrieval returned five, four were thrown away, and the model answered from one. It started opening answers by explaining what it could not tell me. That is what a well-behaved model does when you take its evidence away. The traces showed five retrieved and one kept.

   3. 03
      Multi-turn memory had never worked at all

      Conversation turns were written with one set of field names and read back with another. The history was always empty. Nothing errored. Follow-up questions were simply worse than they should have been, for weeks, and I had no reason to look.

   4. 04
      A live API key sat in plaintext in an export file

      I had built the node from a documentation snippet. It put the key in a node parameter as well as in the credential store. Everything worked, so nothing made me notice the duplicate. Then the parameter left the encrypted store inside an export. I rotated the key that day. In that tool a secret belongs in the credential and nowhere else.

   Before

   

   After

   

6. The blueprint that came first

   Drawn before any code was written: one classifier at the hub, four routes off it, one shared backbone underneath. The classifier was built for four routes on day one, and they activate by phase. Ask the assistant for a route that is not built and it deflects rather than improvises.

   

   Live today

   • Experience Q&A — the route this page demonstrates
   • Fit Analysis — shipped in V1.7
   • Meta, where it explains itself — shipped in V1.5
   • Entry guardrail, classifier, shared backbone, output guardrail
   • Knowledge layer and Langfuse tracing

   Planned

   • Point of View and its vetted corpus
   • The evaluation harness
   • Lead capture
7. What it costs to run

   The site, the API, the progress store and the vector database all sit inside free tiers. Model inference is the only real cost, and it sits under a monthly spend cap. One number moved capacity by two orders of magnitude. The progress store bills rows scanned rather than rows returned.

   

   290

   questions a day before the allowance ran out

   No index: every progress poll scanned the whole table

   ~50,000

   questions a day on the same allowance

   With the index: a poll reads six rows
8. What would change at enterprise scale

   1. 01
      Evaluation has to become systematic

      Today I judge answers by reading them. That works for one person and stops working immediately after that. The next step is a set of graded reference questions, run as a regression suite. It includes adversarial ones, and ones the corpus deliberately cannot answer, so a prompt change has to prove itself.

   2. 02
      The corpus needs an owner

      Twelve documents I wrote myself can be reviewed by eye. Ten thousand from a dozen teams need ownership, review and freshness rules. They also need a route for corrections. Retrieval quality is usually a content problem well before it becomes a model problem.

   3. 03
      Someone has to be watching the failures

      Here, a failed question shows a retry button and I find out when I look. In an operational system, someone needs to know it failed and why. They also need to know whether it is happening more than yesterday. Tracing is the foundation for that; alerting and ownership are the rest of it.

   4. 04
      Cost becomes a design input

      At this scale the free tier absorbs my mistakes. At enterprise volume, retrieval depth, context size and model choice all carry a number. Somebody has to be accountable for that number.