RAG (Retrieval-Augmented Generation)

RAG is how you make Brinpage answer with your own source material (docs, SOPs, product notes, internal policies) instead of relying only on the model’s memory.

In Brinpage, RAG is built around a simple workflow: you store documents (with tags) → you run ingestion (to build/update the retrieval dataset) → you call /api/sdk/ask with tags/k/minScore to retrieve the best chunks and ground the answer.

How Brinpage RAG works

Docs: You upload docs to your workspace (each doc has file, title, content, tags).
Ingest: You run POST /api/sdk/ingest. Brinpage reads all Docs, normalizes tags, chunks the text, computes embeddings server-side, and stores a RAG dataset (embeddings + a tag index).
Ask: You call POST /api/sdk/ask with a question. If you pass tags, Brinpage pre-filters candidate docs by tag and ranks chunks by similarity (top-K, thresholded by minScore).

Endpoints you’ll use for RAG

All /api/sdk/* endpoints require a valid license key via Authorization: Bearer ....

1) Create / update a document

Use POST /api/sdk/doc with mode = create or update. filename becomes the document path/identifier inside the workspace. The doc includes title, tags and full content.

bash

# Create a doc (copy/paste example)
export BRINPAGE_API_BASE="https://platform.brinpage.com"
export BRINPAGE_LICENSE_KEY="bp_xxxx"

curl -sS "$BRINPAGE_API_BASE/api/sdk/doc" \
  -X POST \
  -H "content-type: application/json" \
  -H "authorization: Bearer $BRINPAGE_LICENSE_KEY" \
  -d '{
    "mode": "create",
    "filename": "docs/product/faq.md",
    "data": {
      "title": "Product FAQ",
      "tags": ["product", "faq"]
    },
    "content": "## Refunds\nWe offer refunds within 14 days..."
  }'

If you later update the same doc, send mode: "update" and the same filename.

2) List documents

Use GET /api/sdk/docs to list doc metadata (file/title/tags).

bash

curl -sS "$BRINPAGE_API_BASE/api/sdk/docs" \
  -H "authorization: Bearer $BRINPAGE_LICENSE_KEY"

3) Run ingestion (build/update the RAG dataset)

Ingestion runs per workspace and overwrites the stored dataset. Operationally, it reads all docs, builds a tag index, creates chunks, computes embeddings, and persists the dataset (embeddings + tag index).

bash

curl -sS "$BRINPAGE_API_BASE/api/sdk/ingest" \
  -X POST \
  -H "authorization: Bearer $BRINPAGE_LICENSE_KEY"

4) Ask with RAG controls (tags / k / minScore)

/api/sdk/ask supports tags, k and minScore to control retrieval. It also supports conversation history.

bash

curl -sS "$BRINPAGE_API_BASE/api/sdk/ask" \
  -X POST \
  -H "content-type: application/json" \
  -H "authorization: Bearer $BRINPAGE_LICENSE_KEY" \
  -d '{
    "question": "What is your refund policy?",
    "history": [{"role":"user","content":"Hi"},{"role":"assistant","content":"Hello!"}],
    "tags": ["product", "faq"],
    "k": 6,
    "minScore": 0.18,
    "debugEcho": true
  }'

The response typically includes an answer, the usedDocs list and optional debug metadata when debug is enabled.

Tuning: k and minScore

k controls how many chunks can be included (higher = more recall, sometimes more noise).

minScore is the similarity threshold (higher = stricter, cleaner grounding; lower = more permissive).

Note: if nothing meets the threshold but there are candidates, Brinpage can still fall back to the top candidate so you don’t get an empty retrieval set (useful for early datasets).

Best practices

Tag strategy: keep tags stable and meaningful (product areas, modules, versions, teams). Tags are used as a fast pre-filter before embedding ranking.
Doc hygiene: fewer, clearer docs beats dumping everything. Split big manuals into smaller docs by topic.
Ingest after changes: update docs → run /ingest so retrieval stays accurate.
Start permissive, then tighten: begin with a lower minScore while your corpus is small, then increase as you add more content.

Troubleshooting

401 Unauthorized: missing/invalid license key (all SDK endpoints require license).

No usedDocs: you likely haven’t ingested after uploading docs, or your tags don’t match the document tags.

Irrelevant grounding: increase minScore, reduce k, and/or make tags more specific.