SupportFlow RAG Portfolio

A local, source-grounded assistant for a fictional inquiry-management SaaS. The demo shows document ingestion, hybrid retrieval, citations, unanswerable-question handling, and evaluation.

FastAPI React/Vite Vector-ready SQLite OpenRouter or Ollama
Ask
Documents
Evaluation
Logs
What does WEBHOOK_SIG_INVALID require checking?
api_specs/errors_reference.csv 0.94
release_notes/2026-04-release.md 0.88
support_templates/api_error_response_macros.csv 0.83

What The Demo Proves

40

Sample documents

Eight SupportFlow categories across Markdown, TXT, CSV, JSON, PDF, and DOCX samples.

40

Evaluation cases

Questions cover lookup, synthesis, API entities, unanswerable prompts, and paraphrased user wording.

4

Demo screens

Ask, Documents, Evaluation, and Logs give reviewers the operational picture of the RAG pipeline.

RAG Flow

1. Ingest

Load SupportFlow manuals, FAQ, runbooks, API specs, release notes, tests, templates, and rules.

2. Normalize

Preserve file path, source ID, category, file type, title, and section metadata for every chunk.

3. Retrieve

Blend vector search with exact keyword matching for codes such as AUTH_401 and RATE_429.

4. Answer

Generate concise, grounded answers with citations and refusal behavior when evidence is missing.

5. Evaluate

Measure Recall@3, Recall@5, MRR, cited-answer rate, keyword match, and unanswerable accuracy.

Screen-Share Walkthrough

Ask
Documents
Evaluation
Logs

Evaluation Story

The benchmark is small enough to review manually but broad enough to expose common RAG failure modes: missing citations, weak exact-match retrieval, overconfident answers, and poor synthesis across sources.

Recall@3 / Recall@5

Do expected sources appear near the top?

MRR

How early does the first correct source appear?

Refusal Accuracy

Does the assistant avoid inventing missing facts?

Demo Boundary

This portfolio is meant for local review through Docker Compose. The static presentation can be hosted separately, while the RAG app, sample corpus, and evaluation workflow remain inspectable in the repository.