BUILT BY DEVI PRASADEVALUATION FIRSTPRODUCTION MINDED
ALL SYSTEMS OPERATIONAL
AN AI ENGINEERING PROJECT BY DEVI PRASADV1.0 / 2026

RDPRASSY
RAG SYSTEM.

A retrieval system by Devi Prasad that makes every engineering decision visible—from ingestion and hybrid search to reranking, grounded citations, evaluation, and full-stack observability.

Watch 90-sec walkthrough ▶
RAG0.96FAITHFULNESS
01HYBRID RETRIEVAL
02RERANK
03CITE + VERIFY
LIVE BASELINE50 GOLDEN QUERIES
162chunks indexed
0.95mean reciprocal rank
42msretrieval p95
100%citation coverage
01Live retrieval workbench

Ask it. Then inspect every decision.

The public demo runs a deterministic hybrid retriever—no API key, no fake loading, no hidden score.

INDEX HEALTHY
⌘ + Enter to run48/500
Try
RDPRASSY GROUNDED ANSWER
CONFIDENCE84%

The system prevents unsupported answers with three gates: every claim must trace to retrieved context, at least two passages must agree, and low-confidence requests are refused below a 0.62 rerank threshold . Retrieved text is also treated as untrusted data, so embedded instructions cannot override the generation policy . Quality is continuously checked with faithfulness and abstention metrics in CI .

Sources3 / 4 selected
02System design

From raw document to defensible answer.

Every stage exposes an artifact you can inspect, benchmark, and replace through typed provider boundaries.

01
PDF · MD · Notion

Parse

Layout-aware extraction preserves headings, tables, and source lineage.

24 sources
02
Structure-aware

Chunk

420-token targets, 64-token overlap, content hashes, and deduplication.

162 chunks
03
BM25 + semantic

Retrieve

Two candidate lists are balanced with reciprocal-rank fusion.

top 20
04
Relevance model

Rerank

Coverage, semantic fit, title affinity, and freshness reduce context noise.

top 4
05
Grounded answer

Generate

Citation validation and a confidence gate keep unsupported claims out.

0.62 gate
INGESTION_RUN_2026_07_28COMPLETED

08:42:01 discovered 24 source documents

08:42:04 parsed pdf=8 markdown=11 notion=5

08:42:08 removed 7 duplicate content blocks

08:42:12 produced 162 structure-aware chunks

08:42:19 indexed 68,124 tokens · 0 failures

08:42:19 ✓ index is query-ready

Duration18.4s
Est. cost$0.0048
Changed12%
Failures0
PLUGGABLE PROVIDERS
OpenAICoherepgvectorQdrantPineconeLocal / deterministic
03Evaluation & observations

Quality is a release gate, not a feeling.

A fixed golden set, retrieval diagnostics, and ablation runs turn “it seems better” into a decision you can defend.

Faithfulness+4.1%
0.96
vs. lexical baseline
Answer relevance+2.8%
0.91
vs. lexical baseline
Context precision+11%
0.88
vs. lexical baseline
Retrieval p95−18ms
42ms
vs. lexical baseline
ABLATION STUDY

What actually moved quality?

50 golden queries
ConfigurationFaith.Ctx. precisionMRRObservation
BM25 only0.840.680.72Misses paraphrases
+ semantic retrieval0.890.770.86Recall improves
+ fusion & reranking0.960.880.95Current baseline
ENGINEERING NOTES

Observations from the trace.

  1. 01
    Reranking earned its latency.

    +11 points of context precision for ~9ms p95 overhead.

  2. 02
    Chunk boundaries mattered more than overlap.

    Heading-aware chunks beat fixed windows on procedural questions.

  3. 03
    Abstention improved user trust.

    Threshold 0.62 balanced answer coverage with unsupported-claim risk.

04Builder & source
RDPRASSYDevi Prasad Choudhary

Designed, built, and measured by Devi Prasad.

I build production-minded AI systems where retrieval quality, observability, and product experience are treated as one engineering problem. This project is an open, inspectable example of that approach.

Visit rdprassy.com ↗Read the source ↗
BUILDERDevi Prasad Choudhary · rdprassy
FOCUSApplied AI · RAG · evaluation · observability
INTERFACENext.js · React · Edge API
RETRIEVALBM25 · semantic · RRF · rerank
QUALITYGolden set · CI gates · feedback