• Many formats — docs, sheets, slides, PDFs — each needs text extraction
• Chunking decisions — size and overlap choices govern retrieval quality forever after
• Batch economics — embedding APIs have hard per-request limits; one bad doc shouldn't kill the run
• Idempotency — re-indexing a changed file must not duplicate or leak old chunks
• Same-model invariant — embedder at index time MUST equal embedder at query time

• Context budget — typical 50–80K tokens of context, spread across many sources
• Coverage — cross-cutting queries ("list all X") must see every source, not just the loudest
• Citations — every factual claim cited; nothing invented; links back to source
• Follow-up handling — "compare their margins" embeds badly; needs rewriting
• Structured + retrieved together — first-party data supplements vector search

Turn unstructured documents into a searchable vector index. Chunking, embeddings, vector stores, batch economics.

Use that index to answer questions with citations. Retrieval budgets, context assembly, query rewrite, presentation.

A simple paragraph-aware sliding window — the default in most production systems →

~15 lines. 1500/200 is a fine starting point. Tune chunking before tuning retrieval — bad chunks ⇒ no amount of clever ranking saves you.

def chunk_text(text, target=1500, overlap=200):
    text = (text or "").strip()
    if len(text) <= target:
        return [text] if text else []
    chunks, start, step = [], 0, target - overlap
    while start < len(text):
        end = min(start + target, len(text))
        # Prefer a paragraph break in the back half
        if end < len(text):
            min_break = start + target // 2
            for sep in ("\n\n", "\n", ". "):
                idx = text.rfind(sep, min_break, end)
                if idx != -1:
                    end = idx + len(sep)
                    break
        chunks.append(text[start:end].strip())
        if end == len(text):
            break
        start = max(end - overlap, start + step // 2)
    return [c for c in chunks if c]

-- one-time schema
CREATE EXTENSION IF NOT EXISTS vector;
CREATE TABLE chunks (
    id        bigserial PRIMARY KEY,
    content   text,
    embedding vector(768)
);

# --- index one chunk ---
vec = embed(chunk)          # 768 floats
db.execute(
    "INSERT INTO chunks (content, embedding)"
    " VALUES (%s, %s)",
    (chunk, vec),
)

# embed the query with the SAME model
q = embed("how do I reset my password?")

rows = db.execute(
    "SELECT content"
    "  FROM chunks"
    "  ORDER BY embedding <=> %s"
    "  LIMIT 5",
    (q,),
).fetchall()
# <=>  cosine distance — smaller = closer
# returns the 5 nearest chunks

pgvector operators: <=> cosine, <-> L2, <#> inner product. Add an HNSW index on embedding for O(log n) search.

-- build the index once
CREATE INDEX ON chunks
USING hnsw (embedding vector_cosine_ops)
WITH (m = 16, ef_construction = 64);

-- the query is unchanged: planner
-- uses the index automatically
SET hnsw.ef_search = 40;   -- recall vs speed
SELECT content FROM chunks
  ORDER BY embedding <=> %s
  LIMIT 5;

• m — links per node; ef_construction — build-time effort. Higher = better recall, more memory.
• ef_search — query-time effort; the recall ↔ latency dial you tune live.
• Approximate — may miss the true nearest now and then; match vector_cosine_ops to the <=> operator you query with.

The simplest possible in-memory store (this is just a toy example):

class InMemoryStore:
    def __init__(self):
        self.records = {}  # chunk_id -> record

    def upsert(self, records):
        for r in records:
            self.records[r["chunk_id"]] = r

    def search(self, query_vec, k=5):
        scored = [(_cosine_distance(
                       r["embedding"], query_vec), r)
                  for r in self.records.values()]
        scored.sort(key=lambda x: x[0])
        return [r for _, r in scored[:k]]

Production systems use HNSW or similar approximate-nearest-neighbor (ANN) structures — O(log n) instead of O(n) search. Don't pick the heavyweight option first; in-memory works until it doesn't.

• Most embedders have hard per-request caps (Vertex: 250 inputs and 20K tokens — whichever hits first)
• Non-Latin scripts tokenize denser — char-based estimators undercount
• Adaptive fallback: catch the cap error, halve the batch, retry

• Re-indexing burns money — don't accidentally re-embed on every deploy
• Track per-doc status (fetching / extracting / embedding / indexed / failed) — one bad PDF can't kill the whole run
• Backfills — re-indexing the whole corpus in bulk (after changing chunking, embedding model, or onboarding a new source) — take hours, so they need pause/resume
• Delete-then-insert keeps re-indexing consistent

One verbose document dominates top-K. Other relevant docs never make it. Cross-cutting queries ("list all companies that…") miss most of the answer.

Fix: per-group budgets, MMR diversification, or rerank with a different signal.

Top-K by cosine returns chunks that look textually similar but don't actually answer the question. "Best match per chunk ≠ best match per concept."

Fix: reranker (cheaper LLM scores each candidate), hybrid retrieval (vector + keyword), or query rewriting.

"What about their revenue?" — embedding this matches lots of revenue chunks but for wrong companies. Pronouns kill vector search.

Fix: rewrite the question into a self-contained query using conversation history. A cheap LLM call.

Naively stuff K=50 chunks → 100K tokens → cost & latency balloon, model gets distracted by noise, citations get sloppy.

Fix: budget chunks, group by document, trim ruthlessly.

• Trust — users can verify each claim
• Debuggability — when an answer is wrong you can see which chunks led there
• Defense against hallucination — forcing the model to cite makes it visibly own its claims
• Affordance — users click through to source documents

• Invents [99] for a chunk that doesn't exist
• Cites in jumbled order — [5][1][8] — feels janky
• Skips citations on claims that need them
• Cites the right chunk for the wrong claim
• Refuses to cite, just says "based on the documents…"

• documents/ — 20 Wikipedia articles (Apollo missions + Saturn V, LM, CSM, Mission Control), ~850 KB
• indexer.py — walks docs → paragraph-aware chunking → embed → index.pkl
• backend/app.py — Flask chat: top-K retrieval, citation system prompt, [n] parsing
• frontend/ — React chat UI; renders answers with a Sources: line
• .env.example, requirements.txt, README.md

1. Run the indexer: build the vector index over the documents/ corpus — one command, then confirm chunks landed in the store.
2. Test the app: start the backend + frontend and ask it real questions about the corpus. Watch what it retrieves and how it cites.
3. Show successes & failures: collect a few questions it answers well — and a few where retrieval misses or citations break down. Bring concrete examples of each.

Six PDFs from 14 CFR — Aeronautics & Space, ~15 MB of dense legal text with cross-references, tables, and definitions:

• Vol. 1 — FAA general (Parts 1–59): definitions, certification, airworthiness
• Part 61 — pilot certification  ·  Part 67 — medical standards
• Part 71 / 73 — airspace designations & special-use airspace
• Part 91 — general operating & flight rules

1. What aeronautical experience is required for a private pilot certificate with an airplane single-engine rating? Part 61
2. Which medical conditions disqualify an applicant for a first-class airman medical certificate? Part 67
3. What are the fuel-reserve requirements for VFR flight, day versus night? Part 91
4. How do operating requirements differ between Class B and Class C airspace? Parts 71 + 91
5. What must a pilot do before operating in an active restricted area? Part 73

1. Everyone builds a RAG chat over the FAA corpus and tunes it on the 5 practice questions.
2. On contest day a fresh, unseen question goes to two entries head-to-head.
3. The better-grounded, better-cited answer advances. Repeat until one remains.