1Why this review matters most
This is the largest and most carefully built subsystem: about 73 Python files and ~38,000 lines. Everything else in the platform draws on it. If the retrieval is good, every channel answers well; if it is poor, nothing else can compensate.
2Scores
| Category | Score |
|---|---|
| Architecture & design | 9.5 |
| Ingestion & connectors | 9.5 |
| Retrieval pipeline | 9.5 |
| Quality assurance | 9.5 |
| Functionality | 9.5 |
| Security | 9.5 |
| Scalability | 9.2 |
| Code quality | 9.0 |
| Documentation | 9.0 |
| Performance | 8.8 |
3How it works
Ingestion: the material we actually have
- Documents: PDF (with per-page OCR for scanned pages), Word, Excel (with formula awareness), PowerPoint, text, Markdown, CSV, JSON, XML.
- Web: a crawler that follows links across a public site or intranet, plus a markup extractor that cleans HTML and server-side page templates (PHP, ASP/ASPX, JSP and others) into readable content.
- Email: EML, MBOX and PST, with redaction on ingest (all email addresses, plus keys, tokens, passwords and credentials).
- Databases: a SQL extractor turns named queries into retrievable rows, so it can answer about the numbers as well as the documents.
- Source code and audio/video (via transcription).
- Re-ingesting a source deletes its old chunks first, so content is never duplicated.
Retrieval: exact where it can be, semantic where it must be
- Deterministic branches first. Curated Q&A pairs return the approved answer verbatim, with no generation. Named code files and functions return the exact code.
- Semantic search. Cohere embed-v4.0 embeddings in PostgreSQL + pgvector (HNSW index); 30 candidates by default.
- Rerank. Cohere rerank-v3.5 cross-encoder narrows to the 10 passages the model sees, blended with similarity into a hybrid score.
- Generate with citations. The configured model answers from those passages, and each answer carries its sources.
Governance at the data level
Department, category and confidentiality are real indexed columns on every chunk, applied as SQL filters in the same statement as the vector search. For machine access, permitted tables are resolved server-side from the caller's token, so a caller cannot widen its scope by asking differently. A strict mode restricts answers to the retrieved data only.
4Objections the review answers
| Objection | Answer in the review |
|---|---|
| "Our data is a mess. We are not ready." | Nobody's data is ready. Our AI Data Expert starts with ~50 documents they know well; problems show up immediately as poor answers, at a scale where fixing them takes an afternoon. |
| "What stops it leaking confidential material?" | Chunk-level metadata filters, plus token-scoped table resolution on the server. |
| "How do we know answers are right?" | Every answer carries its source passage. A wrong answer traces to a specific chunk of a specific document. |
| "Why pgvector, not a dedicated vector DB?" | One datastore to run and secure. Vectors sit beside metadata, so filtered search is exact, in one query. |
| "Fixed 1,500-character chunks are naive." | That is only the default for unstructured prose. HTML and Markdown split on headings, spreadsheets keep rows and columns, Q&A stays one pair per chunk. |
| "One bad file in ten thousand?" | Ingestion is asynchronous per file. That file's job fails visibly in the queue and the rest continue. |
5What our team should probe
| Probe | Our finding |
|---|---|
| Retrieval quality on our corpus: 30–50 real questions with known source documents, citations scored | [p_c2_1] |
| Golden-question regression harness, run after a configuration change | [p_c2_2] |
| Embedding and rerank providers (Cohere by default): acceptable under our data policy, or switch? | [p_c2_3] |
| Performance at our corpus size (the lowest score here is 8.8) | [p_c2_4] |
| How our existing SharePoint / file permissions map to chunk metadata | [p_c2_5] |
6Verdict, as stated by the review
Approved for production, grade A. A grounded, multi-source ingestion and retrieval core with a deterministic trust layer. All production-critical checklist items (ingestion, deterministic Q&A, two-stage retrieval, regression harness, pgvector, security) are reported done.
Our assessment: [as_c2]. [as_c2_why]