architecture

Fact vs RAG: When to Use Which

Why hardcoded facts often beat RAG retrieval for Claude Code workflows. Decision rules, hybrid patterns, cost tradeoffs.

8 min read·2026-05-14

FACT: Kill Your Vector Database

Why deterministic prompt caching beats fuzzy embedding search: 42 ms vs 2-5 seconds, 93 % cheaper.

42ms

Avg Latency

93%

Cheaper Than RAG

Embeddings

Live

Data Always

The Two Approaches

RAG vs FACT, Defined

Two fundamentally different strategies for giving an LLM access to external knowledge. Understanding the mechanics of each is the prerequisite for every decision below.

RAG: Retrieval Augmented Generation

Embed all documents into vectors: Chunk text, send each chunk through an embedding model, store in Pinecone / Weaviate / Chroma.

Embed the incoming question: Same model converts your question into a vector at query time.

Cosine similarity search: Find the “closest” vectors. Returns something like what you asked, not exactly what you asked.

Inject top-K chunks into context: Stuff retrieved chunks into the prompt, then generate the answer.

FACT: Fast Access Cached Tooling

No embeddings, no vector DB: Zero embedding API calls. No vector space, no dimensionality, no cosine math.

Deterministic MCP tool calls: “Run this exact SQL. Return this live API result.” The answer is the current truth.

Prompt caching for static context: Schemas, docs, and config are cached at the prompt level, reused across calls without re-sending tokens.

Exact results, 42 ms: Deterministic queries return precise data. Not “something close”: the actual answer.

When Facts Win

Four Scenarios Where Hardcoded Facts Beat RAG

RAG adds latency, cost, and probabilistic noise to problems that don't need it. If your data fits any of these four profiles, skip the vector database entirely.

Structured / relational data

Tables, markdown with clear fields, REST APIs with JSON responses. Deterministic queries return exact rows. No embedding model will improve on a WHERE clause.

Frequently updated data

Data that changes daily or hourly will be stale in any vector index. FACT reads live sources on demand. The answer is always the current truth.

Cost-sensitive workloads

Embedding API calls at scale are a real cost center. FACT's caching layer costs 85-93 % less per query. For high-volume pipelines, that gap compounds fast.

Latency-critical paths

RAG adds embedding + vector search + re-ranking before the LLM even sees a token. FACT collapses the pipeline to a single cached prompt + tool call: 42 ms end-to-end.

When RAG Wins

Three Scenarios Where RAG Is the Right Tool

RAG is not dead. It is just misapplied most of the time. These are the cases where semantic similarity search is genuinely the correct approach.

Unstructured long-form text

Research papers, legal docs, creative writing. When “close enough” is the goal and exact field matches are impossible, cosine similarity is your friend.

Static corpus, embed once

A fixed knowledge base (product docs, historical FAQs, archived content) that never changes. Embed once, query forever. No re-indexing pipeline, no drift.

Cross-document concept search

“Find everything related to payment failures across 400 support tickets.” Semantic search across unstructured text is what embeddings were built for.

Hybrid Patterns

Using Both Together

The best production systems don't pick one. They route intelligently. Structured queries go to FACT; fuzzy semantic search goes to RAG; the orchestration layer decides which path each question takes.

Route by data type

Inspect the query intent. If it targets a known entity (user ID, project name, date range) → FACT tool call. If it asks for conceptually similar documents → RAG retrieval. A simple classifier at the agent level handles this.

FACT for metadata, RAG for content

Use FACT to pull exact metadata (author, date, tags, status) from a database, then optionally use RAG to surface related document bodies. Combine the precision of structured queries with the breadth of semantic search.

Warm the prompt cache with FACT

Pre-load the static context tier (schema, config, system prompt) via FACT's prompt caching. Then let the dynamic retrieval layer use RAG only for the content chunks that change. Cache hits on the static layer cut costs even when RAG handles the rest.

FACT as a pre-filter

Run a deterministic FACT query first to narrow the candidate set (e.g. “all tickets from the last 7 days”), then run semantic search only over that filtered subset. Smaller corpus = better RAG precision + lower cost.

Core Concept

FACT's Tiered Caching Architecture

Three cache tiers with different TTLs. The right data gets the right caching strategy: static context is cached for hours, live data is read fresh on every call.

TIER 1

Static Cache

Schemas, documentation, configuration, system prompts. Changes rarely. Cache aggressively.

TTL: hours / days

TIER 2

Semi-Dynamic Cache

User preferences, settings, feature flags. Changes occasionally. Moderate TTL.

TTL: min / hours

TIER 3

Live Cache

API responses, real-time data, user actions. Changes constantly. Short TTL or bypass.

TTL: sec / min

Head to Head

Cost & Performance Comparison

Every metric that matters. FACT wins on latency, cost, accuracy, and freshness. RAG wins on unstructured semantic coverage.

Metric	RAG	FACT
Latency	2-5 seconds	42 ms
Cost	High (embeddings)	85-93 % cheaper
Accuracy	Fuzzy / probabilistic	Exact / deterministic
Data freshness	Requires re-indexing	Live on demand
Setup complexity	High (vector DB + pipeline)	Low (MCP tools)
Unstructured text	Native strength	Not applicable

Decision Framework

Decision Tree: FACT or RAG?

Five questions. Answer them top to bottom. The first match is your answer.

Is your data structured (tables, markdown fields, live APIs)?

Structured data with clear schemas and relationships is ideal for deterministic queries. A SQL WHERE clause beats cosine similarity every time.

→ Use FACT

Does the data change frequently (daily or hourly)?

Live data will be stale in any vector index. FACT reads from source on demand, so the answer is always current.

→ Use FACT

Is cost or latency a primary concern?

Embedding API calls at scale add up. FACT's caching approach costs 85-93 % less per query and returns results in 42 ms.

→ Use FACT

Do you have a static corpus that never changes?

A fixed knowledge base (product docs, FAQs, historical archives) is RAG's sweet spot. Embed once, query forever. No re-indexing pipeline.

→ Use RAG

Do you need concept similarity across unstructured long-form text?

Research papers, legal docs, support tickets where 'close enough' is the goal and exact field matches are impossible. This is what embeddings were built for.

→ Use RAG

In Practice

Real-World Applications

Four live use cases. Every one replaced a slower RAG-based approach with a FACT tool call.

Client morning briefing

FACT queries all client files (README.md, Progress.md, Communication-Log.md) and returns a structured summary of status, blockers, and next actions. 42 ms instead of manually opening 15 folders.

Engineer capacity tracker

Real-time capacity checks across all Progress.md files. “Who has bandwidth this week?” returns structured data from every active project in one deterministic query.

CRM sync validation

FACT pulls current state from markdown files, compares to CRM data, flags discrepancies. Keeps the second brain and CRM aligned without manual reconciliation or stale embeddings.

Proposal generation

Instant retrieval of similar past projects, pricing, and outcomes. “Find all projects with n8n + Supabase under 25 K” returns exact matches, not fuzzy guesses.

Want this running in your stack?

AY Automate builds agent systems, RAG pipelines, and Claude Code setups for production teams.

Book a free architecture call See what we build