RAG Tools

How Much Does a RAG Chatbot Cost?

A worked example combining LLM, embedding, and infrastructure cost for RAG.

There's no single answer — it depends on your usage, model, and document set. But walking through one example makes the cost drivers concrete. These are illustrative assumptions, not a quote; plug your own numbers into the AI Budget Planner for a real estimate.

Example assumptions

5,000 monthly active users, 15 requests/user/month → 75,000 requests/month
300 input tokens + 4 retrieved chunks of 500 tokens each = 2,300 effective input tokens/request
300 output tokens/request
A budget-tier LLM at roughly $0.15 / $0.60 per 1M input/output tokens
10,000 documents, 500 tokens each, indexed once, on a $0.02/1M-token embedding model
Illustrative infrastructure: $20 vector database + $5 storage + $25 hosting

Where the cost comes from

Formula

LLM cost       = (2,300 × 75,000 / 1,000,000) × $0.15  +  (300 × 75,000 / 1,000,000) × $0.60
               ≈ $25.90 (input)  +  $13.50 (output)  =  ~$39.40

embedding cost = (10,000 × 500 / 1,000,000) × $0.02 ≈ $0.10 (one-time index)

infrastructure = $20 + $5 + $25 = $50

Total: roughly $89/month — call it ≈ ₹8,500/month at current conversion, or well under ₹1/user. The LLM call itself, not embeddings, is the dominant cost here — which is typical: retrieval usually costs far less than the tokens it adds to every request.

What moves this number the most

Retrieved chunks per query. Retrieving fewer, better-ranked chunks cuts input tokens directly — see choosing chunk size and overlap.
Model choice. A frontier model can cost 10–20x more per token than a budget model for the same request volume.
Request volume. Cost is linear in requests — a viral spike changes the bill as much as any technical decision.

Run your own numbers in the AI Project Cost Calculator or the full Planner, which also surfaces rule-based optimization ideas once you have a baseline.