Engineers advised to start RAG with full-text search before adding embeddings

The article presents a staged approach to RAG, advocating that engineers begin with the simplest possible retrieval (full-text search) and only add complexity when data shows it is necessary. This counters the common habit of jumping straight to embeddings and vector databases. The shift is from a "one-size-fits-all" advanced stack to a decision-driven, incremental path. Adopting this mindset can prevent over-engineering early in a project.

Cost of adopting the MVP approach: Using BM25 via Elasticsearch, Postgres, or similar incurs negligible licensing or compute expense, and the article notes zero API costs. The main investment is operational expertise in configuring a traditional search engine, which is generally lower than the skill set needed for embedding pipelines. However, the approach does not handle synonym-rich or semantic queries, which may require additional effort later.

Where the simple approach stops working: When query patterns shift to conversational or semantic forms, full-text search misses intent and synonym matches, leading to poor recall. The article indicates that at this point engineers can introduce agentic query rewriting, which adds a modest per-query cost (~$0.001 per query) and can improve results without altering the underlying index. If query volume exceeds roughly 10 000 per day or the corpus changes rapidly, further stages such as hybrid search or full embedding pipelines become justified.

Broader implications for engineering teams: By following the decision factors, data freshness, corpus characteristics, query patterns, scale, and team capabilities, teams can make evidence-based choices about when to invest in ML components. This reduces the risk of model deprecation and chunking strategy headaches described in the article. Ultimately, the simpler start lowers barriers to entry, speeds up iteration, and aligns infrastructure spend with actual user needs.