RAG explained: how AI answers from your own documents

RAG (retrieval-augmented generation) means the system fetches the most relevant passages from your own documents before the language model answers, and the model answers from them rather than only from what it learned in training. Answers come with sources, and the knowledge stays current without retraining the model. Creating embeddings for 10,000 documents costs under USD 2.

What is RAG?

A language model only knows what it learned during training. It doesn't know your price list, your returns procedure or the contract you signed last year. Ask it anyway, and you risk an answer that sounds right but is made up.

RAG solves this by looking things up before answering. The method was described by Lewis and colleagues in the 2020 paper "Retrieval-Augmented Generation for Knowledge-Intensive NLP Tasks", and it's now the most common way to make AI answer from your own data. The model stays the same, but it gets to read relevant sources before it answers.

When an AI agent answers employees from handbooks and procedures, RAG is almost always what sits underneath.

How RAG works, step by step

1. Split the documents. Long documents are split into chunks of a few hundred words, often with some overlap, so each passage can be found on its own.
2. Create embeddings. An embedding model turns each chunk into a vector, a long series of numbers that describes what the text is about. Texts with similar meaning end up close to each other.
3. Store them in a vector database. The vectors are stored in a database that can find the nearest ones quickly, such as Qdrant, Pinecone, pgvector or Meilisearch.
4. Search. The question is turned into a vector, and the database finds the chunks closest to it. The best systems combine this with ordinary keyword search.
5. Rerank. A reranker reads the question and each candidate and sorts the hits by how well they actually answer it.
6. Answer with sources. The language model gets the best chunks and answers from them, citing where the answer comes from.

We've written more about vectors, hybrid search and how it works in practice in our semantic search case study with Meilisearch.

Which embedding model is best for Norwegian?

It depends on the task. The Scandinavian part of the MTEB benchmark tests models on 28 tasks in Norwegian, Swedish and Danish. The figures below are our own averages, calculated from the published results. NorQuAD measures how well a model finds the right passage for a Norwegian question.

Google's gemini-embedding-001 and Qwen3-Embedding score around six points above OpenAI's best model on average, and most of the lead comes from classifying and clustering text. On the task closest to RAG, finding the right passage for a Norwegian question, OpenAI's text-embedding-3-large is still the best of all the models tested. Google has since launched the successor gemini-embedding-2, whose vectors can't be mixed with the old model's.

In April 2026 the National Library of Norway released a Norwegian model of its own, nb-sbert-v2-large. It's built for Norwegian and can run locally, but it scores lower than OpenAI on the benchmark. Its successor Borealis arrived in September 2026. It's small enough to run on ordinary servers, handles up to 8,192 tokens per text and reports 58.9 on the Scandinavian benchmark, roughly level with OpenAI's small model. It has its own licence from the National Library.

The Danish company The Tech Collective has built TTC-L2V, a larger model based on AI Sweden's Llama-3-8B with an MIT licence. In June 2025 its developer reported that it ranked first for Norwegian, Danish and Swedish on the Scandinavian Embedding Benchmark of the time. It isn't in today's MTEB results, so it can't be compared directly with the models in the table. With around 8 billion parameters it needs a GPU with plenty of memory, so it fits best when the data must stay on your own servers and you have the hardware.

In practice, two other things often decide the choice:

• Where the data ends up. With Qwen3, e5, BGE-M3 and nb-sbert, everything can run on your own servers in Norway or the EU. With the API models, the text is sent to the provider.
• Switching models. Vectors from different models can't be mixed. If you switch models, every document needs new embeddings.

Price is rarely the problem. OpenAI's text-embedding-3-small costs USD 0.02 and large USD 0.13 per million tokens. Ten thousand documents of around a thousand tokens each therefore cost USD 0.20 with small and USD 1.30 with large.

Our advice is to test two or three models on a sample of your own questions and documents before you choose.

Qdrant, Pinecone, pgvector or Meilisearch?

All four can store vectors and find the nearest ones. The difference lies in operations, price and where the data can live.

Qdrant is our default when vector search is the core of the system. It can run on your own servers, has good support for filtering and hybrid search, and the company behind it is based in the EU.

Pinecone is the easiest to get started with, but the free plan runs only in the US. If the data must stay in the EU, you need a paid plan.

pgvector is often the right choice when you already have PostgreSQL, because you avoid running another database. Note that the HNSW index in pgvector supports up to 2,000 dimensions for ordinary vectors, while text-embedding-3-large has 3,072. You then need half precision (halfvec) or shorter vectors.

Meilisearch fits when you need search on the website anyway. The semanticRatio setting controls the mix of keyword search and semantic search.

Hybrid search and reranking

Pure vector search is good at meaning and poor at exact words. When someone searches for a product number, a name or a legal clause, ordinary keyword search does better. Hybrid search combines both, and in most of the RAG systems we build it gives noticeably better hits.

A reranker is the next step. It reads the question together with each of the top candidates and scores them again. It's slower than vector search, so it's only used on the top 20–50 hits. Cohere Rerank is an API, while bge-reranker-v2-m3 has open weights and can run locally. Watch the licence: several popular rerankers and embedding models are free only for non-commercial use.

RAG can still be wrong

RAG reduces made-up answers but doesn't eliminate them. The model can still misread a source, mix up two documents or answer even when search found nothing that fits.

The most common failures we see:

• The documents are split badly, so the answer is spread across chunks that aren't found together.
• Search doesn't find the right document, often because the question uses different words than the source.
• The sources are outdated, and nobody is responsible for keeping them current.
• The model goes beyond the sources and fills in what's missing.
• A user gets answers from documents they shouldn't have access to.

That's why you need to measure the quality. Tools such as Ragas let you track whether answers stay true to the sources and whether search finds what it should.

Privacy and access control

The Norwegian Data Protection Authority (Datatilsynet) followed NTNU in a regulatory sandbox project on Microsoft Copilot, which also searches internal documents. The final report from November 2024 points to the same thing we see in projects: access control and information management must be in order before AI gets to read the documents. Sensitive personal data must be labelled, a data protection impact assessment (DPIA) is normally required, and the rollout should happen in small, controlled steps.

In a RAG system, this means search must respect the access rights of the person asking. An employee should only get answers from documents they already have access to.

If RAG is used in a customer-facing chatbot, the transparency requirements of the EU AI Act also apply: people must know they're talking to a machine.

Checklist: before you build a RAG solution

1. Pick one knowledge area. Start with one collection of documents, such as customer service procedures or the health and safety handbook.
2. Clean up the sources. Remove outdated versions and decide who is responsible for keeping the documents current.
3. Map the access rights. Search must follow the same access rights the documents have today.
4. Decide where the data will live. Choose whether embeddings and the vector database should run in Norway or the EU, or whether an API is acceptable.
5. Test several embedding models. Use a sample of your own questions and pick the model that finds the right passage most often.
6. Use hybrid search. Combine vector search with keyword search, and consider a reranker.
7. Require sources in the answers. Users should be able to see which document an answer is based on.
8. Measure quality continuously. Track accuracy and answers that go beyond the sources, and update when the documents change.

We build RAG systems and AI agents and automation with access control, sources in the answers and quality measurement, and use Python and machine learning where off-the-shelf tools fall short.

Sources and further reading

• Retrieval-Augmented Generation for Knowledge-Intensive NLP Tasks, Lewis et al. 2020
• MTEB results, Massive Text Embedding Benchmark
• Qdrant, Pinecone, pgvector and Meilisearch hybrid search
• NTNU: Copilot through the lens of data protection, Datatilsynet 2024