Retrieval-Augmented Generation is an architecture that builds a model's answer from context fetched out of documents. For each question the model receives the relevant pieces as context; because the source stays separate from the model, changing a document changes the answer.

What does a RAG engineer do?

A RAG engineer parses and chunks documents, turns them into embeddings, builds the search and hands the fetched context to the model. For me the job does not end there: where the key sits, which model is chosen and where spend is cut off are part of the same work.

A fine-tuned model carries frozen knowledge: when the documentation changes you retrain it, and you cannot ask which document an answer came from. What makes RAG the choice is not that it is cheap but that the source stays separate from the model. I also do fine-tuning and model adaptation; the two answer different problems.

How do you build the search in RAG?

Every chunk sits in both a vector index and a full-text index, and I fuse the two rankings. A constant name, an error code or a flag may be something the embedding model has no representation of, and the full-text side catches it. I fuse by rank, not by score: score weights would have to be re-learnt every time the embedding model changed.

Which vector database do you use?

My default is pgvector: I keep vectors in the same PostgreSQL as the user, document and request records and do not set up a second store or a coordination layer. The cost is that everything rests on one PostgreSQL, which has to be sized accordingly; when one PostgreSQL is not enough and a collection grows large I use Milvus, and Chroma for quick prototypes.

Where do you produce embeddings?

In the documentation search tools I run on my own server, built to be queried constantly, the default is the machine's own CPU: a tool that asks for an API key in order to work at all breaks the "runs on your own server" promise at the outset. A few environment variables switch it to cloud embeddings, but the default path never leaves the host. The cost is paid at first start: the model is downloaded and the container takes a few minutes to come up. In gateway setups that go through a provider, limits are checked before retrieval, so a throttled client does not even spend an embedding call.

What happens to a request when retrieval fails?

I do not drop the request: if the reranking service does not answer, the search backend is unavailable or context cannot be fetched, the event is logged and the request continues without context. The cost is plain: in the response, context that could not be fetched and context that matched nothing both come back as the same zero-hit count, and the difference lives only in the log. Returning a 500 looks more honest and is worse in practice: instead of a slightly weaker answer, the user gets none.

Expertise

RAG engineer (Retrieval-Augmented Generation) — since 2024

Deciding on RAG is the easy half: I have built retrieval pipelines since 2024, and the rest of the work shows up in where the key sits, which model is chosen and where spend stops.

years: 2

Let us work together

Me in the RAG ecosystem

The full story: About

After the RAG decision three questions remain, and none of them is part of retrieval: where does the provider key sit, which application uses which model, and where does spend get cut off. They begin once documents are parsed, chunked and embedded and search works; I wrote about how those questions turned into four design decisions on the blog.

RAG itself is for me a question of boundaries before it is a question of search: is the model’s answer built from its own memory, or from the document fetched for this question. When I choose the second, the source stays separate from the model; change the document and the answer changes, change the model and the document stays where it is. The AI systems page places this decision in a wider engineering frame; this page is the retrieval pipeline itself.

Opening context to a model has another route, MCP: there retrieval sits behind a server, and the server becomes a trust boundary. Which route I take depends on where the context lives and who will reach it.

What is the RAG Engineer role?

RAG (Retrieval-Augmented Generation) is an architecture that bases a model's answer not on its own memory but on context fetched from documents at the moment of the question. A RAG engineer designs how documents are split and turned into embeddings, how search is done and how the fetched context is handed to the model.

What does the RAG Engineer do?

In RAG pipelines I parse and chunk documents, turn them into embeddings and combine vector search with full-text search. I keep vectors inside PostgreSQL with pgvector; I reach for Milvus when a collection grows large, Chroma when a quick prototype is needed, and Ollama when a model has to run locally.

Technologies I pair it with

Retrieval-Augmented Generation

The retrieval architecture that builds the model's answer from context fetched out of documents.
Python

The main language of my model-training, fine-tuning and data-preparation work.
pgvector

The extension that keeps vector search inside PostgreSQL instead of a separate store.
Milvus

The distributed vector database I use for large embedding collections.
Chroma

A vector database that can run embedded and sets up quickly for retrieval prototypes.
Ollama

A tool that runs open-weight models on a local machine and serves them behind an API.
PostgreSQL

The database I measured queue-table and index behaviour on.
Model Context Protocol

A protocol that opens tools and context to a model; I treat every connected server as a trust boundary.

Frequently asked

8 questions

How long have you worked with RAG?

Since 2024, for 2 years.
What is RAG?

Retrieval-Augmented Generation is an architecture that builds a model's answer from context fetched out of documents. For each question the model receives the relevant pieces as context; because the source stays separate from the model, changing a document changes the answer.
What does a RAG engineer do?

A RAG engineer parses and chunks documents, turns them into embeddings, builds the search and hands the fetched context to the model. For me the job does not end there: where the key sits, which model is chosen and where spend is cut off are part of the same work.
RAG or fine-tuning?

A fine-tuned model carries frozen knowledge: when the documentation changes you retrain it, and you cannot ask which document an answer came from. What makes RAG the choice is not that it is cheap but that the source stays separate from the model. I also do fine-tuning and model adaptation; the two answer different problems.
How do you build the search in RAG?

Every chunk sits in both a vector index and a full-text index, and I fuse the two rankings. A constant name, an error code or a flag may be something the embedding model has no representation of, and the full-text side catches it. I fuse by rank, not by score: score weights would have to be re-learnt every time the embedding model changed.
Which vector database do you use?

My default is pgvector: I keep vectors in the same PostgreSQL as the user, document and request records and do not set up a second store or a coordination layer. The cost is that everything rests on one PostgreSQL, which has to be sized accordingly; when one PostgreSQL is not enough and a collection grows large I use Milvus, and Chroma for quick prototypes.
Where do you produce embeddings?

In the documentation search tools I run on my own server, built to be queried constantly, the default is the machine's own CPU: a tool that asks for an API key in order to work at all breaks the "runs on your own server" promise at the outset. A few environment variables switch it to cloud embeddings, but the default path never leaves the host. The cost is paid at first start: the model is downloaded and the container takes a few minutes to come up. In gateway setups that go through a provider, limits are checked before retrieval, so a throttled client does not even spend an embedding call.
What happens to a request when retrieval fails?

I do not drop the request: if the reranking service does not answer, the search backend is unavailable or context cannot be fetched, the event is logged and the request continues without context. The cost is plain: in the response, context that could not be fetched and context that matched nothing both come back as the same zero-hit count, and the difference lives only in the log. Returning a 500 looks more honest and is worse in practice: instead of a slightly weaker answer, the user gets none.

Let us work together

If you have work in this ecosystem, tell me what you are building and we will talk about how to build it.

Write to me