Skip to content
Muhammet Şafak
tr
Cover — two panels side by side in a rack: on the left APP: REQUESTS, RAG: OFF, PROVIDER KEYS: 0; on the right LLM: PROCESSED, RAG: INJECTED, PROVIDER KEYS: HELD; a unit labelled DOCUMENT STORE beyond them

Letting an application send RAG requests without ever holding a provider key

A self-hosted Go gateway holds the provider keys and injects RAG context, so the application talks to one OpenAI-compatible endpoint.

What it does today

Lets an application talk to one OpenAI-compatible endpoint while a project's configuration decides which model the request reaches and, when a document store is linked, which documents it draws context from. Anyone who does not want provider keys spread across applications, and wants spend capped per project, can run it on their own server.

Started
August 2026 — September 2026
Left Labs

Form

Open-Source Web

Technologies

Go PostgreSQL pgvector Docker OpenAI Anthropic Claude Google Gemini DeepSeek Ollama Prometheus OpenTelemetry

This one became a product:

Ragmux

In an LLM feature that runs on your own documents, deciding to use RAG is the easy half. The hard half is what carries that retrieval in production.

What is left after the decision

The documents are embedded, the chunks are in the database, search works. Three questions remain: where the provider key sits, which application uses which model, and where spend gets cut off. None of the three is part of retrieval, and solving all three inside every application means copying the same code once per application.

Ragmux moves them out of the application and into the gateway between it and the providers. The application talks to one OpenAI-compatible endpoint; which provider the request reaches, which system prompt it carries and which document store it draws context from is decided by the project’s connection.

The model name the client sends selects nothing

The model field does no routing. Which model the request reaches is decided by the project’s connection; the client’s value is echoed back unchanged in the response and in every stream chunk, so SDKs that compare that field keep working.

What that buys: the provider credential stays on the server, encrypted, and is never read back through any interface. All the application holds is the project’s own key. The cost is plain too — the client can no longer choose a model; model choice stops being a deployment decision and becomes a configuration one.

Budgets are capped in tokens, not in money

Every request’s estimated cost is calculated and shown on the dashboard, but cost is informational: no request is ever refused because a monetary threshold was reached. The way to cap spend is to cap tokens — requests and tokens per minute per project, daily and monthly token budgets.

There is a limit here that is not hidden: token counts are only final once the provider has answered, so a single request can overshoot a budget by its own size, and it is the next one that gets refused.

One PostgreSQL, no second store

Users, connections, projects, documents, chunks, vectors, request logs and usage counters all live in one PostgreSQL with pgvector. No Redis, no separate vector database. Because a counter is a row rather than a process variable, several replicas pointed at the same database share one budget.

The review that found the product breaking its own rule

Three of the five recorded decisions came out of a pre-release review, two of them from places where code and documentation disagreed: the role matrix described viewer as read-only while the code let a viewer mint their own gateway key; a price-table row reported a connection pointed at a paid endpoint as “$0.00”.

The fifth decision was softened within a day of being taken. A hard per-tenant share had been put on the image-download ceiling; measurement showed that on a single-project install — the common case — it left half the ceiling permanently idle. The share was loosened the same day: when nobody is waiting, one tenant may use the whole ceiling.

The product’s own entry is in the portfolio.

Tags: #rag#gateway#llm
Share:

More Records

All records

How a provider-agnostic Go CLI reviews a diff without it leaving the machine, and how the experiment reached signed binaries as CommitBrief.

What it does today

Reviews code locally, with the diff never leaving the machine; the `Provider` interface lets Anthropic, OpenAI, Gemini and Ollama plug into one contract. Signed binaries are published — anyone who does not want to hand their diff to a cloud provider can install it.

Open-Source Go Cobra go-git +4 more
January 2026 — April 2026

A formula sets Devrazzi's 0–100 score; the model only writes the explanation, behind a verification gate. The decision and how it was built.

What it does today

Collects release and vulnerability announcements across the software ecosystem into a single feed, gives every event an importance score from 0 to 100 and explains its impact in Turkish; it also tracks deprecation/EOL and RFC announcements. Useful to software engineers who want to see what changed in their stack without combing through hundreds of sources by hand every day.

Web Laravel PHP PostgreSQL +8 more
October 2026 — October 2026

Build notes on serving the documentation of many projects from one multi-tenant server, each with its own MCP endpoint and documents kept on your machine.

What it does today

Makes a project's folder, git repository, Obsidian vault or Notion workspace searchable behind a single MCP endpoint, so an agent reaches the documentation without the data going to a third party. Anyone who can run Docker can host it and manage it from the dashboard.

Open-Source Web TypeScript Node.js Fastify +6 more
August 2026 — September 2026

Search the site

Start typing to search posts, projects and pages.

Esc to close Powered by Pagefind