Expertise
Backend developer: the queue, the database and the server behind the API
Moving a synchronous flow onto RabbitMQ showed me that a user waiting on a spinner had been acting as an accidental queue for years; the real work began when that queue disappeared.
- Since
- 2010 Since
- years
- 16 years
- posts
- 29 posts
- projects
- 10 projects
Me in this ecosystem
Backend development, for me, is not a language but a boundary of responsibility: everything behind the screen the user sees. Contract, queue, database and server are four faces of the same work, and a decision about one binds the other three.
Observability, infrastructure managed as code and rollback strategy I write about as architectural subjects on sade.dev.
What I do on the backend
What they have in common is sitting behind the screen the user sees: the contract, the queue, the database and the server.
-
The API contract
The response shape, the error contract and the versioning decision are settled before code is written. Otherwise every client team writes its own assumption, and that assumption becomes your problem six months later.
-
The asynchronous delivery path
Queue, consumer and outbox relay; the structure that takes the synchronous wait out from in front of the user. The second half of the same discipline is idempotency: a request that can be repeated without side effects.
-
Database decisions
Index and plan behaviour is chosen by measurement, not intuition; intuition works particularly badly here. The measurement itself does not live on this page but in the research ledger, together with its method and environment.
-
Search and the server layer
Elasticsearch at the boundary where relational stops being enough; the environment with Docker, the deploy pipeline with GitHub Actions, nginx in front. The scope stays at application level.
A queue does not guarantee delivery
The mistake I see most often is assuming work becomes durable the moment it moves onto a queue. The chain is read in order; each step closes what the previous one left open.
- 01 Writing and publishing are two systems
- There is no atomicity between an INSERT into the database and a publish to the queue. If the write succeeds and the publish drops on a network hiccup, the row exists and the event never left.
- 02 The message is written in the same transaction
- The message is not published directly; it is written as a row into an outbox table inside the same transaction. The record and the message then share a single fate.
- 03 A relay publishes the rows
- A separate relay reads those rows and hands them to the queue. This is where the message stops getting lost — but not where the guarantee is complete.
- 04 An idempotent consumer completes it
- The outbox gives you at-least-once: the message is not lost, but it can repeat. Writing every processed event id into a processed_messages table inside the same transaction makes that repeat harmless.
My scope on data and the server
I do not make database decisions on intuition, and on the server side the scope stays at application level.
-
Index decisions made by measurement
Index choices, query plans and behaviour that shifts as a table grows are selected by measuring. A decision made without it can slow tomorrow the query it sped up today.
-
A full-text search layer
It takes over where the relational store stops being enough. Stemming, synonym expansion and relevance ranking exist there too; what brings in a separate layer is not their absence but how far they carry at scale and across languages.
-
The server at application level
Packaging the runtime environment, setting up the test and deploy pipeline, configuring the layer in front of the application.
The order that makes change safe
The hard part of back-end work is not writing a new feature but changing a running system without breaking it.
- 01 Plan the schema around the window where both versions are up
- Old and new code run side by side for a while; a schema change written without accounting for that window always breaks the running system first.
- 02 Split the data move into reversible steps
- A move that runs in one go has no way back. One split into steps can stop at the step that goes wrong.
- 03 Keep the test surface in three separate places
- Where the business rule is exercised, where the outside boundary is imitated, and where the flow runs end to end. The cost of a change has to stay smaller than the relief it brings.
I do not cover infrastructure as architecture here
Observability, IaC and rollback strategy do not stand on their own on this page; my scope here stays at application level. The place I write infrastructure as an architectural subject is sade.dev.
Recent writing in this area
The third MCP server was reasonable. The rest were not
MCP was the right answer for giving an agent a project's documentation. As the count grew, the servers themselves became the thing needing maintenance.
You chose RAG. The real work starts now
RAG or fine-tuning gets answered everywhere. The three questions left after it — the key, the model choice, the budget — get answered nowhere.
18,750 req/s: precise, repeated, and three times wrong
Two phases of the same run measured the same service three times apart. What caught the wrong one was not a better statistic but a second method.
Measurements in this area
One core carries 14,330 OAuth2 requests in Go and 5,152 in PHP-FPM
The same API verifies an RS256 bearer token on every request and then reads or writes one row in PostgreSQL — on one, two and four cores, how much mixed traffic does it carry in Go, PHP-FPM and FrankenPHP worker mode?
Finding
On four cores Go carried 57,321 mixed requests a second, FrankenPHP 25,659 and php-fpm 20,606 — 14,330, 6,415 and 5,152 per core. The number that goes into a capacity plan is not that one but application CPU per request: 66.8, 110.7 and 187.7 microseconds. At saturation FrankenPHP uses only 2.84 of its four cores, against Go's 3.83 and php-fpm's 3.87. The database is not the constraint: on the same four cores PostgreSQL alone writes 68,212 rows a second, above the mixed ceiling of the fastest candidate.
measured 17 days ago
Seven PHP frameworks under identical load: the gap narrows as soon as the request does real work
On the same hardware, the same PHP build and the same seven routes, how many requests a second do Laravel, Symfony, CodeIgniter, Yii2, Phalcon, Laminas and Slim serve, and at what latency?
Finding
On an empty route the fastest is 4.4× the slowest (Slim 25,975, Laravel 5,966 req/s). As soon as the request does real work the gap closes: 3.7× for a single row from the database, 3.5× for twenty rows. Phalcon is third on an empty route and fifth once a query is involved — being a C extension buys nothing while the process waits on MySQL. And the expensive decision is not the framework: Laravel's own default `web` middleware group takes the same response from 5,858 to 2,176 req/s, so one default costs more than most of the distance between the frameworks.
measured 45 days ago
The partial index grew three hundred and five times in fifteen minutes — and autovacuum never ran
Under sustained churn, does a partial index stay small on a queue table, and do the default autovacuum settings keep up with it?
Finding
With the live set holding at five thousand rows until about 845 seconds, the partial index went from 0.125 MB to 38.2 MB — three hundred and five times. Its smallness comes from the live set, its bloat rate comes from throughput, and nothing connects the two. The composite index bloated less in proportion (42%) and more in absolute terms (+126 MB), and while bloating it fell over — most likely because it no longer fit in memory, a cause this run did not measure: its latency went from 0.52 ms to 61 seconds and its backlog climbed to 126,000. Fifteen minutes produced 1.75 million dead rows and autovacuum **did not run once** — the default threshold scales with the whole table (50 + 0.2 × 10 million ≈ 2 million) while the churn happens in a tiny subset.
measured 43 days ago
What I have built in this area
Reaching production data without handing out the password
A self-hosted portal that puts ad-hoc production SQL behind approval, masking and an immutable trail; it became the QueryProxy product.
What it does today
Runs a developer's SQL against production through an approval step rather than directly; results are masked as they are written to disk and every request lands in an immutable record. Teams where production access sits with one person can run it today.
Personal and business finance on one data model
A finance core that keeps multi-account income and expense tracking in a single model; it became Parantaj, running on web, iOS and Android.
What it does today
One account/transaction model carries multi-account management, budgeting, goals and reporting on a single core; the web, iOS and Android clients are all live. Anyone who wants personal and business finances tracked in one place can use it.
A swipe-based true/false trivia game; the real subject is not the game but making a client-computed score verifiable through a signature.
What it does today
The swipe-based true/false quiz runs end to end in its core loop as a mobile app; the score is produced on the client but verified server-side with a server_nonce and an HMAC signature. swipenor.com is the app's landing page, not the game itself.
What shipped in this area
QueryProxy
OngoingFounder & Developer
Tier: Main FocusQueryProxy is a self-hosted query approval portal I built so developers can reach production data without ever holding production credentials. Submitted SQL is parsed and guarded, runs on a queue once a DBA approves it, is masked as the results are written to disk, and every step lands in an immutable audit record.
Parantaj
OngoingOwner
Tier: Active DevelopmentA personal and corporate financial management platform. It offers income-expense tracking, budget planning, detailed reporting, and multi-account management.
academia.sh
OngoingFounder & Author
Tier: Active Developmentacademia.sh is a learning platform with free, no-signup lessons that never separates a lesson's theory from a real problem from the field. It's presented through a curriculum-course-topic-lesson hierarchy, progress and quizzes are measured per lesson without splitting theory from problem, and full-text search runs on Postgres; the course-end exam, certificates and study assistant sit in a Pro tier. Content lives in a separate GitHub repo; the first field is computer science, and the model was designed multidisciplinary from the start.
Questions in this area
24 questions answered on this axis.
My Go worker in a container never receives SIGTERM and won't shut down gracefully—is this a PID 1 problem?
Flip `CMD` to exec form and listen with `signal.NotifyContext(..., syscall.SIGTERM)` instead of `os.Interrupt`; add tini only if the worker forks.
Should I move my API error bodies to the RFC 7807 problem+json format?
Standardize on RFC 9457 rather than 7807, produce every error body in one central exception handler, and manage the `type` URI as a versionable contract.
When should I switch to a covering index with INCLUDE to get an index-only scan?
Add `INCLUDE` only when the query is hot and `EXPLAIN (ANALYZE, BUFFERS)` shows heap access dominating, then tighten autovacuum until `Heap Fetches: 0`.
Technologies I pair it with
-
RabbitMQ
Asynchronous delivery between services; the exit door of the outbox relay.
-
Redis
Cache, counters and short-lived locks.
-
PostgreSQL
The database I measured queue-table and index behaviour on.
-
MySQL
The relational store I have worked with longest.
-
Elasticsearch
Where relational stops being enough: full-text search and log analysis.
-
Docker
Development environment and deployment unit.
-
nginx
The layer in front of the application.
-
GitHub Actions
Test and deploy pipeline; the steps that SSH into the server live here.
Frequently asked
7 questions
-
How long have you worked on the backend?
Since 2010, so 16 years. Two years after I started with PHP the weight of the work moved from the contract to the queue and stayed there; that period is where the 29 posts on this site came from.
-
What do people usually come to you with on the backend?
Most often it is "we moved to a queue, but some jobs vanish." The answer is nearly always in the same place: writing to the database and publishing to the queue are two separate systems with no atomicity between them.
-
Where do you start when designing an API?
With the contract. If the response shape, the error contract and the versioning decision are not settled before code is written, every client team writes its own assumption instead. A good share of the 29 posts on this site circle those three headings.
-
Does using a queue guarantee delivery?
No. A transactional outbox stops messages from being lost, but what it gives you is at-least-once — the message can repeat. What completes the guarantee is not the queue; it is an idempotent consumer.
-
How do you make index decisions?
By looking at how the query actually behaves instead of what sounds fast. Choosing a partial index purely because it is "small and fast" is a trap: if the database switches a prepared statement to a generic plan, the partial index may stop being usable. In my experience this kind of problem stays invisible until the table grows, so I check the plan on realistic data before I commit to an index.
-
Do you set up the server side too?
Yes, at application level. I set up the environment with Docker, the deploy pipeline with GitHub Actions and nginx in front. Infrastructure as an architectural subject is written on sade.dev.
-
How many projects run on this stack?
10 of the projects recorded on this site sit directly on this queue-and-database stack.
Let us work together
If you have work in this ecosystem, tell me what you are building and we will talk about how to build it.