64 goroutines on four cores: Mutex is 1.40 times faster than a one-way channel, but its p99 is four times higher
When G goroutines on P cores hammer the same shared counter, how many operations a second do sync.Mutex, an owner-goroutine channel and a buffered channel manage, and how do tail latency and fairness of waiting change?
Finding
With 64 goroutines on four cores, sync.Mutex is 1.40 times faster than a one-way channel (10.46 and 7.47 million operations/s) but its p99 wait is 4.00 times higher (294,911 and 73,727 ns). On eight cores the Mutex ranges from 16.27 to 26.25 million operations/s across the two runs (a bimodal cell); the one-way channel holds at 7.31 and 7.32 million. More than half of the cells are flagged in both runs; confidence is low.
- G=64 P=4 · operations/s, Mutex → one-way channel (run A)
- 10.46 → 7.47 million (1.40x)
- G=64 P=4 · p99 wait, Mutex → one-way channel
- 294,911 → 73,727 ns (4.00x)
- G=64 P=8 · Mutex (run A / B, bimodal)
- 16.27 / 26.25 million operations/s
- G=64 P=8 · one-way channel (run A / B)
- 7.31 / 7.32 million operations/s
- Cells carrying a quality flag (run A / B)
- 433 / 845 · 440 / 845
Method
go-sync-bench measured three patterns: S1 single counter (sync.Mutex, a one-way channel to an owner goroutine, a request-reply channel, atomic), S2 read ratio (Mutex, RWMutex, owner-goroutine channel), S3 producer-consumer queue (channel buffers of 0, 1, 64, 1024 and a queue built on sync.Cond). Each cell, per G goroutines and P core group (1, 2, 4 and 8 via cpuset), ran 5 rounds; each repetition had a 0.5 s warm-up, a 1.5 s throughput window and a 2.0 s latency window. 845 cells, 4,225 valid repetitions per run. Two independent runs were made: A on 2026-10-18 and B on 2026-10-19 (different day, rebooted machine); each is the other's second run. After every round × core group container came a 60 s cooldown, in a network-less, read-only Docker container with 4 GiB of memory. Cell medians were taken from all 5 repetitions; no repetition was discarded. Flag thresholds are 10% for throughput, 25% for p99 and 5% for clock drift, over the full range. After the first pass of each run, cells still over the threshold were rerun once (A 75, B 73 cells) and replaced as a whole. Second-run tolerance: ±5% for throughput; for p99 ±5% or ±1 histogram bucket, whichever is wider; cells outside it were counted as inconsistent. Flagged cells are 433 in A and 440 in B out of 845 (51.2% and 52.1%); since this exceeds 15%, confidence is low. Latency percentiles are upper bounds of log buckets and are sampled at 1/16.
- Measured on
- Published
- Updated
measured 30 days ago
Environment
- Go
- 1.27.1 · x86_64
- Hardware
- AMD Ryzen 9 5950X · 16 CPU · SMT off · bare metal (no virtualization)
- Operating system
- Ubuntu 26.04.1 · kernel 7.0.0-38-generic · Docker 29.9.1
- Controls
- performance governor · boost off · C2 idle state off · isolcpus=managed_irq,8-15 · nohz_full=8-15 · IRQ affinity 0-7
- Core groups
- P1 = 8 · P2 = 8-9 · P4 = 8-11 · P8 = 8-15 (cpuset, isolated cores); system 0-7
- Container
- network-less · read-only · 4 GiB memory
- Image
- A: first pass sha256:108044bfad86… · rerun sha256:3dc28e004696… · B: first pass sha256:575e4c8ad5de… · rerun sha256:d324581ae7e7…
- Protocol
- 5 rounds · 845 cells · 0.5 s warm-up + 1.5 s throughput + 2.0 s latency · 60 s cooldown (per container)
- Runs
- A: 2026-10-18, seed 4096595856 · B: 2026-10-19, seed 640996940
Technologies
To reproduce
./bench/verify.sh && STAMP=$(date -u +%F) ./bench/run.sh --all When I pointed 64 goroutines at the same counter on four cores, sync.Mutex
was 1.40 times faster than a one-way channel: 10.46 million operations a
second against 7.47 million. In the same cell the p99 wait was 294,911 ns
against 73,727 ns. That pair is where I saw that “which one is faster” has no
single answer; this record writes down what that answer depends on, as far as
I measured it. All the code and the raw runs are in the
go-sync-bench repository.
For a practical walk-through of goroutines and channels, see Concurrency in Go: goroutines and channels in practice. Here there is only measurement.
Remeasured (2026-10-20)
I published this record on 2026-09-30 from a laptop, an Apple M4 Pro, inside a Docker virtual machine and as a single run. I have now measured the same question, with the same method and the same 845 cells, in a controlled environment: an AMD Ryzen 9 5950X running Ubuntu 26.04.1 directly, no virtualization, SMT off, the measurement cores (8-15) separated from system work. Because the environment is controlled I made two independent runs: A on 2026-10-18 and B on 2026-10-19, on different days and on a rebooted machine. The numbers on this page come from those two runs.
The earlier measurement was not deleted; it stays in the bench repository under
results/2026-09-29/.
The new runs are under
results/2026-10-18/
and
results/2026-10-19/.
Remeasuring changed these lines of this page (before → after):
- G=64, P=4. “Mutex and channel run at the same speed” (7.69 and 7.94 million operations/s) → the Mutex is 1.40 times faster than a one-way channel (10.46 and 7.47 million). The p99 gap of 5.00 times (491,519 and 98,303 ns) → 4.00 times (294,911 and 73,727 ns).
- G=64, P=8. A single Mutex number (15.96 million) and ratios of 2.4 and 4.0 over the channels → 16.27 and 26.25 million in the two runs; the cell is bimodal, so I do not write a single ratio. The channel cells agree across the two runs.
- p99 at P=8. “Close across all three models” → raw values: Mutex 98,303 and 90,111, one-way channel 73,727, request-reply channel 81,919 ns; the Mutex cell is inconsistent, so I write no ratio and I no longer say “close”.
- G=10,000, P=8. The Mutex’s ratio over the channels of 7.25 and 8.36 → roughly 3.0 to 3.6. The Mutex’s Jain value 0.8237 → 0.0996 and 0.1436.
- Noise floor. “The measurement setup does not distort fairness on its
own” → in the
atomiccells at G=4, Jain is 0.94 and 0.98 at P=4 and P=8; the sentence holds only for P=2. - Flagged cells. 518 / 845 (61.3%) → 433 / 845 (51.2%) and 440 / 845
(52.1%).
confidenceislowin both measurements. Thecalib_driftflag never appeared in the new runs.
I do not state the cause of these differences. Architecture (aarch64 and x86_64), a Docker virtual machine versus bare metal, kernel, core isolation and core count all changed together; I cannot separate how much each explains with this data. The old measurement was also a single run, so I do not know its run-to-run noise.
What I measured
The main pattern (S1) is this: G goroutines increment a single shared counter. There are four ways to reach the counter.
| Model | Access style |
|---|---|
| Mutex | Counter protected by sync.Mutex |
| One-way channel | A single owner goroutine holds the counter; the others send requests to the channel and do not wait for a reply |
| Request-reply channel | The same owner goroutine, but each request carries a reply channel and waits for the answer |
| atomic | Atomic increment; for context only, not part of the headline claim |
Two more patterns ran: in S2 the read ratio varies (50, 90 and 99 percent
reads), and in S3 a producer-consumer queue is tried with different buffers
(channel 0, 1, 64, 1024 and a queue built on sync.Cond). The core count P is
the size of the core group given to the container via cpuset: 1, 2, 4 and 8.
In this record “operations/s” is the median of a throughput window, and “wait” is the time it takes an operation to reach the counter during the latency window. Latency percentiles are reported as upper bounds of log buckets; so numbers like 294,911 are the ceiling of a bucket, not an exact measured value.
64 goroutines, four cores: the Mutex is fast, its queue is long
| Model | Operations/s (A / B) | p50 | p99 | p99.9 |
|---|---|---|---|---|
| Mutex | 10.46 M / 10.56 M | 287 ns | 294,911 ns | 655,359 ns |
| One-way channel | 7.47 M / 7.48 M | 191 ns | 73,727 ns | 81,919 ns |
On throughput the Mutex leads by 40 percent; both runs show the same direction and the same size. The wait goes the other way: 287 against 191 at p50, 294,911 against 73,727 at p99, 655,359 against 81,919 ns at p99.9. The Mutex does more work but its queue is wider. A benchmark that looked only at throughput would not have shown this difference.
I did not measure why it is this way. I could build an explanation of how the Mutex wakes its waiting goroutines, but it would not be evidence this experiment produced; I am leaving it out of the text.
On eight cores the table turned
G=64: throughput on four and eight cores
The Mutex's P=8 cell is flagged, bimodal and outside the ±5% tolerance (inconsistent) in both runs: 16.27 million in A, 26.25 million in B. The one-way channel barely changed from P=4 to P=8 in either run. The request-reply channel's P=4 cells are not in the chart; its P=8 values are in the table below.
- Mutex (A)
- Mutex (B)
- One-way channel (A)
- One-way channel (B)
M Source: go-sync-bench, results/2026-10-18/merged/summary.json and results/2026-10-19/merged/summary.json, S1 cells
Data table
| Series | P=4 | P=8 |
|---|---|---|
| Mutex (A) | 10.46 M | 16.27 M |
| Mutex (B) | 10.56 M | 26.25 M |
| One-way channel (A) | 7.47 M | 7.31 M |
| One-way channel (B) | 7.48 M | 7.32 M |
The chart is drawn in the browser; the table below carries the same data.
| Model | Operations/s (A / B) | p99 (A / B) | Flag |
|---|---|---|---|
| Mutex | 16.27 M / 26.25 M | 98,303 ns / 90,111 ns | A: spread_throughput, spread_p99 · B: spread_throughput, spread_p99, spread_persistent |
| One-way channel | 7.31 M / 7.32 M | 73,727 ns / 73,727 ns | none |
| Request-reply channel | 4.23 M / 4.25 M | 81,919 ns / 81,919 ns | none |
The Mutex’s cell here counts as bimodal and its repetitions spread over a wide range: the lowest repetition is about 16 million operations/s in both runs (16.10 and 16.31), the highest 22.13 in A and 28.01 in B. That is why the medians of the two runs came out as 16.27 and 26.25. I do not know the cause. In diagnostic runs I ruled out core isolation, the C2 idle state and starvation; no explanation was left. I do not call the gap noise or an architecture effect. The channel cells agree across the two runs to within 1 percent: 7.31 and 7.32, 4.23 and 4.25 million.
From P=4 to P=8 the Mutex went from 10.46 to 16.27 million in A and from 10.56 to 26.25 million in B; because the P=8 cell lies outside the ±5% tolerance in the two runs, I count it as inconsistent and do not write a speed-up factor. The one-way channel fell to 0.98 of its earlier value in both runs: I see that adding cores gave it nothing, and I do not know why.
The p99 at P=8: Mutex 98,303 and 90,111, one-way channel 73,727, request-reply channel 81,919 ns. Because the Mutex cell is inconsistent and flagged, I draw no ratio from these p99 values and no conclusion that compares them with P=4; I give only the raw values.
What I have to leave out of scope: the Mutex cell at P=1 (119.56 and 129.63
million) carries an observer_effect flag in A and observer_effect plus
spread_throughput in B, and the one at P=2 (23.60 and 28.49 million) has a spread of 106 and 76 percent and a spread_persistent
flag. I did not take either into the claim. I also can’t say whether the Mutex
curve is monotonic in P, because the P=1 and P=2 cells are flagged.
10,000 goroutines: the Mutex is fast, its unfairness is unexplained
| Model | Operations/s (A / B) | Jain (A / B) | p99 (A / B) | Flag |
|---|---|---|---|---|
| Mutex | 16.81 M / 16.43 M | 0.0996 / 0.1436 | 16,777,215 ns / 18,874,367 ns | A: fairness_unexplained, spread_throughput, spread_p99 · B: fairness_unexplained, spread_throughput, spread_persistent |
| One-way channel | 5.42 M / 5.45 M | 1 / 0.9992 | 2,359,295 ns / 2,359,295 ns | none |
| Request-reply channel | 4.73 M / 4.74 M | 1 / 1 | 2,621,439 ns / 2,621,439 ns | none |
The Mutex is roughly 3.0 to 3.6 times faster than the channels here; but the
cell is flagged, so this is a direction, not an exact ratio. The Jain fairness
index is 0.0996 and 0.1436, and that value is marked: fairness_unexplained.
The harness can tie unfairness to only one of two sources: park behavior
(sync). The rule is: if G ≤ P, if the per-goroutine quantum limit b ≥ 1, or
if at least half of the sampled acquisitions exceed the park threshold, the
source is sync; otherwise unexplained. In this cell G=10,000 > P=8, b =
0.12 and the slow-acquire share is 0.0191 (A) and 0.0201 (B); none of the three
conditions holds, so the unfairness could not be tied to park behavior. The
rule does not look at the Jain value. The expected Jain the harness computes
from b is 0.12 (expected_quantum_jain); the measured values are close to it.
But I have not verified the quantum model that calculation rests on, so I do
not attribute this to the scheduler. I cannot assign the Mutex’s unfairness to
a cause. By my design rule these cells stay out of cross-model fairness
comparisons; I do not say “the Mutex is more unfair”, I say “its unfairness is
unexplained”.
For context, at four cores the Mutex’s Jain value is 0.9847 and 0.9839 at
G=1,024 and 0.6595 and 0.6833 at G=10,000; both the G=1,024 and the
G=10,000 cells are fairness_unexplained. The one-way channel’s Jain at G=10,000 is 0.9826 and
0.9492 at P=4 and 1 and 0.9992 at P=8. The differences are small; I cannot say
with this data that the channels’ fairness depends on P.
Read ratio and queue buffer
In S2 the Mutex speeds up as the read ratio rises; the owner-goroutine channel barely moves.
S2, G=64, P=4: throughput by read ratio
All the Mutex cells are flagged in both runs (spread_throughput or observer_effect), so the values show direction. The 50% reads Mutex cell (10.09 and 10.99) is outside the ±5% tolerance and inconsistent. The channel cells are unflagged. The RWMutex cells carry an observer_effect flag and are not in the chart.
- Mutex (A)
- Mutex (B)
- Owner-goroutine channel (A)
- Owner-goroutine channel (B)
M Source: go-sync-bench, results/2026-10-18/merged/summary.json and results/2026-10-19/merged/summary.json, S2 cells
Data table
| Series | 50% reads | 90% reads | 99% reads |
|---|---|---|---|
| Mutex (A) | 10.09 M | 14.48 M | 17.63 M |
| Mutex (B) | 10.99 M | 14.44 M | 17.85 M |
| Owner-goroutine channel (A) | 4.02 M | 4.17 M | 4.23 M |
| Owner-goroutine channel (B) | 4.04 M | 4.16 M | 4.23 M |
The chart is drawn in the browser; the table below carries the same data.
At 50 percent reads the Mutex is at 10.09 and 10.99 million and the channel at 4.02 and 4.04; the two Mutex values lie outside the ±5% tolerance, so I count this cell as inconsistent and write no ratio that rests on it (neither the ratio over the channel nor the 99-to-50-percent ratio). The Mutex’s ratio over the channel is 3.47 in both runs at 90 percent reads and 4.17 in A and 4.22 in B at 99 percent. These two Mutex cells are inside the tolerance but flagged; the ratios are not exact, they are numbers that show the same direction in both runs. From 90 to 99 percent reads the Mutex gets faster in both runs: from 14.48 and 14.44 to 17.63 and 17.85.
In S3 at G=64, P=4: the unbuffered channel is at 5.58 and 5.59, the channel
with a buffer of one at 6.84 and 6.86, and the 1,024-slot queue built on
sync.Cond at 4.71 and 4.49 million operations/s. With a buffer of one the
channel is 1.23 times faster in both runs, and 1.45 and 1.53 times faster than
the sync.Cond queue. But in each run at least one of these three cells is
flagged: the unbuffered channel in A, the buffer-of-one channel in B, the
sync.Cond queue in both. These ratios also show direction only. The channels
with buffers of 64 and 1,024 and the sync.Cond queue with 64 slots are
flagged in both runs; the channels’ numbers around 18 to 21 million (18.22 and 18.12 at buffer 64,
20.52 and 20.70 at buffer 1,024) show direction only, and I did not derive
ratios from them.
Result
How I measured
The experiment ran 845 cells; each cell 5 rounds, each round with a 0.5 s warm-up, a 1.5 s throughput window and a 2.0 s latency window per repetition. That is 4,225 valid repetitions per run. The median repetition time was 4.37 s in both runs; 14 repetitions in A and 5 in B exceeded the time budget. The repetitions that hit the 64 s timeout were always in P=2 and G=10,000 cells (3 in A, 1 in B) and passed on retry; I did not investigate why. After every round × core group container there is a 60 s cooldown; there is no waiting between cells inside a container.
The container is network-less and read-only, with 4 GiB of memory. The
measurement cores (8-15) are separated with isolcpus and nohz_full, IRQs
are on 0-7, frequency is fixed with the performance governor, boost is off,
SMT is off and the C2 idle state is off.
The two runs, A (sha256:108044bfad86…, seed 4096595856) and B
(sha256:575e4c8ad5de…, seed 640996940), ran the same main cell set; since
the seed differs, the cell order does too. In each run, cells still over the
threshold after the first pass were rerun once (75 in A, 73 in B) and replaced
as a whole with the rerun’s repetitions; cells still over after the rerun are
44 in A and 39 in B (spread_persistent). The cell median is still taken from
all 5 repetitions. Flagging is a separate criterion: 10% for throughput, 25%
for p99 and 5% for clock drift, over the full range.
Quality flags:
| Flag | Cell count (A / B) | Meaning |
|---|---|---|
| observer_effect | 186 / 180 | Operation rate in the latency window is below 0.9 times that of the throughput window; the measurement affects itself |
| fairness_unexplained | 234 / 234 | Unfairness cannot be tied to park behavior: G > P, b < 1 and fewer than half of the sampled acquisitions are above the park threshold; the Jain value is not consulted |
| spread_p99 | 136 / 134 | Between-repetition p99 spread above 0.25 |
| spread_throughput | 105 / 95 | Between-repetition throughput spread above 0.10 |
| spread_persistent | 44 / 39 | Spread persists after the rerun |
Second-run comparison (A against B). For the eight headline cells, the
tolerance on throughput and p99 numbers is ±5% for throughput and ±5% or ±1
histogram bucket for p99, whichever is wider. Under this tolerance only
s1-mutex-g64-p8-w0 falls outside (+61.32%); the other seven cells are inside.
Across all 845 cells there are 38 cells outside ±5% on throughput: 31 from the
Mutex, RWMutex and sync.Cond family and 7 S3 channel cells; in S1 none of the
atomic, one-way and request-reply channel cells fall outside. On p99 there
are 201 cells outside ±5%; many of them are only one bucket step away and count
as inside under the ±1 bucket tolerance. I count the cells that are outside the
tolerance as inconsistent: in the record they appear only as the two runs’
values (a range), and they support no ratio, speed-up factor or result
sentence. These cells are s1-mutex-g64-p8-w0 (16.27 to 26.25 million) and
s2-mutex-r50-g64-p4-w0 (10.09 to 10.99 million). The Mutex cells at P=1 and
P=2 are also outside ±5%; I left them out of scope above.
Noise floor: the atomic cell at G=2 and P=2 has a Jain of 0.9999 (in both
runs). At G=4 and P=4 it is 0.9383 and 0.9378, at G=4 and P=8 0.9795 and
0.9677; min_max is 0.51 at P=4 and 0.66 and 0.59 at P=8. So the measurement
setup does not distort fairness at P=2, but at P=4 and P=8 I see a floor that
is not perfectly fair; I do not attribute it to a cause. The self-test, 40
trials, passed 40 of 40 in both runs. The reproduction command is in the
reproduce field; the raw data and the license (MIT) are in the repository.
Limits and honesty notes
- More than half of the cells are flagged. 433 of 845 cells in A and 440
in B are flagged. At G=64, P=4 the two channel cells are unflagged in both
runs; the Mutex cell is unflagged in A and carries
spread_throughputin B. The Mutex cell at P=8 and the one at G=10,000 are flagged in both runs. Where flagged cells are cited to show direction, they are labeled that way. - Bimodal cell.
s1-mutex-g64-p8-w0counts as bimodal and spreads over a wide range from repetition to repetition; I do not call the G=10,000 cell bimodal. I do not know the cause; isolation, C2 and starvation were ruled out in diagnostic runs. - Confidence is low, not
high. Because the flagged-cell share exceeds 15% (51.2% and 52.1%), the rubric giveslow; the second-run comparison also failed the ±5% condition. - One machine. Both runs ran on the same physical machine, on different days and after a reboot. I do not know whether other hardware would give the same numbers.
- Comparison with the earlier measurement. The old record was a single run measured on Apple silicon inside a Docker virtual machine. The difference between the old and new numbers is a mix of architecture, virtual machine, kernel and isolation differences; I do not attribute it to a single cause.
- The fairness flag’s rule. The flag looks at G, P, b and park share; it
cannot tell scheduler-caused unfairness apart. I have not verified the
quantum model (
expected_quantum_jain) and did not use it in explanations. - Latency resolution. Latency percentiles are upper bounds of log buckets
and are sampled at 1/16; a one-bucket p99 difference is the instrument’s
resolution. I did not use the
starving_fractionfield because the sample count is small. - The request channel is unbuffered. A buffered request channel could change the channel models’ result.
- Two goroutines at G=1. The owner-goroutine models run 2 goroutines at G=1.
- Out of scope.
sync.Mapwas not measured at all; atomic was not measured in S2 either. A synthetic counter was measured, not a production workload.
To produce your own numbers, run go-sync-bench on your own hardware; choose P for your own target, because the clearest result of this record is that the answer depends on P.
Related posts
Concurrency in Go: goroutines and channels in practice
A hands-on look at Go's concurrency model — goroutines and channels — and what changes when you come from PHP's process model.
18,750 req/s: precise, repeated, and three times wrong
Two phases of the same run measured the same service three times apart. What caught the wrong one was not a better statistic but a second method.