From 10c4edd89f0c80b25eecb0d1ee8cf3cab7212be0 Mon Sep 17 00:00:00 2001 From: Willy Tarreau Date: Thu, 18 Sep 2025 08:38:34 +0200 Subject: [PATCH] OPTIM: sink: reduce contention on sink_announce_dropped() perf top shows that sink_announce_dropped() consumes most of the CPU on a 128-thread x86 system. Digging further reveals that the atomic fetch_or() on the dropped field used to detect the presence of another thread is entirely responsible for this. Indeed, the compiler implements it using a CAS that loops without relaxing and makes all threads wait until they can synchronize on this one, only to discover later that another thread is there and they need to give up. Let's just replace this with a hand-crafted CAS loop that will detect *before* attempting the CAS if another thread is there. Doing so achieves the same goal without forcing threads to agree. With this simple change, the sustained request rate on h1 with all traces on bumped from 110k/s to 244k/s! This should be backported to stable releases where it's often needed to help debugging. (cherry picked from commit 4431e3bd26f5e7af6e229d1d06bbc2749c2272c0) Signed-off-by: Christopher Faulet (cherry picked from commit e7c281fc6e11e774d0f4010758f59f6016c406d7) Signed-off-by: Christopher Faulet (cherry picked from commit a61a2d3c5a16dcc9e02e03a2076e852ec03bbbea) Signed-off-by: Christopher Faulet --- src/sink.c | 12 +++++++----- 1 file changed, 7 insertions(+), 5 deletions(-) diff --git a/src/sink.c b/src/sink.c index 1f3dbbd..be942ad 100644 --- a/src/sink.c +++ b/src/sink.c @@ -232,11 +232,13 @@ int sink_announce_dropped(struct sink *sink, struct log_header hdr) * another thread is already on them and we can just pass and * count another drop (hence add 2). */ - dropped = HA_ATOMIC_FETCH_OR(&sink->ctx.dropped, 1); - if (dropped & 1) { - /* another thread was already on it */ - goto leave; - } + dropped = HA_ATOMIC_LOAD(&sink->ctx.dropped); + do { + if (dropped & 1) { + /* another thread was already on it */ + goto leave; + } + } while (!_HA_ATOMIC_CAS(&sink->ctx.dropped, &dropped, dropped | 1)); last_dropped = 0; dropped >>= 1; -- 1.7.10.4