BUG/MEDIUM: pollers: clear the sleeping bit after waking up, not before

A bug was introduced in 2.1-dev2 by commit 305d5ab46 ("MAJOR: fd: Get
rid of the fd cache."). Pollers "poll" and "evport" had the sleeping
bit accidentally removed before the syscall instead of after. This
results in them not being woken up by inter-thread wakeups, which is
particularly visible with the multi-queue accept() and with queues.

As a work-around, when these pollers are used, "nbthread 1" should
be used.

The fact that it has remained broken for 2 years is a great indication
that threads are definitely not enabled outside of epoll and kqueue,
hence why this patch is only tagged medium.

This must be backported as far as 2.2.

(cherry picked from commit c37ccd70b4f622a4148cb63d3b584357b02cda47)
Signed-off-by: Willy Tarreau <w@1wt.eu>
(cherry picked from commit e148cb042ef64410c80b4d9e62165172c935bba3)
Signed-off-by: Willy Tarreau <w@1wt.eu>

diff --git a/src/ev_evports.c b/src/ev_evports.c
index 2ad546f..2dd0489 100644 (file)
--- a/src/ev_evports.c
+++ b/src/ev_evports.c
@@ -152,8 +152,6 @@ static void _do_poll(struct poller *p, int exp, int wake)
        }
 
        thread_harmless_now();
-       if (sleeping_thread_mask & tid_bit)
-               _HA_ATOMIC_AND(&sleeping_thread_mask, ~tid_bit);
 
        /*
         * Determine how long to wait for events to materialise on the port.
@@ -206,6 +204,8 @@ static void _do_poll(struct poller *p, int exp, int wake)
        tv_leaving_poll(wait_time, nevlist);
 
        thread_harmless_end();
+       if (sleeping_thread_mask & tid_bit)
+               _HA_ATOMIC_AND(&sleeping_thread_mask, ~tid_bit);
 
        if (nevlist > 0)
                activity[tid].poll_io++;

diff --git a/src/ev_poll.c b/src/ev_poll.c
index b0b0e2b..9f3d992 100644 (file)
--- a/src/ev_poll.c
+++ b/src/ev_poll.c
@@ -165,8 +165,6 @@ static void _do_poll(struct poller *p, int exp, int wake)
        } while (!_HA_ATOMIC_CAS(&maxfd, &old_maxfd, new_maxfd));
 
        thread_harmless_now();
-       if (sleeping_thread_mask & tid_bit)
-               _HA_ATOMIC_AND(&sleeping_thread_mask, ~tid_bit);
 
        fd_nbupdt = 0;
 
@@ -210,6 +208,8 @@ static void _do_poll(struct poller *p, int exp, int wake)
        tv_leaving_poll(wait_time, status);
 
        thread_harmless_end();
+       if (sleeping_thread_mask & tid_bit)
+               _HA_ATOMIC_AND(&sleeping_thread_mask, ~tid_bit);
 
        if (status > 0)
                activity[tid].poll_io++;