This is one of only two files that initialize a semaphore to a negative
value. We don't really need the two semaphores here at all, but can do
the same thing in more conventional and more effient way, by using a
single waitqueue and an atomic thread counter.
This gets us a little bit closer to eliminating classic semaphores from
the kernel. It also fixes a corner case where we fail to continue after
one of the threads fails to start up.
An alternative would be to use a split kthread_create()+wake_up_process()
and completely eliminate the separate synchronization.
Signed-off-by: Arnd Bergmann <[email protected]>
---
This is part of a longer, untested, series to remove semaphores
from the kernel, please review as such before applying.
---
lib/test_rhashtable.c | 28 ++++++++++++++++------------
1 file changed, 16 insertions(+), 12 deletions(-)
diff --git a/lib/test_rhashtable.c b/lib/test_rhashtable.c
index 18de5ff1255b..12bdea4f6c20 100644
--- a/lib/test_rhashtable.c
+++ b/lib/test_rhashtable.c
@@ -20,11 +20,11 @@
#include <linux/module.h>
#include <linux/rcupdate.h>
#include <linux/rhashtable.h>
-#include <linux/semaphore.h>
#include <linux/slab.h>
#include <linux/sched.h>
#include <linux/random.h>
#include <linux/vmalloc.h>
+#include <linux/wait.h>
#define MAX_ENTRIES 1000000
#define TEST_INSERT_FAIL INT_MAX
@@ -112,7 +112,8 @@ static struct rhashtable_params test_rht_params_dup = {
.automatic_shrinking = false,
};
-static struct semaphore prestart_sem, startup_sem;
+static atomic_t startup_count;
+static DECLARE_WAIT_QUEUE_HEAD(startup_wait);
static int insert_retry(struct rhashtable *ht, struct test_obj *obj,
const struct rhashtable_params params)
@@ -635,8 +636,9 @@ static int threadfunc(void *data)
int i, step, err = 0, insert_retries = 0;
struct thread_data *tdata = data;
- up(&prestart_sem);
- if (down_interruptible(&startup_sem))
+ if (atomic_dec_and_test(&startup_count))
+ wake_up(&startup_wait);
+ if (wait_event_interruptible(startup_wait, atomic_read(&startup_count) == -1))
pr_err(" thread[%d]: down_interruptible failed\n", tdata->id);
for (i = 0; i < tdata->entries; i++) {
@@ -756,8 +758,7 @@ static int __init test_rht_init(void)
pr_info("Testing concurrent rhashtable access from %d threads\n",
tcount);
- sema_init(&prestart_sem, 1 - tcount);
- sema_init(&startup_sem, 0);
+ atomic_set(&startup_count, tcount);
tdata = vzalloc(array_size(tcount, sizeof(struct thread_data)));
if (!tdata)
return -ENOMEM;
@@ -783,15 +784,18 @@ static int __init test_rht_init(void)
tdata[i].objs = objs + i * entries;
tdata[i].task = kthread_run(threadfunc, &tdata[i],
"rhashtable_thrad[%d]", i);
- if (IS_ERR(tdata[i].task))
+ if (IS_ERR(tdata[i].task)) {
pr_err(" kthread_run failed for thread %d\n", i);
- else
+ atomic_dec(&startup_count);
+ } else {
started_threads++;
+ }
}
- if (down_interruptible(&prestart_sem))
- pr_err(" down interruptible failed\n");
- for (i = 0; i < tcount; i++)
- up(&startup_sem);
+ if (wait_event_interruptible(startup_wait, atomic_read(&startup_count) == 0))
+ pr_err(" wait_event interruptible failed\n");
+ /* count is 0 now, set it to -1 and wake up all threads together */
+ atomic_dec(&startup_count);
+ wake_up_all(&startup_wait);
for (i = 0; i < tcount; i++) {
if (IS_ERR(tdata[i].task))
continue;
--
2.20.0
From: Arnd Bergmann <[email protected]>
Date: Mon, 10 Dec 2018 22:17:20 +0100
> This is one of only two files that initialize a semaphore to a negative
> value. We don't really need the two semaphores here at all, but can do
> the same thing in more conventional and more effient way, by using a
> single waitqueue and an atomic thread counter.
>
> This gets us a little bit closer to eliminating classic semaphores from
> the kernel. It also fixes a corner case where we fail to continue after
> one of the threads fails to start up.
>
> An alternative would be to use a split kthread_create()+wake_up_process()
> and completely eliminate the separate synchronization.
>
> Signed-off-by: Arnd Bergmann <[email protected]>
> ---
> This is part of a longer, untested, series to remove semaphores
> from the kernel, please review as such before applying.
This looks fine to me, although it seems kinda weird how the
synchronization works in that the N - 1 child threads will
be awoken two times, once when child N decrements the count
to zero and once when the parent decrements the count to -1
which lets us past the wait_event_interruptibel().
Nevertheless it should work just fine and I have no problems
with it.
Want me to apply this to net-next?
Thanks.
On Mon, Dec 10, 2018 at 10:17:20PM +0100, Arnd Bergmann wrote:
> This is one of only two files that initialize a semaphore to a negative
> value. We don't really need the two semaphores here at all, but can do
> the same thing in more conventional and more effient way, by using a
> single waitqueue and an atomic thread counter.
>
> This gets us a little bit closer to eliminating classic semaphores from
> the kernel. It also fixes a corner case where we fail to continue after
> one of the threads fails to start up.
>
> An alternative would be to use a split kthread_create()+wake_up_process()
> and completely eliminate the separate synchronization.
>
> Signed-off-by: Arnd Bergmann <[email protected]>
> ---
> This is part of a longer, untested, series to remove semaphores
> from the kernel, please review as such before applying.
> ---
> lib/test_rhashtable.c | 28 ++++++++++++++++------------
> 1 file changed, 16 insertions(+), 12 deletions(-)
This was created by Phil Sutter so I am adding him to the cc list.
> diff --git a/lib/test_rhashtable.c b/lib/test_rhashtable.c
> index 18de5ff1255b..12bdea4f6c20 100644
> --- a/lib/test_rhashtable.c
> +++ b/lib/test_rhashtable.c
> @@ -20,11 +20,11 @@
> #include <linux/module.h>
> #include <linux/rcupdate.h>
> #include <linux/rhashtable.h>
> -#include <linux/semaphore.h>
> #include <linux/slab.h>
> #include <linux/sched.h>
> #include <linux/random.h>
> #include <linux/vmalloc.h>
> +#include <linux/wait.h>
>
> #define MAX_ENTRIES 1000000
> #define TEST_INSERT_FAIL INT_MAX
> @@ -112,7 +112,8 @@ static struct rhashtable_params test_rht_params_dup = {
> .automatic_shrinking = false,
> };
>
> -static struct semaphore prestart_sem, startup_sem;
> +static atomic_t startup_count;
> +static DECLARE_WAIT_QUEUE_HEAD(startup_wait);
>
> static int insert_retry(struct rhashtable *ht, struct test_obj *obj,
> const struct rhashtable_params params)
> @@ -635,8 +636,9 @@ static int threadfunc(void *data)
> int i, step, err = 0, insert_retries = 0;
> struct thread_data *tdata = data;
>
> - up(&prestart_sem);
> - if (down_interruptible(&startup_sem))
> + if (atomic_dec_and_test(&startup_count))
> + wake_up(&startup_wait);
> + if (wait_event_interruptible(startup_wait, atomic_read(&startup_count) == -1))
> pr_err(" thread[%d]: down_interruptible failed\n", tdata->id);
>
> for (i = 0; i < tdata->entries; i++) {
> @@ -756,8 +758,7 @@ static int __init test_rht_init(void)
>
> pr_info("Testing concurrent rhashtable access from %d threads\n",
> tcount);
> - sema_init(&prestart_sem, 1 - tcount);
> - sema_init(&startup_sem, 0);
> + atomic_set(&startup_count, tcount);
> tdata = vzalloc(array_size(tcount, sizeof(struct thread_data)));
> if (!tdata)
> return -ENOMEM;
> @@ -783,15 +784,18 @@ static int __init test_rht_init(void)
> tdata[i].objs = objs + i * entries;
> tdata[i].task = kthread_run(threadfunc, &tdata[i],
> "rhashtable_thrad[%d]", i);
> - if (IS_ERR(tdata[i].task))
> + if (IS_ERR(tdata[i].task)) {
> pr_err(" kthread_run failed for thread %d\n", i);
> - else
> + atomic_dec(&startup_count);
> + } else {
> started_threads++;
> + }
> }
> - if (down_interruptible(&prestart_sem))
> - pr_err(" down interruptible failed\n");
> - for (i = 0; i < tcount; i++)
> - up(&startup_sem);
> + if (wait_event_interruptible(startup_wait, atomic_read(&startup_count) == 0))
> + pr_err(" wait_event interruptible failed\n");
> + /* count is 0 now, set it to -1 and wake up all threads together */
> + atomic_dec(&startup_count);
> + wake_up_all(&startup_wait);
> for (i = 0; i < tcount; i++) {
> if (IS_ERR(tdata[i].task))
> continue;
> --
> 2.20.0
>
--
Email: Herbert Xu <[email protected]>
Home Page: http://gondor.apana.org.au/~herbert/
PGP Key: http://gondor.apana.org.au/~herbert/pubkey.txt
Hi,
On Tue, Dec 11, 2018 at 01:45:52PM +0800, Herbert Xu wrote:
> On Mon, Dec 10, 2018 at 10:17:20PM +0100, Arnd Bergmann wrote:
> > This is one of only two files that initialize a semaphore to a negative
> > value. We don't really need the two semaphores here at all, but can do
> > the same thing in more conventional and more effient way, by using a
> > single waitqueue and an atomic thread counter.
> >
> > This gets us a little bit closer to eliminating classic semaphores from
> > the kernel. It also fixes a corner case where we fail to continue after
> > one of the threads fails to start up.
> >
> > An alternative would be to use a split kthread_create()+wake_up_process()
> > and completely eliminate the separate synchronization.
> >
> > Signed-off-by: Arnd Bergmann <[email protected]>
> > ---
> > This is part of a longer, untested, series to remove semaphores
> > from the kernel, please review as such before applying.
> > ---
> > lib/test_rhashtable.c | 28 ++++++++++++++++------------
> > 1 file changed, 16 insertions(+), 12 deletions(-)
>
> This was created by Phil Sutter so I am adding him to the cc list.
Thanks, I would have missed it otherwise. Just a minor nit:
> > diff --git a/lib/test_rhashtable.c b/lib/test_rhashtable.c
> > index 18de5ff1255b..12bdea4f6c20 100644
> > --- a/lib/test_rhashtable.c
> > +++ b/lib/test_rhashtable.c
[...]
> > @@ -635,8 +636,9 @@ static int threadfunc(void *data)
> > int i, step, err = 0, insert_retries = 0;
> > struct thread_data *tdata = data;
> >
> > - up(&prestart_sem);
> > - if (down_interruptible(&startup_sem))
> > + if (atomic_dec_and_test(&startup_count))
> > + wake_up(&startup_wait);
> > + if (wait_event_interruptible(startup_wait, atomic_read(&startup_count) == -1))
> > pr_err(" thread[%d]: down_interruptible failed\n", tdata->id);
The error message should probably be adjusted as well. Apart from that:
Acked-by: Phil Sutter <[email protected]>
Thanks, Phil
From: Arnd Bergmann <[email protected]>
Date: Mon, 10 Dec 2018 22:17:20 +0100
> @@ -635,8 +636,9 @@ static int threadfunc(void *data)
> int i, step, err = 0, insert_retries = 0;
> struct thread_data *tdata = data;
>
> - up(&prestart_sem);
> - if (down_interruptible(&startup_sem))
> + if (atomic_dec_and_test(&startup_count))
> + wake_up(&startup_wait);
> + if (wait_event_interruptible(startup_wait, atomic_read(&startup_count) == -1))
> pr_err(" thread[%d]: down_interruptible failed\n", tdata->id);
Arnd, please adjust this pr_err() text to match the new code as pointed out by
Phil Sutter.
Thank you.
On Fri, Dec 14, 2018 at 10:25 PM David Miller <[email protected]> wrote:
>
> From: Arnd Bergmann <[email protected]>
> Date: Mon, 10 Dec 2018 22:17:20 +0100
>
> > @@ -635,8 +636,9 @@ static int threadfunc(void *data)
> > int i, step, err = 0, insert_retries = 0;
> > struct thread_data *tdata = data;
> >
> > - up(&prestart_sem);
> > - if (down_interruptible(&startup_sem))
> > + if (atomic_dec_and_test(&startup_count))
> > + wake_up(&startup_wait);
> > + if (wait_event_interruptible(startup_wait, atomic_read(&startup_count) == -1))
> > pr_err(" thread[%d]: down_interruptible failed\n", tdata->id);
>
> Arnd, please adjust this pr_err() text to match the new code as pointed out by
> Phil Sutter.
Done, and resent as v2. I also noticed that the version I sent did not apply
cleanly on current kernels, so I fixed that, and also added a 'goto out' in
the kthread after printing the error message, so an interrupted thread would
exist straight away.
Arnd