If a writer could been woken up, the above branch
if (sem->count == 0)
break;
would have moved us to taking the sem. So, it's
not the time to wake a writer now, and only readers
are allowed now. Thus, 0 must be passed to __rwsem_do_wake().
Next, __rwsem_do_wake() wakes readers unconditionally.
But we mustn't do that if the sem is owned by writer
in the moment. Otherwise, writer and reader own the sem
the same time, which leads to memory corruption in
callers.
rwsem-xadd.c does not need that, as:
1)the similar check is made lockless there,
2)in __rwsem_mark_wake::try_reader_grant we test,
that sem is not owned by writer.
Fixes: 17fcbd590d0c "locking/rwsem: Fix down_write_killable() for CONFIG_RWSEM_GENERIC_SPINLOCK=y"
Signed-off-by: Kirill Tkhai <[email protected]>
CC: Niklas Cassel <[email protected]>
CC: Peter Zijlstra (Intel) <[email protected]>
CC: Ingo Molnar <[email protected]>
---
kernel/locking/rwsem-spinlock.c | 4 ++--
1 file changed, 2 insertions(+), 2 deletions(-)
diff --git a/kernel/locking/rwsem-spinlock.c b/kernel/locking/rwsem-spinlock.c
index c65f7989f850..20819df98125 100644
--- a/kernel/locking/rwsem-spinlock.c
+++ b/kernel/locking/rwsem-spinlock.c
@@ -231,8 +231,8 @@ int __sched __down_write_common(struct rw_semaphore *sem, int state)
out_nolock:
list_del(&waiter.list);
- if (!list_empty(&sem->wait_list))
- __rwsem_do_wake(sem, 1);
+ if (!list_empty(&sem->wait_list) && sem->count >= 0)
+ __rwsem_do_wake(sem, 0);
raw_spin_unlock_irqrestore(&sem->wait_lock, flags);
return -EINTR;
Good catch!
This should go to -stable as well.
Perhaps
if (!list_empty(&sem->wait_list) && sem->count > 0)
__rwsem_do_wake(sem, 0);
Rather than
if (!list_empty(&sem->wait_list) && sem->count >= 0)
__rwsem_do_wake(sem, 0);
Since we have the spinlock, and since we just checked
if sem->count == 0, we still know that it can't be 0.
Either way:
Acked-by: Niklas Cassel <[email protected]>
On 06/16/2017 03:44 PM, Kirill Tkhai wrote:
> If a writer could been woken up, the above branch
>
> if (sem->count == 0)
> break;
>
> would have moved us to taking the sem. So, it's
> not the time to wake a writer now, and only readers
> are allowed now. Thus, 0 must be passed to __rwsem_do_wake().
>
> Next, __rwsem_do_wake() wakes readers unconditionally.
> But we mustn't do that if the sem is owned by writer
> in the moment. Otherwise, writer and reader own the sem
> the same time, which leads to memory corruption in
> callers.
>
> rwsem-xadd.c does not need that, as:
> 1)the similar check is made lockless there,
> 2)in __rwsem_mark_wake::try_reader_grant we test,
> that sem is not owned by writer.
>
> Fixes: 17fcbd590d0c "locking/rwsem: Fix down_write_killable() for CONFIG_RWSEM_GENERIC_SPINLOCK=y"
> Signed-off-by: Kirill Tkhai <[email protected]>
> CC: Niklas Cassel <[email protected]>
> CC: Peter Zijlstra (Intel) <[email protected]>
> CC: Ingo Molnar <[email protected]>
> ---
> kernel/locking/rwsem-spinlock.c | 4 ++--
> 1 file changed, 2 insertions(+), 2 deletions(-)
>
> diff --git a/kernel/locking/rwsem-spinlock.c b/kernel/locking/rwsem-spinlock.c
> index c65f7989f850..20819df98125 100644
> --- a/kernel/locking/rwsem-spinlock.c
> +++ b/kernel/locking/rwsem-spinlock.c
> @@ -231,8 +231,8 @@ int __sched __down_write_common(struct rw_semaphore *sem, int state)
>
> out_nolock:
> list_del(&waiter.list);
> - if (!list_empty(&sem->wait_list))
> - __rwsem_do_wake(sem, 1);
> + if (!list_empty(&sem->wait_list) && sem->count >= 0)
> + __rwsem_do_wake(sem, 0);
> raw_spin_unlock_irqrestore(&sem->wait_lock, flags);
>
> return -EINTR;
>
On Wed, Jun 28, 2017 at 15:20, Niklas Cassel wrote:
> Good catch!
>
> This should go to -stable as well.
>
>
> Perhaps
>
> if (!list_empty(&sem->wait_list) && sem->count > 0)
> __rwsem_do_wake(sem, 0);
>
> Rather than
>
> if (!list_empty(&sem->wait_list) && sem->count >= 0)
> __rwsem_do_wake(sem, 0);
>
> Since we have the spinlock, and since we just checked
> if sem->count == 0, we still know that it can't be 0.
>
> Either way:
>
> Acked-by: Niklas Cassel <[email protected]>
>
[PATCH]rwsem-spinlock: Fix EINTR branch in __down_write_common()
If a writer could been woken up, the above branch
if (sem->count == 0)
break;
would have moved us to taking the sem. So, it's
not the time to wake a writer now, and only readers
are allowed now. Thus, 0 must be passed to __rwsem_do_wake().
Next, __rwsem_do_wake() wakes readers unconditionally.
But we mustn't do that if the sem is owned by writer
in the moment. Otherwise, writer and reader own the sem
the same time, which leads to memory corruption in
callers.
rwsem-xadd.c does not need that, as:
1)the similar check is made lockless there,
2)in __rwsem_mark_wake::try_reader_grant we test,
that sem is not owned by writer.
Fixes: 17fcbd590d0c "locking/rwsem: Fix down_write_killable() for CONFIG_RWSEM_GENERIC_SPINLOCK=y"
Signed-off-by: Kirill Tkhai <[email protected]>
Acked-by: Niklas Cassel <[email protected]>
CC: Peter Zijlstra (Intel) <[email protected]>
CC: Ingo Molnar <[email protected]>
---
kernel/locking/rwsem-spinlock.c | 4 ++--
1 file changed, 2 insertions(+), 2 deletions(-)
diff --git a/kernel/locking/rwsem-spinlock.c b/kernel/locking/rwsem-spinlock.c
index c65f7989f850..3e6feccb8b56 100644
--- a/kernel/locking/rwsem-spinlock.c
+++ b/kernel/locking/rwsem-spinlock.c
@@ -231,8 +231,8 @@ int __sched __down_write_common(struct rw_semaphore *sem, int state)
out_nolock:
list_del(&waiter.list);
- if (!list_empty(&sem->wait_list))
- __rwsem_do_wake(sem, 1);
+ if (!list_empty(&sem->wait_list) && sem->count > 0)
+ __rwsem_do_wake(sem, 0);
raw_spin_unlock_irqrestore(&sem->wait_lock, flags);
return -EINTR;
Commit-ID: a0c4acd2c220376b4e9690e75782d0c0afdaab9f
Gitweb: http://git.kernel.org/tip/a0c4acd2c220376b4e9690e75782d0c0afdaab9f
Author: Kirill Tkhai <[email protected]>
AuthorDate: Fri, 16 Jun 2017 16:44:34 +0300
Committer: Ingo Molnar <[email protected]>
CommitDate: Wed, 5 Jul 2017 12:26:29 +0200
locking/rwsem-spinlock: Fix EINTR branch in __down_write_common()
If a writer could been woken up, the above branch
if (sem->count == 0)
break;
would have moved us to taking the sem. So, it's
not the time to wake a writer now, and only readers
are allowed now. Thus, 0 must be passed to __rwsem_do_wake().
Next, __rwsem_do_wake() wakes readers unconditionally.
But we mustn't do that if the sem is owned by writer
in the moment. Otherwise, writer and reader own the sem
the same time, which leads to memory corruption in
callers.
rwsem-xadd.c does not need that, as:
1) the similar check is made lockless there,
2) in __rwsem_mark_wake::try_reader_grant we test,
that sem is not owned by writer.
Signed-off-by: Kirill Tkhai <[email protected]>
Acked-by: Peter Zijlstra <[email protected]>
Cc: <[email protected]>
Cc: Linus Torvalds <[email protected]>
Cc: Niklas Cassel <[email protected]>
Cc: Peter Zijlstra (Intel) <[email protected]>
Cc: Peter Zijlstra <[email protected]>
Cc: Thomas Gleixner <[email protected]>
Fixes: 17fcbd590d0c "locking/rwsem: Fix down_write_killable() for CONFIG_RWSEM_GENERIC_SPINLOCK=y"
Link: http://lkml.kernel.org/r/149762063282.19811.9129615532201147826.stgit@localhost.localdomain
Signed-off-by: Ingo Molnar <[email protected]>
---
kernel/locking/rwsem-spinlock.c | 4 ++--
1 file changed, 2 insertions(+), 2 deletions(-)
diff --git a/kernel/locking/rwsem-spinlock.c b/kernel/locking/rwsem-spinlock.c
index c65f798..20819df 100644
--- a/kernel/locking/rwsem-spinlock.c
+++ b/kernel/locking/rwsem-spinlock.c
@@ -231,8 +231,8 @@ int __sched __down_write_common(struct rw_semaphore *sem, int state)
out_nolock:
list_del(&waiter.list);
- if (!list_empty(&sem->wait_list))
- __rwsem_do_wake(sem, 1);
+ if (!list_empty(&sem->wait_list) && sem->count >= 0)
+ __rwsem_do_wake(sem, 0);
raw_spin_unlock_irqrestore(&sem->wait_lock, flags);
return -EINTR;
On 07/05/2017 04:27 PM, tip-bot for Kirill Tkhai wrote:
> Commit-ID: a0c4acd2c220376b4e9690e75782d0c0afdaab9f
> Gitweb: http://git.kernel.org/tip/a0c4acd2c220376b4e9690e75782d0c0afdaab9f
> Author: Kirill Tkhai <[email protected]>
> AuthorDate: Fri, 16 Jun 2017 16:44:34 +0300
> Committer: Ingo Molnar <[email protected]>
> CommitDate: Wed, 5 Jul 2017 12:26:29 +0200
>
> locking/rwsem-spinlock: Fix EINTR branch in __down_write_common()
>
> If a writer could been woken up, the above branch
>
> if (sem->count == 0)
> break;
>
> would have moved us to taking the sem. So, it's
> not the time to wake a writer now, and only readers
> are allowed now. Thus, 0 must be passed to __rwsem_do_wake().
>
> Next, __rwsem_do_wake() wakes readers unconditionally.
> But we mustn't do that if the sem is owned by writer
> in the moment. Otherwise, writer and reader own the sem
> the same time, which leads to memory corruption in
> callers.
>
> rwsem-xadd.c does not need that, as:
>
> 1) the similar check is made lockless there,
> 2) in __rwsem_mark_wake::try_reader_grant we test,
>
> that sem is not owned by writer.
>
> Signed-off-by: Kirill Tkhai <[email protected]>
> Acked-by: Peter Zijlstra <[email protected]>
> Cc: <[email protected]>
> Cc: Linus Torvalds <[email protected]>
> Cc: Niklas Cassel <[email protected]>
> Cc: Peter Zijlstra (Intel) <[email protected]>
> Cc: Peter Zijlstra <[email protected]>
> Cc: Thomas Gleixner <[email protected]>
> Fixes: 17fcbd590d0c "locking/rwsem: Fix down_write_killable() for CONFIG_RWSEM_GENERIC_SPINLOCK=y"
> Link: http://lkml.kernel.org/r/149762063282.19811.9129615532201147826.stgit@localhost.localdomain
> Signed-off-by: Ingo Molnar <[email protected]>
> ---
> kernel/locking/rwsem-spinlock.c | 4 ++--
> 1 file changed, 2 insertions(+), 2 deletions(-)
>
> diff --git a/kernel/locking/rwsem-spinlock.c b/kernel/locking/rwsem-spinlock.c
> index c65f798..20819df 100644
> --- a/kernel/locking/rwsem-spinlock.c
> +++ b/kernel/locking/rwsem-spinlock.c
> @@ -231,8 +231,8 @@ int __sched __down_write_common(struct rw_semaphore *sem, int state)
>
> out_nolock:
> list_del(&waiter.list);
> - if (!list_empty(&sem->wait_list))
> - __rwsem_do_wake(sem, 1);
> + if (!list_empty(&sem->wait_list) && sem->count >= 0)
> + __rwsem_do_wake(sem, 0);
> raw_spin_unlock_irqrestore(&sem->wait_lock, flags);
>
> return -EINTR;
>
For the record, there is actually a v2 of this:
http://marc.info/?l=linux-kernel&m=149866422128912
Regards,
Niklas
* Niklas Cassel <[email protected]> wrote:
> On 07/05/2017 04:27 PM, tip-bot for Kirill Tkhai wrote:
> > Commit-ID: a0c4acd2c220376b4e9690e75782d0c0afdaab9f
> > Gitweb: http://git.kernel.org/tip/a0c4acd2c220376b4e9690e75782d0c0afdaab9f
> > Author: Kirill Tkhai <[email protected]>
> > AuthorDate: Fri, 16 Jun 2017 16:44:34 +0300
> > Committer: Ingo Molnar <[email protected]>
> > CommitDate: Wed, 5 Jul 2017 12:26:29 +0200
> >
> > locking/rwsem-spinlock: Fix EINTR branch in __down_write_common()
> >
> > If a writer could been woken up, the above branch
> >
> > if (sem->count == 0)
> > break;
> >
> > would have moved us to taking the sem. So, it's
> > not the time to wake a writer now, and only readers
> > are allowed now. Thus, 0 must be passed to __rwsem_do_wake().
> >
> > Next, __rwsem_do_wake() wakes readers unconditionally.
> > But we mustn't do that if the sem is owned by writer
> > in the moment. Otherwise, writer and reader own the sem
> > the same time, which leads to memory corruption in
> > callers.
> >
> > rwsem-xadd.c does not need that, as:
> >
> > 1) the similar check is made lockless there,
> > 2) in __rwsem_mark_wake::try_reader_grant we test,
> >
> > that sem is not owned by writer.
> >
> > Signed-off-by: Kirill Tkhai <[email protected]>
> > Acked-by: Peter Zijlstra <[email protected]>
> > Cc: <[email protected]>
> > Cc: Linus Torvalds <[email protected]>
> > Cc: Niklas Cassel <[email protected]>
> > Cc: Peter Zijlstra (Intel) <[email protected]>
> > Cc: Peter Zijlstra <[email protected]>
> > Cc: Thomas Gleixner <[email protected]>
> > Fixes: 17fcbd590d0c "locking/rwsem: Fix down_write_killable() for CONFIG_RWSEM_GENERIC_SPINLOCK=y"
> > Link: http://lkml.kernel.org/r/149762063282.19811.9129615532201147826.stgit@localhost.localdomain
> > Signed-off-by: Ingo Molnar <[email protected]>
> > ---
> > kernel/locking/rwsem-spinlock.c | 4 ++--
> > 1 file changed, 2 insertions(+), 2 deletions(-)
> >
> > diff --git a/kernel/locking/rwsem-spinlock.c b/kernel/locking/rwsem-spinlock.c
> > index c65f798..20819df 100644
> > --- a/kernel/locking/rwsem-spinlock.c
> > +++ b/kernel/locking/rwsem-spinlock.c
> > @@ -231,8 +231,8 @@ int __sched __down_write_common(struct rw_semaphore *sem, int state)
> >
> > out_nolock:
> > list_del(&waiter.list);
> > - if (!list_empty(&sem->wait_list))
> > - __rwsem_do_wake(sem, 1);
> > + if (!list_empty(&sem->wait_list) && sem->count >= 0)
> > + __rwsem_do_wake(sem, 0);
> > raw_spin_unlock_irqrestore(&sem->wait_lock, flags);
> >
> > return -EINTR;
> >
>
> For the record, there is actually a v2 of this:
>
> http://marc.info/?l=linux-kernel&m=149866422128912
Hm, so I missed that because it was within the discussion - please post v2 patches
with a new subject line next time around.
But I also disagree with -v2 mildly: in practice a >= test has the same CPU
overhead as a > test, and if we rely on the earlier "sem->count == 0" test then we
should also comment on that.
It's more straightforward to just do the canonical sem->count >= 0 test that we do
elsewhere in the rwsem-spinlock code.
PeterZ, what's your preference?
Thanks,
Ingo
On Thu, Jul 06, 2017 at 09:28:58AM +0200, Ingo Molnar wrote:
> It's more straightforward to just do the canonical sem->count >= 0 test that we do
> elsewhere in the rwsem-spinlock code.
>
> PeterZ, what's your preference?
Leave it as is.. it doesn't matter (the 0 case shouldn't happen) and as
you say >= 0 is what most other code does.