Optimise signal_pending() by checking both TIF_SIGPENDING and
TIF_NOTIFY_SIGNAL at once. Saves quite a bit of generated instructions,
e.g. sheds 240B from io_uring alone, some including ones in hot paths.
text data bss dec hex filename
84087 12414 8 96509 178fd ./fs/io_uring.o
83847 12414 8 96269 1780d ./fs/io_uring.o
Signed-off-by: Pavel Begunkov <[email protected]>
---
Suggestions on how to make it less disruptive to abstractions are most
welcome, as even the one below fails to generated anything sane because
of test_bit()
return unlikely(test_ti_thread_flag(ti, TIF_SIGPENDING) |
test_ti_thread_flag(ti, TIF_SIGPENDING));
include/linux/sched/signal.h | 6 +++---
1 file changed, 3 insertions(+), 3 deletions(-)
diff --git a/include/linux/sched/signal.h b/include/linux/sched/signal.h
index 3f6a0fcaa10c..97e1963a13fc 100644
--- a/include/linux/sched/signal.h
+++ b/include/linux/sched/signal.h
@@ -361,14 +361,14 @@ static inline int task_sigpending(struct task_struct *p)
static inline int signal_pending(struct task_struct *p)
{
+ struct thread_info *ti = task_thread_info(p);
+
/*
* TIF_NOTIFY_SIGNAL isn't really a signal, but it requires the same
* behavior in terms of ensuring that we break out of wait loops
* so that notify signal callbacks can be processed.
*/
- if (unlikely(test_tsk_thread_flag(p, TIF_NOTIFY_SIGNAL)))
- return 1;
- return task_sigpending(p);
+ return unlikely(ti->flags & (_TIF_SIGPENDING | _TIF_NOTIFY_SIGNAL));
}
static inline int __fatal_signal_pending(struct task_struct *p)
--
2.31.1
Pavel Begunkov <[email protected]> writes:
> Optimise signal_pending() by checking both TIF_SIGPENDING and
> TIF_NOTIFY_SIGNAL at once. Saves quite a bit of generated instructions,
> e.g. sheds 240B from io_uring alone, some including ones in hot paths.
>
> text data bss dec hex filename
> 84087 12414 8 96509 178fd ./fs/io_uring.o
> 83847 12414 8 96269 1780d ./fs/io_uring.o
I believe the atomic test_bit is pretty fundamental, especially with
it's implied barriers. I believe you are optimizing out the code
that will makes signal_pending work in a loop.
I have tried looking and I really don't understand why TIF_NOTIFY_SIGNAL
was added. Perhaps instead of trying to optimize the test, you should
optimize by combining TIF_NOTIFY_SIGNAL with TIF_SIGPENDING.
Perhaps set_notify_signal could be optimized to set both. I think I
only see 4 calls in the tree.
> Signed-off-by: Pavel Begunkov <[email protected]>
> ---
>
> Suggestions on how to make it less disruptive to abstractions are most
> welcome, as even the one below fails to generated anything sane because
> of test_bit()
>
> return unlikely(test_ti_thread_flag(ti, TIF_SIGPENDING) |
> test_ti_thread_flag(ti, TIF_SIGPENDING));
>
> include/linux/sched/signal.h | 6 +++---
> 1 file changed, 3 insertions(+), 3 deletions(-)
>
> diff --git a/include/linux/sched/signal.h b/include/linux/sched/signal.h
> index 3f6a0fcaa10c..97e1963a13fc 100644
> --- a/include/linux/sched/signal.h
> +++ b/include/linux/sched/signal.h
> @@ -361,14 +361,14 @@ static inline int task_sigpending(struct task_struct *p)
>
> static inline int signal_pending(struct task_struct *p)
> {
> + struct thread_info *ti = task_thread_info(p);
> +
> /*
> * TIF_NOTIFY_SIGNAL isn't really a signal, but it requires the same
> * behavior in terms of ensuring that we break out of wait loops
> * so that notify signal callbacks can be processed.
> */
> - if (unlikely(test_tsk_thread_flag(p, TIF_NOTIFY_SIGNAL)))
> - return 1;
> - return task_sigpending(p);
> + return unlikely(ti->flags & (_TIF_SIGPENDING | _TIF_NOTIFY_SIGNAL));
> }
>
> static inline int __fatal_signal_pending(struct task_struct *p)
On 5/17/21 6:22 PM, Eric W. Biederman wrote:
> Pavel Begunkov <[email protected]> writes:
>
>> Optimise signal_pending() by checking both TIF_SIGPENDING and
>> TIF_NOTIFY_SIGNAL at once. Saves quite a bit of generated instructions,
>> e.g. sheds 240B from io_uring alone, some including ones in hot paths.
>>
>> text data bss dec hex filename
>> 84087 12414 8 96509 178fd ./fs/io_uring.o
>> 83847 12414 8 96269 1780d ./fs/io_uring.o
>
> I believe the atomic test_bit is pretty fundamental, especially with
> it's implied barriers. I believe you are optimizing out the code
> that will makes signal_pending work in a loop.
Hmm, does it? I agree that at least it should volatile, but unlike
set_bit(), which is in atomic.h and has a non atomic __set_bit()
counter part, test_bit() is bitops/non-atomic.h, and I don't see
any implementation of test_bit() or arch_test_bit() having any
barriers.
READ_ONCE() should cover volatile, would it be better?
or test_mask() operating withing one word?
> I have tried looking and I really don't understand why TIF_NOTIFY_SIGNAL
> was added. Perhaps instead of trying to optimize the test, you should
> optimize by combining TIF_NOTIFY_SIGNAL with TIF_SIGPENDING.
I'm speculating, but it looks to me that TIF_NOTIFY_SIGNAL was
specifically added to not rely on and separate from TIF_SIGPENDING
for task_work notification delivery.
>
> Perhaps set_notify_signal could be optimized to set both. I think I
> only see 4 calls in the tree.
>
>> Signed-off-by: Pavel Begunkov <[email protected]>
>> ---
>>
>> Suggestions on how to make it less disruptive to abstractions are most
>> welcome, as even the one below fails to generated anything sane because
>> of test_bit()
>>
>> return unlikely(test_ti_thread_flag(ti, TIF_SIGPENDING) |
>> test_ti_thread_flag(ti, TIF_SIGPENDING));
>>
>> include/linux/sched/signal.h | 6 +++---
>> 1 file changed, 3 insertions(+), 3 deletions(-)
>>
>> diff --git a/include/linux/sched/signal.h b/include/linux/sched/signal.h
>> index 3f6a0fcaa10c..97e1963a13fc 100644
>> --- a/include/linux/sched/signal.h
>> +++ b/include/linux/sched/signal.h
>> @@ -361,14 +361,14 @@ static inline int task_sigpending(struct task_struct *p)
>>
>> static inline int signal_pending(struct task_struct *p)
>> {
>> + struct thread_info *ti = task_thread_info(p);
>> +
>> /*
>> * TIF_NOTIFY_SIGNAL isn't really a signal, but it requires the same
>> * behavior in terms of ensuring that we break out of wait loops
>> * so that notify signal callbacks can be processed.
>> */
>> - if (unlikely(test_tsk_thread_flag(p, TIF_NOTIFY_SIGNAL)))
>> - return 1;
>> - return task_sigpending(p);
>> + return unlikely(ti->flags & (_TIF_SIGPENDING | _TIF_NOTIFY_SIGNAL));
>> }
>>
>> static inline int __fatal_signal_pending(struct task_struct *p)
--
Pavel Begunkov