2022-07-14 16:44:44

by Uros Bizjak

Subject: [PATCH] io_uring: Use atomic_long_try_cmpxchg in __io_account_mem

Use atomic_long_try_cmpxchg instead of
atomic_long_cmpxchg (*ptr, old, new) == old in __io_account_mem.
x86 CMPXCHG instruction returns success in ZF flag, so this
change saves a compare after cmpxchg (and related move
instruction in front of cmpxchg).

Also, atomic_long_try_cmpxchg implicitly assigns old *ptr value
to "old" when cmpxchg fails, enabling further code simplifications.

No functional change intended.

Signed-off-by: Uros Bizjak <[email protected]>
Cc: Jens Axboe <[email protected]>
Cc: Pavel Begunkov <[email protected]>
---
fs/io_uring.c | 7 +++----
1 file changed, 3 insertions(+), 4 deletions(-)

diff --git a/fs/io_uring.c b/fs/io_uring.c
index a01ea49f3017..9f82904dcdae 100644
--- a/fs/io_uring.c
+++ b/fs/io_uring.c
@@ -10504,14 +10504,13 @@ static inline int __io_account_mem(struct user_struct *user,
/* Don't allow more pages than we can safely lock */
page_limit = rlimit(RLIMIT_MEMLOCK) >> PAGE_SHIFT;

+ cur_pages = atomic_long_read(&user->locked_vm);
do {
- cur_pages = atomic_long_read(&user->locked_vm);
new_pages = cur_pages + nr_pages;
if (new_pages > page_limit)
return -ENOMEM;
- } while (atomic_long_cmpxchg(&user->locked_vm, cur_pages,
- new_pages) != cur_pages);
-
+ } while (!atomic_long_try_cmpxchg(&user->locked_vm,
+ &cur_pages, new_pages));
return 0;
}

--
2.35.3

2022-07-14 17:21:01

by Jens Axboe

Subject: Re: [PATCH] io_uring: Use atomic_long_try_cmpxchg in __io_account_mem

On 7/14/22 10:33 AM, Uros Bizjak wrote:
> Use atomic_long_try_cmpxchg instead of
> atomic_long_cmpxchg (*ptr, old, new) == old in __io_account_mem.
> x86 CMPXCHG instruction returns success in ZF flag, so this
> change saves a compare after cmpxchg (and related move
> instruction in front of cmpxchg).
>
> Also, atomic_long_try_cmpxchg implicitly assigns old *ptr value
> to "old" when cmpxchg fails, enabling further code simplifications.
>
> No functional change intended.

This will be io_uring/rsrc.c for the for-next branches, but it'll apply
directly as that with a slight offset:

checking file io_uring/rsrc.c
Hunk #1 succeeded at 56 (offset -10448 lines).

I'll do that, thanks.

--
Jens Axboe