2020-07-22 22:12:36

by Thomas Gleixner

Subject: [patch V5 08/15] x86/entry: Move user return notifier out of loop

From: Thomas Gleixner <[email protected]>

Guests and user space share certain MSRs. KVM sets these MSRs to guest
values once and does not set them back to user space values on every VM
exit to spare the costly MSR operations.

User return notifiers ensure that these MSRs are set back to the correct
values before returning to user space in exit_to_usermode_loop().

There is no reason to evaluate the TIF flag indicating that user return
notifiers need to be invoked in the loop. The important point is that they
are invoked before returning to user space.

Move the invocation out of the loop into the section which does the last
preperatory steps before returning to user space. That section is not
preemptible and runs with interrupts disabled until the actual return.

Signed-off-by: Thomas Gleixner <[email protected]>

---
V4: New patch
---
arch/x86/entry/common.c | 8 ++++----
1 file changed, 4 insertions(+), 4 deletions(-)

--- a/arch/x86/entry/common.c
+++ b/arch/x86/entry/common.c
@@ -208,7 +208,7 @@ static long syscall_trace_enter(struct p

#define EXIT_TO_USERMODE_LOOP_FLAGS \
(_TIF_SIGPENDING | _TIF_NOTIFY_RESUME | _TIF_UPROBE | \
- _TIF_NEED_RESCHED | _TIF_USER_RETURN_NOTIFY | _TIF_PATCH_PENDING)
+ _TIF_NEED_RESCHED | _TIF_PATCH_PENDING)

static void exit_to_usermode_loop(struct pt_regs *regs, u32 cached_flags)
{
@@ -242,9 +242,6 @@ static void exit_to_usermode_loop(struct
rseq_handle_notify_resume(NULL, regs);
}

- if (cached_flags & _TIF_USER_RETURN_NOTIFY)
- fire_user_return_notifiers();
-
/* Disable IRQs and retry */
local_irq_disable();

@@ -273,6 +270,9 @@ static void __prepare_exit_to_usermode(s
/* Reload ti->flags; we may have rescheduled above. */
cached_flags = READ_ONCE(ti->flags);

+ if (cached_flags & _TIF_USER_RETURN_NOTIFY)
+ fire_user_return_notifiers();
+
if (unlikely(cached_flags & _TIF_IO_BITMAP))
tss_update_io_bitmap();

2020-07-23 23:43:33

by Sean Christopherson

Subject: Re: [patch V5 08/15] x86/entry: Move user return notifier out of loop

On Thu, Jul 23, 2020 at 12:00:02AM +0200, Thomas Gleixner wrote:
> From: Thomas Gleixner <[email protected]>
>
> Guests and user space share certain MSRs. KVM sets these MSRs to guest
> values once and does not set them back to user space values on every VM
> exit to spare the costly MSR operations.
>
> User return notifiers ensure that these MSRs are set back to the correct
> values before returning to user space in exit_to_usermode_loop().
>
> There is no reason to evaluate the TIF flag indicating that user return
> notifiers need to be invoked in the loop. The important point is that they
> are invoked before returning to user space.
>
> Move the invocation out of the loop into the section which does the last
> preperatory steps before returning to user space. That section is not
> preemptible and runs with interrupts disabled until the actual return.
>
> Signed-off-by: Thomas Gleixner <[email protected]>

Reviewed-and-tested-by: Sean Christopherson <[email protected]>

2020-07-24 20:14:57

by tip-bot2 for Alexey Makhalov

Subject: [tip: x86/entry] x86/entry: Move user return notifier out of loop

The following commit has been merged into the x86/entry branch of tip:

Commit-ID: a377ac1cd9d7b9ac8d546dceb3d74956fbfd443f
Gitweb: https://git.kernel.org/tip/a377ac1cd9d7b9ac8d546dceb3d74956fbfd443f
Author: Thomas Gleixner <[email protected]>
AuthorDate: Thu, 23 Jul 2020 00:00:02 +02:00
Committer: Thomas Gleixner <[email protected]>
CommitterDate: Fri, 24 Jul 2020 15:04:58 +02:00

x86/entry: Move user return notifier out of loop

Guests and user space share certain MSRs. KVM sets these MSRs to guest
values once and does not set them back to user space values on every VM
exit to spare the costly MSR operations.

User return notifiers ensure that these MSRs are set back to the correct
values before returning to user space in exit_to_usermode_loop().

There is no reason to evaluate the TIF flag indicating that user return
notifiers need to be invoked in the loop. The important point is that they
are invoked before returning to user space.

Move the invocation out of the loop into the section which does the last
preperatory steps before returning to user space. That section is not
preemptible and runs with interrupts disabled until the actual return.

Signed-off-by: Thomas Gleixner <[email protected]>
Link: https://lkml.kernel.org/r/[email protected]


---
arch/x86/entry/common.c | 8 ++++----
1 file changed, 4 insertions(+), 4 deletions(-)

diff --git a/arch/x86/entry/common.c b/arch/x86/entry/common.c
index 68d5c86..9415ae5 100644
--- a/arch/x86/entry/common.c
+++ b/arch/x86/entry/common.c
@@ -208,7 +208,7 @@ static long syscall_trace_enter(struct pt_regs *regs)

#define EXIT_TO_USERMODE_LOOP_FLAGS \
(_TIF_SIGPENDING | _TIF_NOTIFY_RESUME | _TIF_UPROBE | \
- _TIF_NEED_RESCHED | _TIF_USER_RETURN_NOTIFY | _TIF_PATCH_PENDING)
+ _TIF_NEED_RESCHED | _TIF_PATCH_PENDING)

static void exit_to_usermode_loop(struct pt_regs *regs, u32 cached_flags)
{
@@ -242,9 +242,6 @@ static void exit_to_usermode_loop(struct pt_regs *regs, u32 cached_flags)
rseq_handle_notify_resume(NULL, regs);
}

- if (cached_flags & _TIF_USER_RETURN_NOTIFY)
- fire_user_return_notifiers();
-
/* Disable IRQs and retry */
local_irq_disable();

@@ -273,6 +270,9 @@ static void __prepare_exit_to_usermode(struct pt_regs *regs)
/* Reload ti->flags; we may have rescheduled above. */
cached_flags = READ_ONCE(ti->flags);

+ if (cached_flags & _TIF_USER_RETURN_NOTIFY)
+ fire_user_return_notifiers();
+
if (unlikely(cached_flags & _TIF_IO_BITMAP))
tss_update_io_bitmap();