LinuxLists.cc - [PATCH] xen/gntdev: fix unmap notification order

2021-12-10 09:28:23

Subject: [PATCH] xen/gntdev: fix unmap notification order

From: Oleksandr Andrushchenko <[email protected]>

While working with Xen's libxenvchan library I have faced an issue with
unmap notifications sent in wrong order if both UNMAP_NOTIFY_SEND_EVENT
and UNMAP_NOTIFY_CLEAR_BYTE were requested: first we send an event channel
notification and then clear the notification byte which renders in the below
inconsistency (cli_live is the byte which was requested to be cleared on unmap):

[ 444.514243] gntdev_put_map UNMAP_NOTIFY_SEND_EVENT map->notify.event 6
libxenvchan_is_open cli_live 1
[ 444.515239] __unmap_grant_pages UNMAP_NOTIFY_CLEAR_BYTE at 14

Thus it is not possible to reliably implement the checks like
- wait for the notification (UNMAP_NOTIFY_SEND_EVENT)
- check the variable (UNMAP_NOTIFY_CLEAR_BYTE)
because it is possible that the variable gets checked before it is cleared
by the kernel.

To fix that we need to re-order the notifications, so the variable is first
gets cleared and then the event channel notification is sent.
With this fix I can see the correct order of execution:

[ 54.522611] __unmap_grant_pages UNMAP_NOTIFY_CLEAR_BYTE at 14
[ 54.537966] gntdev_put_map UNMAP_NOTIFY_SEND_EVENT map->notify.event 6
libxenvchan_is_open cli_live 0

Cc: [email protected]
Signed-off-by: Oleksandr Andrushchenko <[email protected]>
---
drivers/xen/gntdev.c | 6 +++---
1 file changed, 3 insertions(+), 3 deletions(-)

diff --git a/drivers/xen/gntdev.c b/drivers/xen/gntdev.c
index fec1b6537166..59ffea800079 100644
--- a/drivers/xen/gntdev.c
+++ b/drivers/xen/gntdev.c
@@ -250,13 +250,13 @@ void gntdev_put_map(struct gntdev_priv *priv, struct gntdev_grant_map *map)
if (!refcount_dec_and_test(&map->users))
return;

+ if (map->pages && !use_ptemod)
+ unmap_grant_pages(map, 0, map->count);
+
if (map->notify.flags & UNMAP_NOTIFY_SEND_EVENT) {
notify_remote_via_evtchn(map->notify.event);
evtchn_put(map->notify.event);
}
-
- if (map->pages && !use_ptemod)
- unmap_grant_pages(map, 0, map->count);
gntdev_free_map(map);
}

--
2.25.1

2021-12-11 00:47:35

by Boris Ostrovsky

[permalink] [raw]

Subject: Re: [PATCH] xen/gntdev: fix unmap notification order

On 12/10/21 4:28 AM, Oleksandr Andrushchenko wrote:
> From: Oleksandr Andrushchenko <[email protected]>
>
> While working with Xen's libxenvchan library I have faced an issue with
> unmap notifications sent in wrong order if both UNMAP_NOTIFY_SEND_EVENT
> and UNMAP_NOTIFY_CLEAR_BYTE were requested: first we send an event channel
> notification and then clear the notification byte which renders in the below
> inconsistency (cli_live is the byte which was requested to be cleared on unmap):
>
> [ 444.514243] gntdev_put_map UNMAP_NOTIFY_SEND_EVENT map->notify.event 6
> libxenvchan_is_open cli_live 1
> [ 444.515239] __unmap_grant_pages UNMAP_NOTIFY_CLEAR_BYTE at 14
>
> Thus it is not possible to reliably implement the checks like
> - wait for the notification (UNMAP_NOTIFY_SEND_EVENT)
> - check the variable (UNMAP_NOTIFY_CLEAR_BYTE)
> because it is possible that the variable gets checked before it is cleared
> by the kernel.
>
> To fix that we need to re-order the notifications, so the variable is first
> gets cleared and then the event channel notification is sent.
> With this fix I can see the correct order of execution:
>
> [ 54.522611] __unmap_grant_pages UNMAP_NOTIFY_CLEAR_BYTE at 14
> [ 54.537966] gntdev_put_map UNMAP_NOTIFY_SEND_EVENT map->notify.event 6
> libxenvchan_is_open cli_live 0
>
> Cc: [email protected]
> Signed-off-by: Oleksandr Andrushchenko <[email protected]>

Reviewed-by: Boris Ostrovsky <[email protected]>

2021-12-13 13:46:52

by Boris Ostrovsky

[permalink] [raw]