The general rule to use a swap entry is as follows.
When we get a swap entry, if there aren't some other ways to prevent
swapoff, such as the folio in swap cache is locked, page table lock is
held, etc., the swap entry may become invalid because of swapoff.
Then, we need to enclose all swap related functions with
get_swap_device() and put_swap_device(), unless the swap functions
call get/put_swap_device() by themselves.
Based on the above rule, all get/put_swap_device() usage are checked
and cleaned up if necessary.
Changelogs:
V3:
- Fix build error in [2/5], Thanks David!
- Fix comments and patch description about the folio in swap cache, Thanks David!
- Collected reviewed-by.
V2:
- Split patch per David's comments. Thanks!
Best Regards,
Huang, Ying
The general rule to use a swap entry is as follows.
When we get a swap entry, if there aren't some other ways to prevent
swapoff, such as the folio in swap cache is locked, page table lock is
held, etc., the swap entry may become invalid because of swapoff.
Then, we need to enclose all swap related functions with
get_swap_device() and put_swap_device(), unless the swap functions
call get/put_swap_device() by themselves.
Add the rule as comments of get_swap_device().
Signed-off-by: "Huang, Ying" <[email protected]>
Reviewed-by: David Hildenbrand <[email protected]>
Reviewed-by: Yosry Ahmed <[email protected]>
Cc: Hugh Dickins <[email protected]>
Cc: Johannes Weiner <[email protected]>
Cc: Matthew Wilcox <[email protected]>
Cc: Michal Hocko <[email protected]>
Cc: Minchan Kim <[email protected]>
Cc: Tim Chen <[email protected]>
Cc: Yang Shi <[email protected]>
Cc: Yu Zhao <[email protected]>
Cc: Chris Li <[email protected]>
---
mm/swapfile.c | 12 +++++++++---
1 file changed, 9 insertions(+), 3 deletions(-)
diff --git a/mm/swapfile.c b/mm/swapfile.c
index 4dbaea64635d..3d0e932497f0 100644
--- a/mm/swapfile.c
+++ b/mm/swapfile.c
@@ -1219,6 +1219,13 @@ static unsigned char __swap_entry_free_locked(struct swap_info_struct *p,
}
/*
+ * When we get a swap entry, if there aren't some other ways to
+ * prevent swapoff, such as the folio in swap cache is locked, page
+ * table lock is held, etc., the swap entry may become invalid because
+ * of swapoff. Then, we need to enclose all swap related functions
+ * with get_swap_device() and put_swap_device(), unless the swap
+ * functions call get/put_swap_device() by themselves.
+ *
* Check whether swap entry is valid in the swap device. If so,
* return pointer to swap_info_struct, and keep the swap entry valid
* via preventing the swap device from being swapoff, until
@@ -1227,9 +1234,8 @@ static unsigned char __swap_entry_free_locked(struct swap_info_struct *p,
* Notice that swapoff or swapoff+swapon can still happen before the
* percpu_ref_tryget_live() in get_swap_device() or after the
* percpu_ref_put() in put_swap_device() if there isn't any other way
- * to prevent swapoff, such as page lock, page table lock, etc. The
- * caller must be prepared for that. For example, the following
- * situation is possible.
+ * to prevent swapoff. The caller must be prepared for that. For
+ * example, the following situation is possible.
*
* CPU1 CPU2
* do_swap_page()
--
2.39.2
get/put_swap_device() are added to __swap_count() in commit
eb085574a752 ("mm, swap: fix race between swapoff and some swap
operations"). Later, in commit 2799e77529c2 ("swap: fix
do_swap_page() race with swapoff"), get/put_swap_device() are added to
do_swap_page(). And they enclose the only call site of
__swap_count(). So, it's safe to remove get/put_swap_device() in
__swap_count() now.
Signed-off-by: "Huang, Ying" <[email protected]>
Reviewed-by: Yosry Ahmed <[email protected]>
Reviewed-by: David Hildenbrand <[email protected]>
Cc: Hugh Dickins <[email protected]>
Cc: Johannes Weiner <[email protected]>
Cc: Matthew Wilcox <[email protected]>
Cc: Michal Hocko <[email protected]>
Cc: Minchan Kim <[email protected]>
Cc: Tim Chen <[email protected]>
Cc: Yang Shi <[email protected]>
Cc: Yu Zhao <[email protected]>
Cc: Chris Li <[email protected]>
---
mm/swapfile.c | 10 ++--------
1 file changed, 2 insertions(+), 8 deletions(-)
diff --git a/mm/swapfile.c b/mm/swapfile.c
index 274bbf797480..8419cba9c192 100644
--- a/mm/swapfile.c
+++ b/mm/swapfile.c
@@ -1432,16 +1432,10 @@ void swapcache_free_entries(swp_entry_t *entries, int n)
int __swap_count(swp_entry_t entry)
{
- struct swap_info_struct *si;
+ struct swap_info_struct *si = swp_swap_info(entry);
pgoff_t offset = swp_offset(entry);
- int count = 0;
- si = get_swap_device(entry);
- if (si) {
- count = swap_count(si->swap_map[offset]);
- put_swap_device(si);
- }
- return count;
+ return swap_count(si->swap_map[offset]);
}
/*
--
2.39.2
__swap_duplicate() is called by
- swap_shmem_alloc(): the folio in swap cache is locked.
- copy_nonpresent_pte() -> swap_duplicate() and try_to_unmap_one() ->
swap_duplicate(): the page table lock is held.
- __read_swap_cache_async() -> swapcache_prepare(): enclosed with
get/put_swap_device() in __read_swap_cache_async() already.
So, it's safe to remove get/put_swap_device() in __swap_duplicate().
Signed-off-by: "Huang, Ying" <[email protected]>
Reviewed-by: Yosry Ahmed <[email protected]>
Cc: David Hildenbrand <[email protected]>
Cc: Hugh Dickins <[email protected]>
Cc: Johannes Weiner <[email protected]>
Cc: Matthew Wilcox <[email protected]>
Cc: Michal Hocko <[email protected]>
Cc: Minchan Kim <[email protected]>
Cc: Tim Chen <[email protected]>
Cc: Yang Shi <[email protected]>
Cc: Yu Zhao <[email protected]>
Cc: Chris Li <[email protected]>
---
mm/swapfile.c | 5 +----
1 file changed, 1 insertion(+), 4 deletions(-)
diff --git a/mm/swapfile.c b/mm/swapfile.c
index e9cce775fb25..4dbaea64635d 100644
--- a/mm/swapfile.c
+++ b/mm/swapfile.c
@@ -3264,9 +3264,7 @@ static int __swap_duplicate(swp_entry_t entry, unsigned char usage)
unsigned char has_cache;
int err;
- p = get_swap_device(entry);
- if (!p)
- return -EINVAL;
+ p = swp_swap_info(entry);
offset = swp_offset(entry);
ci = lock_cluster_or_swap_info(p, offset);
@@ -3313,7 +3311,6 @@ static int __swap_duplicate(swp_entry_t entry, unsigned char usage)
unlock_out:
unlock_cluster_or_swap_info(p, ci);
- put_swap_device(p);
return err;
}
--
2.39.2
On 29.05.23 08:13, Huang Ying wrote:
> __swap_duplicate() is called by
>
> - swap_shmem_alloc(): the folio in swap cache is locked.
>
> - copy_nonpresent_pte() -> swap_duplicate() and try_to_unmap_one() ->
> swap_duplicate(): the page table lock is held.
>
> - __read_swap_cache_async() -> swapcache_prepare(): enclosed with
> get/put_swap_device() in __read_swap_cache_async() already.
>
> So, it's safe to remove get/put_swap_device() in __swap_duplicate().
>
> Signed-off-by: "Huang, Ying" <[email protected]>
> Reviewed-by: Yosry Ahmed <[email protected]>
> Cc: David Hildenbrand <[email protected]>
> Cc: Hugh Dickins <[email protected]>
> Cc: Johannes Weiner <[email protected]>
> Cc: Matthew Wilcox <[email protected]>
> Cc: Michal Hocko <[email protected]>
> Cc: Minchan Kim <[email protected]>
> Cc: Tim Chen <[email protected]>
> Cc: Yang Shi <[email protected]>
> Cc: Yu Zhao <[email protected]>
> Cc: Chris Li <[email protected]>
> ---
Reviewed-by: David Hildenbrand <[email protected]>
--
Thanks,
David / dhildenb
On Mon, May 29, 2023 at 02:13:51PM +0800, Huang Ying wrote:
> get/put_swap_device() are added to __swap_count() in commit
> eb085574a752 ("mm, swap: fix race between swapoff and some swap
> operations"). Later, in commit 2799e77529c2 ("swap: fix
> do_swap_page() race with swapoff"), get/put_swap_device() are added to
> do_swap_page(). And they enclose the only call site of
> __swap_count(). So, it's safe to remove get/put_swap_device() in
> __swap_count() now.
>
> Signed-off-by: "Huang, Ying" <[email protected]>
> Reviewed-by: Yosry Ahmed <[email protected]>
> Reviewed-by: David Hildenbrand <[email protected]>
> Cc: Hugh Dickins <[email protected]>
> Cc: Johannes Weiner <[email protected]>
> Cc: Matthew Wilcox <[email protected]>
> Cc: Michal Hocko <[email protected]>
> Cc: Minchan Kim <[email protected]>
> Cc: Tim Chen <[email protected]>
> Cc: Yang Shi <[email protected]>
> Cc: Yu Zhao <[email protected]>
> Cc: Chris Li <[email protected]>
Reviewed-By: Chris Li (Google) <[email protected]>
Chris
> ---
> mm/swapfile.c | 10 ++--------
> 1 file changed, 2 insertions(+), 8 deletions(-)
>
> diff --git a/mm/swapfile.c b/mm/swapfile.c
> index 274bbf797480..8419cba9c192 100644
> --- a/mm/swapfile.c
> +++ b/mm/swapfile.c
> @@ -1432,16 +1432,10 @@ void swapcache_free_entries(swp_entry_t *entries, int n)
>
> int __swap_count(swp_entry_t entry)
> {
> - struct swap_info_struct *si;
> + struct swap_info_struct *si = swp_swap_info(entry);
> pgoff_t offset = swp_offset(entry);
> - int count = 0;
>
> - si = get_swap_device(entry);
> - if (si) {
> - count = swap_count(si->swap_map[offset]);
> - put_swap_device(si);
> - }
> - return count;
> + return swap_count(si->swap_map[offset]);
> }
>
> /*
> --
> 2.39.2
>
On Mon, May 29, 2023 at 02:13:55PM +0800, Huang Ying wrote:
> The general rule to use a swap entry is as follows.
>
> When we get a swap entry, if there aren't some other ways to prevent
> swapoff, such as the folio in swap cache is locked, page table lock is
> held, etc., the swap entry may become invalid because of swapoff.
> Then, we need to enclose all swap related functions with
> get_swap_device() and put_swap_device(), unless the swap functions
> call get/put_swap_device() by themselves.
>
> Add the rule as comments of get_swap_device().
>
> Signed-off-by: "Huang, Ying" <[email protected]>
> Reviewed-by: David Hildenbrand <[email protected]>
> Reviewed-by: Yosry Ahmed <[email protected]>
> Cc: Hugh Dickins <[email protected]>
> Cc: Johannes Weiner <[email protected]>
> Cc: Matthew Wilcox <[email protected]>
> Cc: Michal Hocko <[email protected]>
> Cc: Minchan Kim <[email protected]>
> Cc: Tim Chen <[email protected]>
> Cc: Yang Shi <[email protected]>
> Cc: Yu Zhao <[email protected]>
> Cc: Chris Li <[email protected]>
> ---
> mm/swapfile.c | 12 +++++++++---
> 1 file changed, 9 insertions(+), 3 deletions(-)
>
> diff --git a/mm/swapfile.c b/mm/swapfile.c
> index 4dbaea64635d..3d0e932497f0 100644
> --- a/mm/swapfile.c
> +++ b/mm/swapfile.c
> @@ -1219,6 +1219,13 @@ static unsigned char __swap_entry_free_locked(struct swap_info_struct *p,
> }
>
> /*
> + * When we get a swap entry, if there aren't some other ways to
> + * prevent swapoff, such as the folio in swap cache is locked, page
> + * table lock is held, etc., the swap entry may become invalid because
> + * of swapoff. Then, we need to enclose all swap related functions
> + * with get_swap_device() and put_swap_device(), unless the swap
> + * functions call get/put_swap_device() by themselves.
> + *
> * Check whether swap entry is valid in the swap device. If so,
> * return pointer to swap_info_struct, and keep the swap entry valid
> * via preventing the swap device from being swapoff, until
> @@ -1227,9 +1234,8 @@ static unsigned char __swap_entry_free_locked(struct swap_info_struct *p,
> * Notice that swapoff or swapoff+swapon can still happen before the
> * percpu_ref_tryget_live() in get_swap_device() or after the
> * percpu_ref_put() in put_swap_device() if there isn't any other way
> - * to prevent swapoff, such as page lock, page table lock, etc. The
> - * caller must be prepared for that. For example, the following
> - * situation is possible.
> + * to prevent swapoff. The caller must be prepared for that. For
> + * example, the following situation is possible.
> *
> * CPU1 CPU2
> * do_swap_page()
> --
> 2.39.2
>
Reviewed-by: Chris Li (Google) <[email protected]>
Chris
On Mon, May 29, 2023 at 02:13:54PM +0800, Huang Ying wrote:
> __swap_duplicate() is called by
>
> - swap_shmem_alloc(): the folio in swap cache is locked.
>
> - copy_nonpresent_pte() -> swap_duplicate() and try_to_unmap_one() ->
> swap_duplicate(): the page table lock is held.
>
> - __read_swap_cache_async() -> swapcache_prepare(): enclosed with
> get/put_swap_device() in __read_swap_cache_async() already.
>
> So, it's safe to remove get/put_swap_device() in __swap_duplicate().
>
> Signed-off-by: "Huang, Ying" <[email protected]>
> Reviewed-by: Yosry Ahmed <[email protected]>
> Cc: David Hildenbrand <[email protected]>
> Cc: Hugh Dickins <[email protected]>
> Cc: Johannes Weiner <[email protected]>
> Cc: Matthew Wilcox <[email protected]>
> Cc: Michal Hocko <[email protected]>
> Cc: Minchan Kim <[email protected]>
> Cc: Tim Chen <[email protected]>
> Cc: Yang Shi <[email protected]>
> Cc: Yu Zhao <[email protected]>
> Cc: Chris Li <[email protected]>
> ---
> mm/swapfile.c | 5 +----
> 1 file changed, 1 insertion(+), 4 deletions(-)
>
> diff --git a/mm/swapfile.c b/mm/swapfile.c
> index e9cce775fb25..4dbaea64635d 100644
> --- a/mm/swapfile.c
> +++ b/mm/swapfile.c
> @@ -3264,9 +3264,7 @@ static int __swap_duplicate(swp_entry_t entry, unsigned char usage)
> unsigned char has_cache;
> int err;
>
> - p = get_swap_device(entry);
> - if (!p)
> - return -EINVAL;
> + p = swp_swap_info(entry);
>
> offset = swp_offset(entry);
> ci = lock_cluster_or_swap_info(p, offset);
> @@ -3313,7 +3311,6 @@ static int __swap_duplicate(swp_entry_t entry, unsigned char usage)
>
> unlock_out:
> unlock_cluster_or_swap_info(p, ci);
> - put_swap_device(p);
> return err;
> }
>
> --
> 2.39.2
>
Reviewed-by: Chris Li (Google) <[email protected]>
Chris