From: Huang Ying <[email protected]>
Before using cluster lock in free_swap_and_cache(), the
swap_info_struct->lock will be held during freeing the swap entry and
acquiring page lock, so the page swap count will not change when
testing page information later. But after using cluster lock, the
cluster lock (or swap_info_struct->lock) will be held only during
freeing the swap entry. So before acquiring the page lock, the page
swap count may be changed in another thread. If the page swap count
is not 0, we should not delete the page from the swap cache. This is
fixed via checking page swap count again after acquiring the page
lock.
Signed-off-by: "Huang, Ying" <[email protected]>
Cc: Hugh Dickins <[email protected]>
Cc: Shaohua Li <[email protected]>
Cc: Minchan Kim <[email protected]>
Cc: Rik van Riel <[email protected]>
Cc: Tim Chen <[email protected]>
---
mm/swapfile.c | 25 ++++++++++++++++---------
1 file changed, 16 insertions(+), 9 deletions(-)
diff --git a/mm/swapfile.c b/mm/swapfile.c
index fadc6a1c0da0..5b67f8ce424c 100644
--- a/mm/swapfile.c
+++ b/mm/swapfile.c
@@ -1109,6 +1109,18 @@ int page_swapcount(struct page *page)
return count;
}
+static int swap_swapcount(struct swap_info_struct *si, swp_entry_t entry)
+{
+ int count = 0;
+ pgoff_t offset = swp_offset(entry);
+ struct swap_cluster_info *ci;
+
+ ci = lock_cluster_or_swap_info(si, offset);
+ count = swap_count(si->swap_map[offset]);
+ unlock_cluster_or_swap_info(si, ci);
+ return count;
+}
+
/*
* How many references to @entry are currently swapped out?
* This does not give an exact answer when swap count is continued,
@@ -1117,17 +1129,11 @@ int page_swapcount(struct page *page)
int __swp_swapcount(swp_entry_t entry)
{
int count = 0;
- pgoff_t offset;
struct swap_info_struct *si;
- struct swap_cluster_info *ci;
si = __swap_info_get(entry);
- if (si) {
- offset = swp_offset(entry);
- ci = lock_cluster_or_swap_info(si, offset);
- count = swap_count(si->swap_map[offset]);
- unlock_cluster_or_swap_info(si, ci);
- }
+ if (si)
+ count = swap_swapcount(si, entry);
return count;
}
@@ -1289,7 +1295,8 @@ int free_swap_and_cache(swp_entry_t entry)
* Also recheck PageSwapCache now page is locked (above).
*/
if (PageSwapCache(page) && !PageWriteback(page) &&
- (!page_mapped(page) || mem_cgroup_swap_full(page))) {
+ (!page_mapped(page) || mem_cgroup_swap_full(page)) &&
+ !swap_swapcount(p, entry)) {
delete_from_swap_cache(page);
SetPageDirty(page);
}
--
2.11.0
On Wed, 1 Mar 2017 22:38:09 +0800 "Huang, Ying" <[email protected]> wrote:
> Before using cluster lock in free_swap_and_cache(), the
> swap_info_struct->lock will be held during freeing the swap entry and
> acquiring page lock, so the page swap count will not change when
> testing page information later. But after using cluster lock, the
> cluster lock (or swap_info_struct->lock) will be held only during
> freeing the swap entry. So before acquiring the page lock, the page
> swap count may be changed in another thread. If the page swap count
> is not 0, we should not delete the page from the swap cache. This is
> fixed via checking page swap count again after acquiring the page
> lock.
What are the user-visible runtime effects of this bug? Please always
include this info when fixing things, thanks.
Hi, Andrew,
Sorry, I clicked the wrong button in my mail client, so forgot Ccing
mailing list. Sorry for duplicated mail.
On Sat, Mar 4, 2017 at 6:43 AM, Andrew Morton <[email protected]> wrote:
> On Wed, 1 Mar 2017 22:38:09 +0800 "Huang, Ying" <[email protected]> wrote:
>
>> Before using cluster lock in free_swap_and_cache(), the
>> swap_info_struct->lock will be held during freeing the swap entry and
>> acquiring page lock, so the page swap count will not change when
>> testing page information later. But after using cluster lock, the
>> cluster lock (or swap_info_struct->lock) will be held only during
>> freeing the swap entry. So before acquiring the page lock, the page
>> swap count may be changed in another thread. If the page swap count
>> is not 0, we should not delete the page from the swap cache. This is
>> fixed via checking page swap count again after acquiring the page
>> lock.
>
> What are the user-visible runtime effects of this bug? Please always
> include this info when fixing things, thanks.
Sure. I find the race when I review the code, so I didn't trigger the
race via a test program. If the race occurs for an anonymous page
shared by multiple processes via fork, multiple pages will be
allocated and swapped in from the swap device for the previously
shared one page. That is, the user-visible runtime effect is more
memory will be used and the access latency for the page will be
higher, that is, the performance regression.
Best Regards,
Huang, Ying