Received: by 10.223.185.116 with SMTP id b49csp6728464wrg; Wed, 28 Feb 2018 14:33:16 -0800 (PST) X-Google-Smtp-Source: AG47ELtJw7sDZAoHWmgC9vUBQCB7lQYvdOQ2MkOC5FZoiNbSP5t7BUnoKp6hMWmyKvgL1KWDum6R X-Received: by 10.167.131.86 with SMTP id z22mr9253827pfm.185.1519857196247; Wed, 28 Feb 2018 14:33:16 -0800 (PST) ARC-Seal: i=1; a=rsa-sha256; t=1519857196; cv=none; d=google.com; s=arc-20160816; b=tX8CTw3rsNepwJfXBlF/N6p2Km819DkqkyMAjhUPl49buCMIUTI/q70EpG9jrxdbeQ iLEwxFBibu6dkAahqd4s2NSsFMPdLsAUIrKHRT4X97Bqh39xShkvL/IcTfoFP13Xyh2u wCefUVahrQfEJ9jBRJsedoHbyL1NMYunbJWzDLrZUf/soOfS1yROQJvgsEcnA4g7KwE6 FcR09LG+vdUL83FzlOum4hWwZtnA5YHF2SXE4PS7sM8iJnhf6b9YcRbzTX/rw/ChhVXy HDqgmkDXDXT+h5My8y2z5EdZsONKWE79iws8DRx6v4kpHNj3EwWKobwPNryrhXlG3479 WSiA== ARC-Message-Signature: i=1; a=rsa-sha256; c=relaxed/relaxed; d=google.com; s=arc-20160816; h=list-id:precedence:sender:references:in-reply-to:message-id:date :subject:cc:to:from:dkim-signature:arc-authentication-results; bh=UJxqn+S1TQ0BTAhlyRYtYx2hC5B4YePZZjxE3RmOyD0=; b=MWjQziggrHdULtCf2WuamDyT3qMz9kZgvMTshxaMV+jNDFOGmD/ZcAxNPUWPVZEGe0 HS5PT1glgPOzcGbbOWUwGZ9AnTgVmetX0NJ6oAUI3fuwV7hfUwQUsuYz3N3+9HdegZ5n vTTXldDfOKANW7vNEiWfQjrpgvDRT0VbHxFfXLLXJWDCK17c0jf+1vEeK31pPACszAJV b8ZqUvhTdOgtPMbplJDvCy48T3mVE/cGv/RLyDaoWBB7GjAuiaFNTJqhV2nUahMh9Waj E9kb6P6+TwJl+xD8GsAv8v+HFBH3zJgFf60LAd5dEYoEkA6dz8pa8zJwr/pIkjK/yoxp 3Drg== ARC-Authentication-Results: i=1; mx.google.com; dkim=fail header.i=@infradead.org header.s=bombadil.20170209 header.b=dtI8NqAo; spf=pass (google.com: best guess record for domain of linux-kernel-owner@vger.kernel.org designates 209.132.180.67 as permitted sender) smtp.mailfrom=linux-kernel-owner@vger.kernel.org Return-Path: Received: from vger.kernel.org (vger.kernel.org. [209.132.180.67]) by mx.google.com with ESMTP id y9si1545750pgv.776.2018.02.28.14.33.01; Wed, 28 Feb 2018 14:33:16 -0800 (PST) Received-SPF: pass (google.com: best guess record for domain of linux-kernel-owner@vger.kernel.org designates 209.132.180.67 as permitted sender) client-ip=209.132.180.67; Authentication-Results: mx.google.com; dkim=fail header.i=@infradead.org header.s=bombadil.20170209 header.b=dtI8NqAo; spf=pass (google.com: best guess record for domain of linux-kernel-owner@vger.kernel.org designates 209.132.180.67 as permitted sender) smtp.mailfrom=linux-kernel-owner@vger.kernel.org Received: (majordomo@vger.kernel.org) by vger.kernel.org via listexpand id S935505AbeB1WcN (ORCPT + 99 others); Wed, 28 Feb 2018 17:32:13 -0500 Received: from bombadil.infradead.org ([198.137.202.133]:44416 "EHLO bombadil.infradead.org" rhost-flags-OK-OK-OK-OK) by vger.kernel.org with ESMTP id S935433AbeB1WcJ (ORCPT ); Wed, 28 Feb 2018 17:32:09 -0500 DKIM-Signature: v=1; a=rsa-sha256; q=dns/txt; c=relaxed/relaxed; d=infradead.org; s=bombadil.20170209; h=References:In-Reply-To:Message-Id: Date:Subject:Cc:To:From:Sender:Reply-To:MIME-Version:Content-Type: Content-Transfer-Encoding:Content-ID:Content-Description:Resent-Date: Resent-From:Resent-Sender:Resent-To:Resent-Cc:Resent-Message-ID:List-Id: List-Help:List-Unsubscribe:List-Subscribe:List-Post:List-Owner:List-Archive; bh=UJxqn+S1TQ0BTAhlyRYtYx2hC5B4YePZZjxE3RmOyD0=; b=dtI8NqAoj0kQ31zVKznwrWQ9m 7RMObIAl0QIXtMblDgut/fa6UC/DelGJBU6MjPhBgNYKJiLdenTceGqxSCw6diIOhNSmBdTinMzLv TioBG7CpuMoK5HrQjFKnDKagVFkbY9B7wtvMAS2chEW70A2LRqrJFtnKE66duV9LmcbJTCUHZqx9z oIWrCigGu9D/+Aey7QsfSLA1RV7FzzqMcfx5mnhmUz+U+A8Qn9FJqf/apmFPk3G4g8yiYqI+yD+1U YLPQdO8WRiJGPKtf+LG+ncJQm7RTm4ZMY/nBwSSlvWCiAxHlkXeOx+lNO6aO9uREgPpFEsMV9Ujue a9rwC9PYQ==; Received: from willy by bombadil.infradead.org with local (Exim 4.89 #1 (Red Hat Linux)) id 1erAGc-0002ZL-I0; Wed, 28 Feb 2018 22:32:06 +0000 From: Matthew Wilcox To: linux-mm@kvack.org Cc: Matthew Wilcox , Martin Schwidefsky , linux-kernel@vger.kernel.org Subject: [PATCH v3 1/4] s390: Use _refcount for pgtables Date: Wed, 28 Feb 2018 14:31:54 -0800 Message-Id: <20180228223157.9281-2-willy@infradead.org> X-Mailer: git-send-email 2.14.3 In-Reply-To: <20180228223157.9281-1-willy@infradead.org> References: <20180228223157.9281-1-willy@infradead.org> Sender: linux-kernel-owner@vger.kernel.org Precedence: bulk List-ID: X-Mailing-List: linux-kernel@vger.kernel.org From: Matthew Wilcox s390 borrows the storage used for _mapcount in struct page in order to account whether the bottom or top half is being used for 2kB page tables. I want to use that for something else, so use the top byte of _refcount instead of the bottom byte of _mapcount. _refcount may temporarily be incremented by other CPUs that see a stale pointer to this page in the page cache, but each CPU can only increment it by one, and there are no systems with 2^24 CPUs today, so they will not change the upper byte of _refcount. We do have to be a little careful not to lose any of their writes (as they will subsequently decrement the counter). Signed-off-by: Matthew Wilcox --- arch/s390/mm/pgalloc.c | 21 ++++++++++++--------- 1 file changed, 12 insertions(+), 9 deletions(-) diff --git a/arch/s390/mm/pgalloc.c b/arch/s390/mm/pgalloc.c index cb364153c43c..412c5f48a8e7 100644 --- a/arch/s390/mm/pgalloc.c +++ b/arch/s390/mm/pgalloc.c @@ -189,14 +189,15 @@ unsigned long *page_table_alloc(struct mm_struct *mm) if (!list_empty(&mm->context.pgtable_list)) { page = list_first_entry(&mm->context.pgtable_list, struct page, lru); - mask = atomic_read(&page->_mapcount); + mask = atomic_read(&page->_refcount) >> 24; mask = (mask | (mask >> 4)) & 3; if (mask != 3) { table = (unsigned long *) page_to_phys(page); bit = mask & 1; /* =1 -> second 2K */ if (bit) table += PTRS_PER_PTE; - atomic_xor_bits(&page->_mapcount, 1U << bit); + atomic_xor_bits(&page->_refcount, + 1U << (bit + 24)); list_del(&page->lru); } } @@ -217,12 +218,12 @@ unsigned long *page_table_alloc(struct mm_struct *mm) table = (unsigned long *) page_to_phys(page); if (mm_alloc_pgste(mm)) { /* Return 4K page table with PGSTEs */ - atomic_set(&page->_mapcount, 3); + atomic_xor_bits(&page->_refcount, 3 << 24); memset64((u64 *)table, _PAGE_INVALID, PTRS_PER_PTE); memset64((u64 *)table + PTRS_PER_PTE, 0, PTRS_PER_PTE); } else { /* Return the first 2K fragment of the page */ - atomic_set(&page->_mapcount, 1); + atomic_xor_bits(&page->_refcount, 1 << 24); memset64((u64 *)table, _PAGE_INVALID, 2 * PTRS_PER_PTE); spin_lock_bh(&mm->context.lock); list_add(&page->lru, &mm->context.pgtable_list); @@ -241,7 +242,8 @@ void page_table_free(struct mm_struct *mm, unsigned long *table) /* Free 2K page table fragment of a 4K page */ bit = (__pa(table) & ~PAGE_MASK)/(PTRS_PER_PTE*sizeof(pte_t)); spin_lock_bh(&mm->context.lock); - mask = atomic_xor_bits(&page->_mapcount, 1U << bit); + mask = atomic_xor_bits(&page->_refcount, 1U << (bit + 24)); + mask >>= 24; if (mask & 3) list_add(&page->lru, &mm->context.pgtable_list); else @@ -252,7 +254,6 @@ void page_table_free(struct mm_struct *mm, unsigned long *table) } pgtable_page_dtor(page); - atomic_set(&page->_mapcount, -1); __free_page(page); } @@ -273,7 +274,8 @@ void page_table_free_rcu(struct mmu_gather *tlb, unsigned long *table, } bit = (__pa(table) & ~PAGE_MASK) / (PTRS_PER_PTE*sizeof(pte_t)); spin_lock_bh(&mm->context.lock); - mask = atomic_xor_bits(&page->_mapcount, 0x11U << bit); + mask = atomic_xor_bits(&page->_refcount, 0x11U << (bit + 24)); + mask >>= 24; if (mask & 3) list_add_tail(&page->lru, &mm->context.pgtable_list); else @@ -295,12 +297,13 @@ static void __tlb_remove_table(void *_table) break; case 1: /* lower 2K of a 4K page table */ case 2: /* higher 2K of a 4K page table */ - if (atomic_xor_bits(&page->_mapcount, mask << 4) != 0) + mask = atomic_xor_bits(&page->_refcount, mask << (4 + 24)); + mask >>= 24; + if (mask != 0) break; /* fallthrough */ case 3: /* 4K page table with pgstes */ pgtable_page_dtor(page); - atomic_set(&page->_mapcount, -1); __free_page(page); break; } -- 2.16.1