Received: by 2002:a05:6a10:206:0:0:0:0 with SMTP id 6csp1401448pxj; Fri, 21 May 2021 13:21:25 -0700 (PDT) X-Google-Smtp-Source: ABdhPJxN4yll9W7kriSA9OKoZTe2ozKwqh1PF56YV1umz5r27R8ju6dGTW54dnP5wcd978gulZYh X-Received: by 2002:a02:5b0a:: with SMTP id g10mr2073642jab.2.1621628484386; Fri, 21 May 2021 13:21:24 -0700 (PDT) ARC-Seal: i=1; a=rsa-sha256; t=1621628484; cv=none; d=google.com; s=arc-20160816; b=mpyeyKNIECWADKalW6Tg6VhdGXtBgOHBDxZCZbi+MA2gYvAQ6f6OWlioKkgHdLTZA7 FAJHYs9XCDCg0/fJbiUfkfa5j9A819fKEW1bT++GEStgU1zN561Ht8ZYuDAOu8cE0qQp F+Rd7Drs6XsY/LG4/hggulZ1XuFYwkNPD7QHbyA0JPi/UdI4VCho2r0+NjBVvtNcsObY qNylFIRepekMISlvXYNf5UoVHIWopE66BWz5TOVpJqdtU8c6MP8dqZtCDfaLoYWDqvCC jIeVRA59wONTwMqsE8Hhyx/ScHpO/opejyhRHZ/XgnM/yuOpyz+P692TgNiZS5PF3DmR FzRQ== ARC-Message-Signature: i=1; a=rsa-sha256; c=relaxed/relaxed; d=google.com; s=arc-20160816; h=list-id:precedence:content-transfer-encoding:mime-version :references:in-reply-to:message-id:date:subject:cc:to:from; bh=DXxhNsSA0w48JzABDCQS0aHG0R3WQuhO/gqWHcWbvcI=; b=u9XNJgBPp2+2UZhV2HlT+2LnwEC+mZLUombR43MkIRjHZxYZ0oeY4pXcRZc6Gx6m87 CrLNpReTV7rhZoGdAuv937eGNRB8sBXICA+iBQWSt41SB9X6HXngOG9AWZMc+d7MeJy2 Em3p2qXakL+wL2HPPFbRE67YRS/V+wo1D7u0RWuBOEgLR1cn+fPrNztPzQLyu+SdlEb+ CoZo9pMHiLhZluSSmAqdQe35691gcxodo3vQ4L7QFoFhXuaeffpsoQp0MDwHsfxaz3+X bSUX2tih4KH6Mk/IY+HyLKtW8813AD2xv26yyl/jubL8bqNfox/Zclhq6/Oa47YGaQrU Cqqw== ARC-Authentication-Results: i=1; mx.google.com; spf=pass (google.com: domain of linux-kernel-owner@vger.kernel.org designates 23.128.96.18 as permitted sender) smtp.mailfrom=linux-kernel-owner@vger.kernel.org; dmarc=fail (p=NONE sp=NONE dis=NONE) header.from=linux.microsoft.com Return-Path: Received: from vger.kernel.org (vger.kernel.org. [23.128.96.18]) by mx.google.com with ESMTP id w16si6148868ilj.132.2021.05.21.13.21.11; Fri, 21 May 2021 13:21:24 -0700 (PDT) Received-SPF: pass (google.com: domain of linux-kernel-owner@vger.kernel.org designates 23.128.96.18 as permitted sender) client-ip=23.128.96.18; Authentication-Results: mx.google.com; spf=pass (google.com: domain of linux-kernel-owner@vger.kernel.org designates 23.128.96.18 as permitted sender) smtp.mailfrom=linux-kernel-owner@vger.kernel.org; dmarc=fail (p=NONE sp=NONE dis=NONE) header.from=linux.microsoft.com Received: (majordomo@vger.kernel.org) by vger.kernel.org via listexpand id S236108AbhEUQRX (ORCPT + 99 others); Fri, 21 May 2021 12:17:23 -0400 Received: from mail-ed1-f53.google.com ([209.85.208.53]:39463 "EHLO mail-ed1-f53.google.com" rhost-flags-OK-OK-OK-OK) by vger.kernel.org with ESMTP id S231995AbhEUQRW (ORCPT ); Fri, 21 May 2021 12:17:22 -0400 Received: by mail-ed1-f53.google.com with SMTP id h16so23913754edr.6; Fri, 21 May 2021 09:15:58 -0700 (PDT) X-Google-DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=1e100.net; s=20161025; h=x-gm-message-state:from:to:cc:subject:date:message-id:in-reply-to :references:mime-version:content-transfer-encoding; bh=DXxhNsSA0w48JzABDCQS0aHG0R3WQuhO/gqWHcWbvcI=; b=T7sRSbMZgCMxZ8xMfZ4Ky+e3YYdBJ5yeY5JxS3yiC2JK86fCOr3pfui5UxCoNLMcke ZyusgyR2XNw0NC3riR85L2q2FnDlANOcnbp4bItkOnlcRtHNkrv8VYYl8wopnlxSSt+J WPULU5wr57UA/sdAzshGET/GhlwTnKBpBaZ3i99BNwdYr3d4TSqFYfFWhacO35TW/4rJ SG5bBK3eAGbxiFlzPFQ3MBOw+DG8hB/3CGeJF7FeOi1Ku6kE/HgZAw3izRltqhmxHuY/ 5u67CkGkDI8QMrSlXM8Ma1go161UYB1QBzXQnMibFiAaBPH5EiR64Wcz3bxOXRj+Yurz eu4g== X-Gm-Message-State: AOAM533bM4ImnDiOMiZRRBeaCjMknSwN81+MuiLD/GU6pVNz65fluYFM Vv9ItitugBgef3auiUePieoIgJa0ApZo+aps X-Received: by 2002:a50:fd13:: with SMTP id i19mr12078450eds.386.1621613757321; Fri, 21 May 2021 09:15:57 -0700 (PDT) Received: from msft-t490s.teknoraver.net (net-5-94-253-60.cust.vodafonedsl.it. [5.94.253.60]) by smtp.gmail.com with ESMTPSA id f7sm3871644ejz.95.2021.05.21.09.15.55 (version=TLS1_3 cipher=TLS_AES_256_GCM_SHA384 bits=256/256); Fri, 21 May 2021 09:15:56 -0700 (PDT) From: Matteo Croce To: netdev@vger.kernel.org, linux-mm@kvack.org Cc: Ayush Sawal , Vinay Kumar Yadav , Rohit Maheshwari , "David S. Miller" , Jakub Kicinski , Thomas Petazzoni , Marcin Wojtas , Russell King , Mirko Lindner , Stephen Hemminger , Tariq Toukan , Jesper Dangaard Brouer , Ilias Apalodimas , Alexei Starovoitov , Daniel Borkmann , John Fastabend , Boris Pismenny , Arnd Bergmann , Andrew Morton , "Peter Zijlstra (Intel)" , Vlastimil Babka , Yu Zhao , Will Deacon , Fenghua Yu , Roman Gushchin , Hugh Dickins , Peter Xu , Jason Gunthorpe , Jonathan Lemon , Alexander Lobakin , Cong Wang , wenxu , Kevin Hao , Jakub Sitnicki , Marco Elver , Willem de Bruijn , Miaohe Lin , Yunsheng Lin , Guillaume Nault , linux-kernel@vger.kernel.org, linux-rdma@vger.kernel.org, bpf@vger.kernel.org, Matthew Wilcox , Eric Dumazet , David Ahern , Lorenzo Bianconi , Saeed Mahameed , Andrew Lunn , Paolo Abeni , Sven Auhagen Subject: [PATCH net-next v6 1/5] mm: add a signature in struct page Date: Fri, 21 May 2021 18:15:23 +0200 Message-Id: <20210521161527.34607-2-mcroce@linux.microsoft.com> X-Mailer: git-send-email 2.31.1 In-Reply-To: <20210521161527.34607-1-mcroce@linux.microsoft.com> References: <20210521161527.34607-1-mcroce@linux.microsoft.com> MIME-Version: 1.0 Content-Transfer-Encoding: 8bit Precedence: bulk List-ID: X-Mailing-List: linux-kernel@vger.kernel.org From: Matteo Croce This is needed by the page_pool to avoid recycling a page not allocated via page_pool. The page->signature field is aliased to page->lru.next and page->compound_head, but it can't be set by mistake because the signature value is a bad pointer, and can't trigger a false positive in PageTail() because the last bit is 0. Co-developed-by: Matthew Wilcox (Oracle) Signed-off-by: Matthew Wilcox (Oracle) Signed-off-by: Matteo Croce --- include/linux/mm.h | 12 +++++++----- include/linux/mm_types.h | 12 +++++++++++- include/linux/poison.h | 3 +++ net/core/page_pool.c | 6 ++++++ 4 files changed, 27 insertions(+), 6 deletions(-) diff --git a/include/linux/mm.h b/include/linux/mm.h index 322ec61d0da7..4ecfd8472a17 100644 --- a/include/linux/mm.h +++ b/include/linux/mm.h @@ -1668,10 +1668,12 @@ struct address_space *page_mapping(struct page *page); static inline bool page_is_pfmemalloc(const struct page *page) { /* - * Page index cannot be this large so this must be - * a pfmemalloc page. + * This is not a tail page; compound_head of a head page is unused + * at return from the page allocator, and will be overwritten + * by callers who do not care whether the page came from the + * reserves. */ - return page->index == -1UL; + return page->compound_head & BIT(1); } /* @@ -1680,12 +1682,12 @@ static inline bool page_is_pfmemalloc(const struct page *page) */ static inline void set_page_pfmemalloc(struct page *page) { - page->index = -1UL; + page->compound_head = BIT(1); } static inline void clear_page_pfmemalloc(struct page *page) { - page->index = 0; + page->compound_head = 0; } /* diff --git a/include/linux/mm_types.h b/include/linux/mm_types.h index 5aacc1c10a45..09f90598ff63 100644 --- a/include/linux/mm_types.h +++ b/include/linux/mm_types.h @@ -96,6 +96,13 @@ struct page { unsigned long private; }; struct { /* page_pool used by netstack */ + /** + * @pp_magic: magic value to avoid recycling non + * page_pool allocated pages. + */ + unsigned long pp_magic; + struct page_pool *pp; + unsigned long _pp_mapping_pad; /** * @dma_addr: might require a 64-bit value on * 32-bit architectures. @@ -130,7 +137,10 @@ struct page { }; }; struct { /* Tail pages of compound page */ - unsigned long compound_head; /* Bit zero is set */ + /* Bit zero is set + * Bit one if pfmemalloc page + */ + unsigned long compound_head; /* First tail page only */ unsigned char compound_dtor; diff --git a/include/linux/poison.h b/include/linux/poison.h index aff1c9250c82..d62ef5a6b4e9 100644 --- a/include/linux/poison.h +++ b/include/linux/poison.h @@ -78,4 +78,7 @@ /********** security/ **********/ #define KEY_DESTROY 0xbd +/********** net/core/page_pool.c **********/ +#define PP_SIGNATURE (0x40 + POISON_POINTER_DELTA) + #endif diff --git a/net/core/page_pool.c b/net/core/page_pool.c index 3c4c4c7a0402..e1321bc9d316 100644 --- a/net/core/page_pool.c +++ b/net/core/page_pool.c @@ -17,6 +17,7 @@ #include #include #include /* for __put_page() */ +#include #include @@ -221,6 +222,8 @@ static struct page *__page_pool_alloc_page_order(struct page_pool *pool, return NULL; } + page->pp_magic |= PP_SIGNATURE; + /* Track how many pages are held 'in-flight' */ pool->pages_state_hold_cnt++; trace_page_pool_state_hold(pool, page, pool->pages_state_hold_cnt); @@ -263,6 +266,7 @@ static struct page *__page_pool_alloc_pages_slow(struct page_pool *pool, put_page(page); continue; } + page->pp_magic |= PP_SIGNATURE; pool->alloc.cache[pool->alloc.count++] = page; /* Track how many pages are held 'in-flight' */ pool->pages_state_hold_cnt++; @@ -341,6 +345,8 @@ void page_pool_release_page(struct page_pool *pool, struct page *page) DMA_ATTR_SKIP_CPU_SYNC); page_pool_set_dma_addr(page, 0); skip_dma_unmap: + page->pp_magic = 0; + /* This may be the last page returned, releasing the pool, so * it is not safe to reference pool afterwards. */ -- 2.31.1