Received: by 2002:a05:6a10:6d10:0:0:0:0 with SMTP id gq16csp3782404pxb; Tue, 19 Apr 2022 09:42:27 -0700 (PDT) X-Google-Smtp-Source: ABdhPJxyqUprldftT68jSnixV8SMhqB/5c4QLKXzb4q6B73tcR04gGTbLNxZQAJBmpySJWaj4sF5 X-Received: by 2002:a17:907:7246:b0:6ef:828d:4b49 with SMTP id ds6-20020a170907724600b006ef828d4b49mr13557648ejc.172.1650386547527; Tue, 19 Apr 2022 09:42:27 -0700 (PDT) ARC-Seal: i=1; a=rsa-sha256; t=1650386547; cv=none; d=google.com; s=arc-20160816; b=R5zsW1izAUVN3vVd/Kh5KAfdm//LiNk1ewfFcqrIVcO+p3+DK0ROrmlkga7nb8nf9h tKpKftKbXQWb6eLrXy5Up+mRSLvYvAOHLHMwaYnZasA//phUYR+4xhITEofv3ZWI75em TZnnpXyTGDF8GKT3oablUFSpwe01ZW67YJqE471Okk9Aew8QjaD5VG9QrhD2E/sXTTNz 6+SgbO1cz1xtFIcv1sYpwDYEbKojvhQKCpxr/tdl1x2waewxTNsHwIIfEZRawtzQGZg+ NezEv27qBWmIoPBcEuxYtetqTdKXYZ+T/Rd031r0tGU9xOnqmSb5Fl5dZmJDUdiCflsf IZ0g== ARC-Message-Signature: i=1; a=rsa-sha256; c=relaxed/relaxed; d=google.com; s=arc-20160816; h=list-id:precedence:content-transfer-encoding:in-reply-to:from :references:cc:to:content-language:subject:user-agent:mime-version :date:message-id; bh=XzSqS92uyCNhv3DHnFXSPb5muLGikjL76HyMUw/4M/8=; b=gOvC8HAr7G6MVSz2e3/LWrtNpS8CKwtArEoBCzY0iu4HwChHGpV1I7cHnMKT05qq0m mcwUoehUBv3Eqbz5b+268GEEg7zqPrnkFIHD1oHSgAWZ9ukk9puRKldD02hCFt9U6RZR 7KCHuraQtKD/GsZmeHgoPUfpWC+33nd15gVraOouHajEb1ak5IZ67HGAtjJfW+gkYDIo l6I4h4/C6zJd4LhhopvEAC3vz9/njVpvterjVN+YnQDnPS5iXx3nHQJgvsqIAsmM088S 3oNUdciDegnAz5AxN6TLqqKaEVQAq7SuBQHILkPtx7R3ZS9mOg5so2Jq0wCHfJitss7p mogw== ARC-Authentication-Results: i=1; mx.google.com; spf=pass (google.com: domain of linux-kernel-owner@vger.kernel.org designates 2620:137:e000::1:20 as permitted sender) smtp.mailfrom=linux-kernel-owner@vger.kernel.org; dmarc=fail (p=NONE sp=NONE dis=NONE) header.from=arm.com Return-Path: Received: from out1.vger.email (out1.vger.email. [2620:137:e000::1:20]) by mx.google.com with ESMTP id b9-20020a056402350900b004240849e896si789270edd.283.2022.04.19.09.42.03; Tue, 19 Apr 2022 09:42:27 -0700 (PDT) Received-SPF: pass (google.com: domain of linux-kernel-owner@vger.kernel.org designates 2620:137:e000::1:20 as permitted sender) client-ip=2620:137:e000::1:20; Authentication-Results: mx.google.com; spf=pass (google.com: domain of linux-kernel-owner@vger.kernel.org designates 2620:137:e000::1:20 as permitted sender) smtp.mailfrom=linux-kernel-owner@vger.kernel.org; dmarc=fail (p=NONE sp=NONE dis=NONE) header.from=arm.com Received: (majordomo@vger.kernel.org) by vger.kernel.org via listexpand id S1349296AbiDSH1J (ORCPT + 99 others); Tue, 19 Apr 2022 03:27:09 -0400 Received: from lindbergh.monkeyblade.net ([23.128.96.19]:47444 "EHLO lindbergh.monkeyblade.net" rhost-flags-OK-OK-OK-OK) by vger.kernel.org with ESMTP id S232480AbiDSH1H (ORCPT ); Tue, 19 Apr 2022 03:27:07 -0400 Received: from foss.arm.com (foss.arm.com [217.140.110.172]) by lindbergh.monkeyblade.net (Postfix) with ESMTP id C3854326EA for ; Tue, 19 Apr 2022 00:24:25 -0700 (PDT) Received: from usa-sjc-imap-foss1.foss.arm.com (unknown [10.121.207.14]) by usa-sjc-mx-foss1.foss.arm.com (Postfix) with ESMTP id 78E55106F; Tue, 19 Apr 2022 00:24:25 -0700 (PDT) Received: from [10.163.40.223] (unknown [10.163.40.223]) by usa-sjc-imap-foss1.foss.arm.com (Postfix) with ESMTPSA id C1E7D3F766; Tue, 19 Apr 2022 00:24:17 -0700 (PDT) Message-ID: <99df7899-1820-d5b0-a72a-97dab1364075@arm.com> Date: Tue, 19 Apr 2022 12:55:01 +0530 MIME-Version: 1.0 User-Agent: Mozilla/5.0 (X11; Linux x86_64; rv:91.0) Gecko/20100101 Thunderbird/91.5.0 Subject: Re: [PATCH -next v4 3/4] arm64: mm: add support for page table check Content-Language: en-US To: Pasha Tatashin , Tong Tiangen Cc: Thomas Gleixner , Ingo Molnar , Borislav Petkov , Dave Hansen , "maintainer:X86 ARCHITECTURE (32-BIT AND 64-BIT)" , "H. Peter Anvin" , Andrew Morton , Catalin Marinas , Will Deacon , Paul Walmsley , Palmer Dabbelt , Albert Ou , LKML , linux-mm , Linux ARM , linux-riscv@lists.infradead.org, Kefeng Wang , Guohanjun References: <20220418034444.520928-1-tongtiangen@huawei.com> <20220418034444.520928-4-tongtiangen@huawei.com> <1c314feb-cd78-2bb3-462e-4ea3cefe122e@arm.com> From: Anshuman Khandual In-Reply-To: Content-Type: text/plain; charset=UTF-8 Content-Transfer-Encoding: 8bit X-Spam-Status: No, score=-10.2 required=5.0 tests=BAYES_00,NICE_REPLY_A, RCVD_IN_DNSWL_HI,SPF_HELO_NONE,SPF_PASS,T_SCC_BODY_TEXT_LINE autolearn=ham autolearn_force=no version=3.4.6 X-Spam-Checker-Version: SpamAssassin 3.4.6 (2021-04-09) on lindbergh.monkeyblade.net Precedence: bulk List-ID: X-Mailing-List: linux-kernel@vger.kernel.org On 4/18/22 21:50, Pasha Tatashin wrote: > On Mon, Apr 18, 2022 at 11:47 AM Tong Tiangen wrote: >> >> >> >> 在 2022/4/18 17:28, Anshuman Khandual 写道: >>> On 4/18/22 09:14, Tong Tiangen wrote: >>>> From: Kefeng Wang >>>> >> [...] >>>> #endif >>> >>> Ran this series on arm64 platform after enabling >>> >>> - CONFIG_PAGE_TABLE_CHECK >>> - CONFIG_PAGE_TABLE_CHECK_ENFORCED (avoiding kernel command line option) >>> >>> After some time, the following error came up >>> >>> [ 23.266013] ------------[ cut here ]------------ >>> [ 23.266807] kernel BUG at mm/page_table_check.c:90! >>> [ 23.267609] Internal error: Oops - BUG: 0 [#1] PREEMPT SMP >>> [ 23.268503] Modules linked in: >>> [ 23.269012] CPU: 1 PID: 30 Comm: khugepaged Not tainted 5.18.0-rc3-00004-g60aa8e363a91 #2 >>> [ 23.270383] Hardware name: linux,dummy-virt (DT) >>> [ 23.271210] pstate: 40400005 (nZcv daif +PAN -UAO -TCO -DIT -SSBS BTYPE=--) >>> [ 23.272445] pc : page_table_check_clear.isra.6+0x114/0x148 >>> [ 23.273429] lr : page_table_check_clear.isra.6+0x64/0x148 >>> [ 23.274395] sp : ffff80000afb3ca0 >>> [ 23.274994] x29: ffff80000afb3ca0 x28: fffffc00022558e8 x27: ffff80000a27f628 >>> [ 23.276260] x26: ffff800009f9f2b0 x25: ffff00008a8d5000 x24: ffff800009f09fa0 >>> [ 23.277527] x23: 0000ffff89e00000 x22: ffff800009f09fb8 x21: ffff000089414cc0 >>> [ 23.278798] x20: 0000000000000200 x19: fffffc00022a0000 x18: 0000000000000001 >>> [ 23.280066] x17: 0000000000000001 x16: 0000000000000000 x15: 0000000000000003 >>> [ 23.281331] x14: 0000000000000068 x13: 00000000000000c0 x12: 0000000000000010 >>> [ 23.282602] x11: fffffc0002320008 x10: fffffc0002320000 x9 : ffff800009fa1000 >>> [ 23.283868] x8 : 00000000ffffffff x7 : 0000000000000001 x6 : ffff800009fa1f08 >>> [ 23.285135] x5 : 0000000000000000 x4 : 0000000000000000 x3 : 0000000000000000 >>> [ 23.286406] x2 : 00000000ffffffff x1 : ffff000080f2800c x0 : ffff000080f28000 >>> [ 23.287673] Call trace: >>> [ 23.288123] page_table_check_clear.isra.6+0x114/0x148 >>> [ 23.289043] __page_table_check_pmd_clear+0x3c/0x50 >>> [ 23.289918] pmdp_collapse_flush+0x114/0x370 >>> [ 23.290692] khugepaged+0x1170/0x19e0 >>> [ 23.291356] kthread+0x110/0x120 >>> [ 23.291945] ret_from_fork+0x10/0x20 >>> [ 23.292596] Code: 91001041 b8e80024 51000482 36fffd62 (d4210000) >>> [ 23.293678] ---[ end trace 0000000000000000 ]--- >>> [ 23.294511] note: khugepaged[30] exited with preempt_count 2 >>> >>> Looking into file mm/page_table_check.c where this problem occured. >>> >>> /* >>> * An enty is removed from the page table, decrement the counters for that page >>> * verify that it is of correct type and counters do not become negative. >>> */ >>> static void page_table_check_clear(struct mm_struct *mm, unsigned long addr, >>> unsigned long pfn, unsigned long pgcnt) >>> { >>> struct page_ext *page_ext; >>> struct page *page; >>> unsigned long i; >>> bool anon; >>> >>> if (!pfn_valid(pfn)) >>> return; >>> >>> page = pfn_to_page(pfn); >>> page_ext = lookup_page_ext(page); >>> anon = PageAnon(page); >>> >>> for (i = 0; i < pgcnt; i++) { >>> struct page_table_check *ptc = get_page_table_check(page_ext); >>> >>> if (anon) { >>> BUG_ON(atomic_read(&ptc->file_map_count)); >>> BUG_ON(atomic_dec_return(&ptc->anon_map_count) < 0); >>> } else { >>> BUG_ON(atomic_read(&ptc->anon_map_count)); >>> Triggered here ====>> BUG_ON(atomic_dec_return(&ptc->file_map_count) < 0); >>> } >>> page_ext = page_ext_next(page_ext); >>> } >>> } >>> >>> Could you explain what was expected during pmdp_collapse_flush() which when >>> failed, triggered this BUG_ON() ? This counter seems to be page table check >>> specific, could it just go wrong ? I have not looked into the details about >>> page table check mechanism. >>> >>> - Anshuman >>> . >> >> Hi Anshuman: >> >> Thanks for your job. >> >> Let me briefly explain the principle of page table check(PTC). >> >> PTC introduces the following struct for page mapping type count: >> struct page_table_check { >> atomic_t anon_map_count; >> atomic_t file_map_count; >> }; >> This structure can be obtained by "lookup_page_ext(page)" >> >> When page table entries are set(pud/pmd/pte), page_table_check_set() is >> called to increase the page mapping count, Also check for errors (eg:if >> a page is used for anonymous mapping, then the page cannot be used for >> file mapping at the same time). >> >> When page table entries are clear(pud/pmd/pte), page_table_check_clear() >> is called to decrease the page mapping count, Also check for errors. >> >> The error check rules are described in the following documents: >> Documentation/vm/page_table_check.rst >> >> The setting and clearing of page table entries are symmetrical. >> >> Here __page_table_check_pmd_clear() trigger BUGON which indicates that >> the pmd entry file mapping count has become negative. >> >> I guess if PTC didn't detect this exception, would there have been any >> problems? > > It is hard to tell what sort of problem has been detected. More > debugging is needed in order to understand it. A huge file entry is > being removed from the page table. However, at least one sub page of > that entry does not have a record that it was added as a file entry to I guess PMD splitting scenarios should also be taken care as sub pages will also go via appropriate XXX_set_at() helpers ? > the page table. At Google we found a few internal security bugs using > PTCs. However, this being new on ARM64, it is possible that the bug is > in PTC/khugepaged itself. > > Anshuman is it possible to repro your scenario in QEMU? I have been unable to reproduce this reported problem. Last time it just happened after a fresh boot without anything in particular running. Will continue experimenting.