Received: by 2002:a25:ad19:0:0:0:0:0 with SMTP id y25csp1703070ybi; Wed, 17 Jul 2019 20:25:28 -0700 (PDT) X-Google-Smtp-Source: APXvYqylu2rTZnZ6Oyqw7UZEvjwLEFqTYSMykSKS1Rab6MwoI+y+/bEGTBJhC0Z1Q5gs9c+xjx/b X-Received: by 2002:a17:902:f204:: with SMTP id gn4mr47865258plb.3.1563420328514; Wed, 17 Jul 2019 20:25:28 -0700 (PDT) ARC-Seal: i=1; a=rsa-sha256; t=1563420328; cv=none; d=google.com; s=arc-20160816; b=bIAlAtK9PQbswBw1WgyCAUDxHdyL+Ulw2rDJrfPNM5klmJk9u072QTpsNTJUsRGuIl 5W80elOX+vRcd1/56ccjlIOyPj8YygKQz96VPqH3RDoguqNwijYNiamZE+zu1ELsr9Zf RSW4FeofYd8+M0/RPICXXnx/lxJzgd0SZKjNkVRS76H+JU2zAA9SuizSyjbT9HDJdaLs ceD4s3eZ2fXtjNUmQ9IUFGYMmi2a0lpGGt0YxxIpPh92x6Xj7SYLISMxr5q+CCMxgPcv dipd7BgOUlh9erD8YiLC8Ohgr7wjzMhsxFLT4vw38gtA3IT9R7iq3n115KrjYz+xTOwQ ssXQ== ARC-Message-Signature: i=1; a=rsa-sha256; c=relaxed/relaxed; d=google.com; s=arc-20160816; h=list-id:precedence:sender:content-transfer-encoding:mime-version :user-agent:references:in-reply-to:message-id:date:subject:cc:to :from:dkim-signature; bh=l69UY9+u9sdK3k/cjLmSt7I/3sI24OUxW4CkYphA/bE=; b=DQZIvgZFBIFhfuGXI9vZiASyuB28anmIRemUr8pHSE/ydBkNAI0MUT+XIL1BAnwjkm QUG4b9Fdhwc0cY3c7vNpc0ZW4ZMKUxDio5nQ4PVMU9YBo1XWvCZrQ0tOkxCQFXiiSuHl nQGTQJt1YfG/uMfeXj14/RQG200as4fbbb0hOPAEAgdRJK/GAoNJVcmdy9va3mNYikII T8NCdd9Qk5VIZlo8+XbOCAO3tSfRH8c0/wIM/B6OD0AMkW7SJMYkyBaBUMdwhx1mw8kT k2szBcEmIc5JYpoYl21xCNC2amlo6wC2okqh+vbPZ/gpsdKlRgTkHOcTBBwSOO6tg9bz mTfQ== ARC-Authentication-Results: i=1; mx.google.com; dkim=pass header.i=@kernel.org header.s=default header.b=T8y6oqNw; spf=pass (google.com: best guess record for domain of linux-kernel-owner@vger.kernel.org designates 209.132.180.67 as permitted sender) smtp.mailfrom=linux-kernel-owner@vger.kernel.org Return-Path: Received: from vger.kernel.org (vger.kernel.org. [209.132.180.67]) by mx.google.com with ESMTP id e2si25452463pgf.256.2019.07.17.20.25.12; Wed, 17 Jul 2019 20:25:28 -0700 (PDT) Received-SPF: pass (google.com: best guess record for domain of linux-kernel-owner@vger.kernel.org designates 209.132.180.67 as permitted sender) client-ip=209.132.180.67; Authentication-Results: mx.google.com; dkim=pass header.i=@kernel.org header.s=default header.b=T8y6oqNw; spf=pass (google.com: best guess record for domain of linux-kernel-owner@vger.kernel.org designates 209.132.180.67 as permitted sender) smtp.mailfrom=linux-kernel-owner@vger.kernel.org Received: (majordomo@vger.kernel.org) by vger.kernel.org via listexpand id S2390393AbfGRDHS (ORCPT + 99 others); Wed, 17 Jul 2019 23:07:18 -0400 Received: from mail.kernel.org ([198.145.29.99]:38640 "EHLO mail.kernel.org" rhost-flags-OK-OK-OK-OK) by vger.kernel.org with ESMTP id S2390360AbfGRDHM (ORCPT ); Wed, 17 Jul 2019 23:07:12 -0400 Received: from localhost (115.42.148.210.bf.2iij.net [210.148.42.115]) (using TLSv1.2 with cipher ECDHE-RSA-AES256-GCM-SHA384 (256/256 bits)) (No client certificate requested) by mail.kernel.org (Postfix) with ESMTPSA id BAD642053B; Thu, 18 Jul 2019 03:07:10 +0000 (UTC) DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/simple; d=kernel.org; s=default; t=1563419231; bh=baBU9P10qlX5IXyVCz2/GKUVrIU3uPv5UWjez8prMQU=; h=From:To:Cc:Subject:Date:In-Reply-To:References:From; b=T8y6oqNws5iTDLNQk9fQuqZI8Y6PuJPZlcwRi2GAt0+cLegUbWkq0wM+/ejOsl0pF 80ovCbt3TVKvsfnpDOQiD1eSutrZNeIrl3r4+OPf/U7B9lOtXL0I2jznM7Xzy/INXS CkkZVJ64SJ9T+3w6pAOQpmOk1Uf7QXNcrwduDsrI= From: Greg Kroah-Hartman To: linux-kernel@vger.kernel.org Cc: Greg Kroah-Hartman , stable@vger.kernel.org, "Kirill A. Shutemov" , Thomas Gleixner , Borislav Petkov , "H. Peter Anvin" , Dave Hansen , Andy Lutomirski , Peter Zijlstra , Sasha Levin Subject: [PATCH 4.19 22/47] x86/boot/64: Fix crash if kernel image crosses page table boundary Date: Thu, 18 Jul 2019 12:01:36 +0900 Message-Id: <20190718030050.584579412@linuxfoundation.org> X-Mailer: git-send-email 2.22.0 In-Reply-To: <20190718030045.780672747@linuxfoundation.org> References: <20190718030045.780672747@linuxfoundation.org> User-Agent: quilt/0.66 MIME-Version: 1.0 Content-Type: text/plain; charset=UTF-8 Content-Transfer-Encoding: 8bit Sender: linux-kernel-owner@vger.kernel.org Precedence: bulk List-ID: X-Mailing-List: linux-kernel@vger.kernel.org [ Upstream commit 81c7ed296dcd02bc0b4488246d040e03e633737a ] A kernel which boots in 5-level paging mode crashes in a small percentage of cases if KASLR is enabled. This issue was tracked down to the case when the kernel image unpacks in a way that it crosses an 1G boundary. The crash is caused by an overrun of the PMD page table in __startup_64() and corruption of P4D page table allocated next to it. This particular issue is not visible with 4-level paging as P4D page tables are not used. But the P4D and the PUD calculation have similar problems. The PMD index calculation is wrong due to operator precedence, which fails to confine the PMDs in the PMD array on wrap around. The P4D calculation for 5-level paging and the PUD calculation calculate the first index correctly, but then blindly increment it which causes the same issue when a kernel image is located across a 512G and for 5-level paging across a 46T boundary. This wrap around mishandling was introduced when these parts moved from assembly to C. Restore it to the correct behaviour. Fixes: c88d71508e36 ("x86/boot/64: Rewrite startup_64() in C") Signed-off-by: Kirill A. Shutemov Signed-off-by: Thomas Gleixner Cc: Borislav Petkov Cc: "H. Peter Anvin" Cc: Dave Hansen Cc: Andy Lutomirski Cc: Peter Zijlstra Link: https://lkml.kernel.org/r/20190620112345.28833-1-kirill.shutemov@linux.intel.com Signed-off-by: Sasha Levin --- arch/x86/kernel/head64.c | 17 +++++++++-------- 1 file changed, 9 insertions(+), 8 deletions(-) diff --git a/arch/x86/kernel/head64.c b/arch/x86/kernel/head64.c index ddee1f0870c4..cc5b519dc687 100644 --- a/arch/x86/kernel/head64.c +++ b/arch/x86/kernel/head64.c @@ -190,18 +190,18 @@ unsigned long __head __startup_64(unsigned long physaddr, pgd[i + 0] = (pgdval_t)p4d + pgtable_flags; pgd[i + 1] = (pgdval_t)p4d + pgtable_flags; - i = (physaddr >> P4D_SHIFT) % PTRS_PER_P4D; - p4d[i + 0] = (pgdval_t)pud + pgtable_flags; - p4d[i + 1] = (pgdval_t)pud + pgtable_flags; + i = physaddr >> P4D_SHIFT; + p4d[(i + 0) % PTRS_PER_P4D] = (pgdval_t)pud + pgtable_flags; + p4d[(i + 1) % PTRS_PER_P4D] = (pgdval_t)pud + pgtable_flags; } else { i = (physaddr >> PGDIR_SHIFT) % PTRS_PER_PGD; pgd[i + 0] = (pgdval_t)pud + pgtable_flags; pgd[i + 1] = (pgdval_t)pud + pgtable_flags; } - i = (physaddr >> PUD_SHIFT) % PTRS_PER_PUD; - pud[i + 0] = (pudval_t)pmd + pgtable_flags; - pud[i + 1] = (pudval_t)pmd + pgtable_flags; + i = physaddr >> PUD_SHIFT; + pud[(i + 0) % PTRS_PER_PUD] = (pudval_t)pmd + pgtable_flags; + pud[(i + 1) % PTRS_PER_PUD] = (pudval_t)pmd + pgtable_flags; pmd_entry = __PAGE_KERNEL_LARGE_EXEC & ~_PAGE_GLOBAL; /* Filter out unsupported __PAGE_KERNEL_* bits: */ @@ -211,8 +211,9 @@ unsigned long __head __startup_64(unsigned long physaddr, pmd_entry += physaddr; for (i = 0; i < DIV_ROUND_UP(_end - _text, PMD_SIZE); i++) { - int idx = i + (physaddr >> PMD_SHIFT) % PTRS_PER_PMD; - pmd[idx] = pmd_entry + i * PMD_SIZE; + int idx = i + (physaddr >> PMD_SHIFT); + + pmd[idx % PTRS_PER_PMD] = pmd_entry + i * PMD_SIZE; } /* -- 2.20.1