Received: by 2002:a05:7412:cfc7:b0:fc:a2b0:25d7 with SMTP id by7csp2303725rdb; Wed, 21 Feb 2024 03:38:35 -0800 (PST) X-Forwarded-Encrypted: i=3; AJvYcCVowMK7xaCcZEpJwUUGroqpYacyCC9rEVCbKoF1Iepp6LM2sTVCfC6enMbHqaruxGcGV7+u0p70/AbQB93n50a3gD4GeAj6n+0Ly3Z+SA== X-Google-Smtp-Source: AGHT+IG+7Fda+WnPy19fYjDJoQiiV9ox4+KXau9tXvLEvHo4jgg8dJY0l5awLdwhrk3zOkiElEmj X-Received: by 2002:a05:6402:43c5:b0:564:9628:2e2f with SMTP id p5-20020a05640243c500b0056496282e2fmr7194636edc.17.1708515515786; Wed, 21 Feb 2024 03:38:35 -0800 (PST) ARC-Seal: i=2; a=rsa-sha256; t=1708515515; cv=pass; d=google.com; s=arc-20160816; b=Efr50bgh0LxRkt9HgGNMkpdCZnR3irYhp1But5THQXvbl1qJu2YWwFhsl75DD/rWKH ahRvef0LahZTzxqWt3OWrEJEx/DI3sQIxxKgayoD2jqVZjDmrlFJo7OzcPrumS6jJABZ IGD3Kw+AxrLTi5kGVORgkmwKRm2LjOv32PNKAWU32p3aeK7LH+a+1jJMtaiY63qMz7C8 DdNw82YBiYyfb6/YUfLhKSH1Q8RJXBB6xCwd8894+6EtnfDLdPE3oCnTZ/9VctxthuE2 UjFF3FNmGjAcIh9HRP2UQ9pD/nzDFh8HIcLkNo/sYtxqnJS3QndYsUtciRw0j7hRKl3S Djng== ARC-Message-Signature: i=2; a=rsa-sha256; c=relaxed/relaxed; d=google.com; s=arc-20160816; h=cc:to:from:subject:message-id:references:mime-version :list-unsubscribe:list-subscribe:list-id:precedence:in-reply-to:date :dkim-signature; bh=StiHl4wqKJLROSGDCkKhKFhOVcRJSTp0SGrZr/uAC14=; fh=fMhMPvo8duafGkM4xZHdH7fULbau3etefTGsKa+Qy7I=; b=h8eHt/NB2wnoA4sD/nbmVEu+0ocvUT1eV69amjTY7QcRxeU/4uvZoAGZjUvRSuyoNb jv/P9YGQVj1J3YT/kiDMAWOwvUgqvy9YINGxSCDybV7li8u3+0NNko8CVnWVnsdyyrRu 9oCRKBhkhITXx40qes5H/l9TUitHKi61owf0bUcZqqH8FR+qY4KBfOFJnFVo05SN0EIO 5W6RBnaSXdtQjfqT5OUx488xVPVzoKtvIhXJ9na3E9SaA6uUCRmM4a0r01VBG72dHNkY evps94xTCvFqcFC64oXJPGgkFuycGWeB+FdTqmGfXn2djlszTeKjyet59GKiDoTskJt0 2MDw==; dara=google.com ARC-Authentication-Results: i=2; mx.google.com; dkim=pass header.i=@google.com header.s=20230601 header.b=nOEY4Zuw; arc=pass (i=1 spf=pass spfdomain=flex--ardb.bounces.google.com dkim=pass dkdomain=google.com dmarc=pass fromdomain=google.com); spf=pass (google.com: domain of linux-kernel+bounces-74639-linux.lists.archive=gmail.com@vger.kernel.org designates 2604:1380:4601:e00::3 as permitted sender) smtp.mailfrom="linux-kernel+bounces-74639-linux.lists.archive=gmail.com@vger.kernel.org"; dmarc=pass (p=REJECT sp=REJECT dis=NONE) header.from=google.com Return-Path: Received: from am.mirrors.kernel.org (am.mirrors.kernel.org. [2604:1380:4601:e00::3]) by mx.google.com with ESMTPS id t28-20020a50d71c000000b005645b115772si3146762edi.220.2024.02.21.03.38.35 for (version=TLS1_3 cipher=TLS_AES_256_GCM_SHA384 bits=256/256); Wed, 21 Feb 2024 03:38:35 -0800 (PST) Received-SPF: pass (google.com: domain of linux-kernel+bounces-74639-linux.lists.archive=gmail.com@vger.kernel.org designates 2604:1380:4601:e00::3 as permitted sender) client-ip=2604:1380:4601:e00::3; Authentication-Results: mx.google.com; dkim=pass header.i=@google.com header.s=20230601 header.b=nOEY4Zuw; arc=pass (i=1 spf=pass spfdomain=flex--ardb.bounces.google.com dkim=pass dkdomain=google.com dmarc=pass fromdomain=google.com); spf=pass (google.com: domain of linux-kernel+bounces-74639-linux.lists.archive=gmail.com@vger.kernel.org designates 2604:1380:4601:e00::3 as permitted sender) smtp.mailfrom="linux-kernel+bounces-74639-linux.lists.archive=gmail.com@vger.kernel.org"; dmarc=pass (p=REJECT sp=REJECT dis=NONE) header.from=google.com Received: from smtp.subspace.kernel.org (wormhole.subspace.kernel.org [52.25.139.140]) (using TLSv1.2 with cipher ECDHE-RSA-AES256-GCM-SHA384 (256/256 bits)) (No client certificate requested) by am.mirrors.kernel.org (Postfix) with ESMTPS id 3E2951F2232C for ; Wed, 21 Feb 2024 11:38:35 +0000 (UTC) Received: from localhost.localdomain (localhost.localdomain [127.0.0.1]) by smtp.subspace.kernel.org (Postfix) with ESMTP id CE54D46420; Wed, 21 Feb 2024 11:35:59 +0000 (UTC) Authentication-Results: smtp.subspace.kernel.org; dkim=pass (2048-bit key) header.d=google.com header.i=@google.com header.b="nOEY4Zuw" Received: from mail-yb1-f202.google.com (mail-yb1-f202.google.com [209.85.219.202]) (using TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 (128/128 bits)) (No client certificate requested) by smtp.subspace.kernel.org (Postfix) with ESMTPS id 5CAA24DA12 for ; Wed, 21 Feb 2024 11:35:57 +0000 (UTC) Authentication-Results: smtp.subspace.kernel.org; arc=none smtp.client-ip=209.85.219.202 ARC-Seal:i=1; a=rsa-sha256; d=subspace.kernel.org; s=arc-20240116; t=1708515359; cv=none; b=Y6slJFDhNImJsUzX/r/pJODdgdNS0jGRV73kWz0eV37yicBDJ+jaZvV7nCyecUHD1lvz2yBehexWGZG8oJy3vStI23MH+4vGXz7PaKuIV5bf8N2usU+SLYmq57xYcBtAlvBCtPs7irRCP/QJHbkGBgeR1G81fd6fLVM+aUAOQZ0= ARC-Message-Signature:i=1; a=rsa-sha256; d=subspace.kernel.org; s=arc-20240116; t=1708515359; c=relaxed/simple; bh=sFg81TTWm8wXNN7PrKjakH1R2yINKbC6VHscAn/WI6g=; h=Date:In-Reply-To:Mime-Version:References:Message-ID:Subject:From: To:Cc:Content-Type; b=HcFjS5yBAkU95JUCkrHssFRJD99joelXs9uHl99a/FCzmuGgcdQaCrk024YJLLWHmlcZJe37p/nrZjRJHDdket93wpUz0uUFJgWZ9aJw0O+nXyGxspJsUcwDwWYISLzefCt6ZSwJ0DUCf0VMJeDEdz1DP25uuRSI/a4dVn2A6aQ= ARC-Authentication-Results:i=1; smtp.subspace.kernel.org; dmarc=pass (p=reject dis=none) header.from=google.com; spf=pass smtp.mailfrom=flex--ardb.bounces.google.com; dkim=pass (2048-bit key) header.d=google.com header.i=@google.com header.b=nOEY4Zuw; arc=none smtp.client-ip=209.85.219.202 Authentication-Results: smtp.subspace.kernel.org; dmarc=pass (p=reject dis=none) header.from=google.com Authentication-Results: smtp.subspace.kernel.org; spf=pass smtp.mailfrom=flex--ardb.bounces.google.com Received: by mail-yb1-f202.google.com with SMTP id 3f1490d57ef6-dc74ac7d015so8247302276.0 for ; Wed, 21 Feb 2024 03:35:57 -0800 (PST) DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=google.com; s=20230601; t=1708515356; x=1709120156; darn=vger.kernel.org; h=cc:to:from:subject:message-id:references:mime-version:in-reply-to :date:from:to:cc:subject:date:message-id:reply-to; bh=StiHl4wqKJLROSGDCkKhKFhOVcRJSTp0SGrZr/uAC14=; b=nOEY4ZuwYnUemy5QE8lfocHWUrTUBMlf7x2hSNcNuKDt0JVRMtBGocvfZUoG8yd6QW IvFaKhOuDnTxRwG3f2ClgiC0OSs1qVW9wKn/JYqPIXM5sDQOjCMJcx5lEFQMxVmkbsG4 DXmqtlhGXMZMJbqSho8m4yiXLSgJMJE5F+McItHYyo248tCjyAfiK28zFYFksuD/sEP7 ftjbbhNhvdNYUnpk4ExA1AYDfYKJbYUjsyw9yaaVeKhyR8q2MK73WEk+7N42hRJ8zHhZ 49BZ2o1yREoTEcB5zdn6F5Jje0arYAFecz6I1qtppAJYhrs+RTOUZ5exO/keAuU3J7DA zhDw== X-Google-DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=1e100.net; s=20230601; t=1708515356; x=1709120156; h=cc:to:from:subject:message-id:references:mime-version:in-reply-to :date:x-gm-message-state:from:to:cc:subject:date:message-id:reply-to; bh=StiHl4wqKJLROSGDCkKhKFhOVcRJSTp0SGrZr/uAC14=; b=d1KBQn61ikUXykpnXOmjcH4g6FOpieL3fYDrqGlk0PDGZnmHHd6m0AzMy+ehRRATB+ znIhrNiithZaTlpTMWCBPZnk842FOjAXIE9Tw00taHZZVu5Gx1J8izIccfrlxi3D5BiJ QdqpW1Y87wR3bZNdmHxQwoCBSrMizY2igDd9fSGQNUF0Ks374heaLPfMH/OfgurswZYG WHRHy+B5bxLC7cWlUvYULGb8sXrnsxWZ8d3EvWv/8/soM5Qf8jbH9+4y3cdW5N7c6HAA mq461XaQjsLdrMBIzuQORi5apNVDnMSRPjRRIdVscDj5c67G8LjDr9l5nriizEa6e510 ve5w== X-Gm-Message-State: AOJu0YwLhBvdOuuWuQwt+QMR9MkPMggV54Ipkmwh+zFnl8bJuhreHQM/ jI1pVR11PXlmWniWFR+ibdXz3Oh9dYguSIJURF++uCHgEmPNc5/ZQutAZpWLN5b4VPB8vGlj34r SU8wp9Or2a0u2tV7Ek5KNhmMbTRcDGpmbgb0eS2BXMxMctoOw81sswKY2FM+jLIHrD3kC8sgrQA Uh6pczYYBVWeE0TplSY7fIf78WhVQJ4A== X-Received: from palermo.c.googlers.com ([fda3:e722:ac3:cc00:28:9cb1:c0a8:118a]) (user=ardb job=sendgmr) by 2002:a05:6902:1001:b0:dcc:79ab:e522 with SMTP id w1-20020a056902100100b00dcc79abe522mr731868ybt.11.1708515356342; Wed, 21 Feb 2024 03:35:56 -0800 (PST) Date: Wed, 21 Feb 2024 12:35:16 +0100 In-Reply-To: <20240221113506.2565718-18-ardb+git@google.com> Precedence: bulk X-Mailing-List: linux-kernel@vger.kernel.org List-Id: List-Subscribe: List-Unsubscribe: Mime-Version: 1.0 References: <20240221113506.2565718-18-ardb+git@google.com> X-Developer-Key: i=ardb@kernel.org; a=openpgp; fpr=F43D03328115A198C90016883D200E9CA6329909 X-Developer-Signature: v=1; a=openpgp-sha256; l=2709; i=ardb@kernel.org; h=from:subject; bh=l8yZBrOFoYa/63WN0vjjrRhrZ18bZIs3VDCL1kGdQvs=; b=owGbwMvMwCFmkMcZplerG8N4Wi2JIfXq/Y/SAgpzDwQo/t+8UsaGgZGnNTtB0tP6SOMEvj7r9 bnBk8M7SlkYxDgYZMUUWQRm/3238/REqVrnWbIwc1iZQIYwcHEKwESuczEy7HkrtmrlmlsVE/6y rebZrWbY2Ru8c/L2xheu/8PPScle6mP4za68hrVhTciUbfzS0/YXybbfFL7BPKVxwclLnTWynnf ZmQE= X-Mailer: git-send-email 2.44.0.rc0.258.g7320e95886-goog Message-ID: <20240221113506.2565718-27-ardb+git@google.com> Subject: [PATCH v5 09/16] x86/startup_64: Simplify calculation of initial page table address From: Ard Biesheuvel To: linux-kernel@vger.kernel.org Cc: Ard Biesheuvel , Kevin Loughlin , Tom Lendacky , Dionna Glaze , Thomas Gleixner , Ingo Molnar , Borislav Petkov , Dave Hansen , Andy Lutomirski , Arnd Bergmann , Nathan Chancellor , Nick Desaulniers , Justin Stitt , Kees Cook , Brian Gerst , linux-arch@vger.kernel.org, llvm@lists.linux.dev Content-Type: text/plain; charset="UTF-8" From: Ard Biesheuvel Determining the address of the initial page table to program into CR3 involves: - taking the physical address - adding the SME encryption mask On the primary entry path, the code is mapped using a 1:1 virtual to physical translation, so the physical address can be taken directly using a RIP-relative LEA instruction. On the secondary entry path, the address can be obtained by taking the offset from the virtual kernel base (__START_kernel_map) and adding the physical kernel base. This is implemented in a slightly confusing way, so clean this up. Signed-off-by: Ard Biesheuvel --- arch/x86/kernel/head_64.S | 25 ++++++-------------- 1 file changed, 7 insertions(+), 18 deletions(-) diff --git a/arch/x86/kernel/head_64.S b/arch/x86/kernel/head_64.S index 426f6fdc0075..b92031d7e006 100644 --- a/arch/x86/kernel/head_64.S +++ b/arch/x86/kernel/head_64.S @@ -111,13 +111,11 @@ SYM_CODE_START_NOALIGN(startup_64) call __startup_64 /* Form the CR3 value being sure to include the CR3 modifier */ - addq $(early_top_pgt - __START_KERNEL_map), %rax + leaq early_top_pgt(%rip), %rcx + addq %rcx, %rax #ifdef CONFIG_AMD_MEM_ENCRYPT mov %rax, %rdi - mov %rax, %r14 - - addq phys_base(%rip), %rdi /* * For SEV guests: Verify that the C-bit is correct. A malicious @@ -126,12 +124,6 @@ SYM_CODE_START_NOALIGN(startup_64) * the next RET instruction. */ call sev_verify_cbit - - /* - * Restore CR3 value without the phys_base which will be added - * below, before writing %cr3. - */ - mov %r14, %rax #endif jmp 1f @@ -171,18 +163,18 @@ SYM_INNER_LABEL(secondary_startup_64_no_verify, SYM_L_GLOBAL) /* Clear %R15 which holds the boot_params pointer on the boot CPU */ xorq %r15, %r15 + /* Derive the runtime physical address of init_top_pgt[] */ + movq phys_base(%rip), %rax + addq $(init_top_pgt - __START_KERNEL_map), %rax + /* * Retrieve the modifier (SME encryption mask if SME is active) to be * added to the initial pgdir entry that will be programmed into CR3. */ #ifdef CONFIG_AMD_MEM_ENCRYPT - movq sme_me_mask, %rax -#else - xorq %rax, %rax + addq sme_me_mask(%rip), %rax #endif - /* Form the CR3 value being sure to include the CR3 modifier */ - addq $(init_top_pgt - __START_KERNEL_map), %rax 1: /* Create a mask of CR4 bits to preserve */ @@ -204,9 +196,6 @@ SYM_INNER_LABEL(secondary_startup_64_no_verify, SYM_L_GLOBAL) btsl $X86_CR4_PSE_BIT, %ecx movq %rcx, %cr4 - /* Setup early boot stage 4-/5-level pagetables. */ - addq phys_base(%rip), %rax - /* * Switch to new page-table * -- 2.44.0.rc0.258.g7320e95886-goog