Received: by 2002:a05:6a10:f347:0:0:0:0 with SMTP id d7csp1561894pxu; Fri, 27 Nov 2020 09:52:27 -0800 (PST) X-Google-Smtp-Source: ABdhPJzS6pcbrcn5QToGXp/AVvwIJ+eYdzwALjyS9Brw/aZ+0ZvkSr3orOcCI+AHQ3CvaNoxriEv X-Received: by 2002:a05:6402:553:: with SMTP id i19mr9362738edx.194.1606499547463; Fri, 27 Nov 2020 09:52:27 -0800 (PST) ARC-Seal: i=1; a=rsa-sha256; t=1606499547; cv=none; d=google.com; s=arc-20160816; b=eHkO5/vMMD0nFcyOdJU/cgHoW0fMCe0+ah6T4bhlkzvPWa7Pf6N7hET2cvf7gpOad6 VlgeJLhWQG8WnFy26zc55QVUs+XWKcB+9USr+krhI7NKunGaecGE5CPC/x6bIa/16BcY dEpLRaMyfUasmayFyxyJBl0TUiXs+bcbDQWgei11M0Fyz1r7THYOlLAk8uUzDXWUmHWt O/xfMI7EwAUeW8okZVueIzFU1ZNqX23gAyn+lX9CrM4lVKycVKFKyq1eUx0MkFQSQ1Pf heJ1OVzOSdF/jiT4osPTyhl/Nf1iMIwL6onz/mnaeEFc/fjfEpkcagdtpuba9mk+1WIO e0nQ== ARC-Message-Signature: i=1; a=rsa-sha256; c=relaxed/relaxed; d=google.com; s=arc-20160816; h=list-id:precedence:user-agent:in-reply-to:content-disposition :mime-version:references:message-id:subject:cc:to:from:date; bh=xObA27a5UXXq6Fe5ae7t3BCbZ0D+sL6A7iY8V5MUNCQ=; b=k7pHG27noc2dOAnpBmbzMKt428Z26PIZdOUM4Vuy03eprr+U26QFFVbMcydSy6Xm7Q FpJkHmMNtER6//bPGdVhBY9A+uy5Wx4htCNEQpWB2b4wM31qinc10MGgzHq4pGoDM5aO oEIo8DhPC5b5p4Uy9hOYGYan+p6JqxAtO5jiOi4Ai4ZI+n5GxeupU4Tce/xYmpQtYG1r iNHVbPb6TIx1sfbLfpVp7dadNfCOzafXNs7qSLdHyHQ55YBva6EomERqr8DTtjpjVbz6 TuBej7f0cT7YHvfLLve67Kw9Op6DzBL2L3qkLgay647VwdbZbh+fPnbHndLgjwA7B2AI t4UQ== ARC-Authentication-Results: i=1; mx.google.com; spf=pass (google.com: domain of linux-kernel-owner@vger.kernel.org designates 23.128.96.18 as permitted sender) smtp.mailfrom=linux-kernel-owner@vger.kernel.org; dmarc=fail (p=NONE sp=NONE dis=NONE) header.from=arm.com Return-Path: Received: from vger.kernel.org (vger.kernel.org. [23.128.96.18]) by mx.google.com with ESMTP id jo9si5280696ejb.571.2020.11.27.09.52.05; Fri, 27 Nov 2020 09:52:27 -0800 (PST) Received-SPF: pass (google.com: domain of linux-kernel-owner@vger.kernel.org designates 23.128.96.18 as permitted sender) client-ip=23.128.96.18; Authentication-Results: mx.google.com; spf=pass (google.com: domain of linux-kernel-owner@vger.kernel.org designates 23.128.96.18 as permitted sender) smtp.mailfrom=linux-kernel-owner@vger.kernel.org; dmarc=fail (p=NONE sp=NONE dis=NONE) header.from=arm.com Received: (majordomo@vger.kernel.org) by vger.kernel.org via listexpand id S1732453AbgK0Rrf (ORCPT + 99 others); Fri, 27 Nov 2020 12:47:35 -0500 Received: from foss.arm.com ([217.140.110.172]:47578 "EHLO foss.arm.com" rhost-flags-OK-OK-OK-OK) by vger.kernel.org with ESMTP id S1731834AbgK0Rre (ORCPT ); Fri, 27 Nov 2020 12:47:34 -0500 Received: from usa-sjc-imap-foss1.foss.arm.com (unknown [10.121.207.14]) by usa-sjc-mx-foss1.foss.arm.com (Postfix) with ESMTP id 33FAD31B; Fri, 27 Nov 2020 09:47:33 -0800 (PST) Received: from bogus (unknown [10.57.59.53]) by usa-sjc-imap-foss1.foss.arm.com (Postfix) with ESMTPSA id 4F5AA3F71F; Fri, 27 Nov 2020 09:47:30 -0800 (PST) Date: Fri, 27 Nov 2020 17:47:26 +0000 From: Sudeep Holla To: David Brazdil Cc: kvmarm@lists.cs.columbia.edu, Jonathan Corbet , Catalin Marinas , Will Deacon , Marc Zyngier , James Morse , Julien Thierry , Suzuki K Poulose , Dennis Zhou , Tejun Heo , Christoph Lameter , Mark Rutland , Lorenzo Pieralisi , linux-doc@vger.kernel.org, linux-kernel@vger.kernel.org, linux-arm-kernel@lists.infradead.org, kernel-team@android.com Subject: Re: [PATCH v3 19/23] kvm: arm64: Intercept host's CPU_ON SMCs Message-ID: <20201127174726.4b6azdyzn5j6qmao@bogus> References: <20201126155421.14901-1-dbrazdil@google.com> <20201126155421.14901-20-dbrazdil@google.com> MIME-Version: 1.0 Content-Type: text/plain; charset=us-ascii Content-Disposition: inline In-Reply-To: <20201126155421.14901-20-dbrazdil@google.com> User-Agent: NeoMutt/20171215 Precedence: bulk List-ID: X-Mailing-List: linux-kernel@vger.kernel.org On Thu, Nov 26, 2020 at 03:54:17PM +0000, David Brazdil wrote: > Add a handler of the CPU_ON PSCI call from host. When invoked, it looks > up the logical CPU ID corresponding to the provided MPIDR and populates > the state struct of the target CPU with the provided x0, pc. It then > calls CPU_ON itself, with an entry point in hyp that initializes EL2 > state before returning ERET to the provided PC in EL1. > > There is a simple atomic lock around the boot args struct. If it is > already locked, CPU_ON will return PENDING_ON error code. > > Signed-off-by: David Brazdil > --- > arch/arm64/kvm/hyp/nvhe/hyp-init.S | 30 ++++++++ > arch/arm64/kvm/hyp/nvhe/psci-relay.c | 109 +++++++++++++++++++++++++++ > 2 files changed, 139 insertions(+) > > diff --git a/arch/arm64/kvm/hyp/nvhe/hyp-init.S b/arch/arm64/kvm/hyp/nvhe/hyp-init.S > index 98ce40e17b42..ea71f653af55 100644 > --- a/arch/arm64/kvm/hyp/nvhe/hyp-init.S > +++ b/arch/arm64/kvm/hyp/nvhe/hyp-init.S > @@ -9,6 +9,7 @@ > > #include > #include > +#include > #include > #include > #include > @@ -161,6 +162,35 @@ alternative_else_nop_endif > ret > SYM_CODE_END(___kvm_hyp_init) > > +SYM_CODE_START(__kvm_hyp_cpu_on_entry) > + msr SPsel, #1 // We want to use SP_EL{1,2} > + > + /* Check that the core was booted in EL2. */ > + mrs x1, CurrentEL > + cmp x1, #CurrentEL_EL2 > + b.eq 2f > + > + /* The core booted in EL1. KVM cannot be initialized on it. */ > +1: wfe > + wfi > + b 1b > + > + /* Initialize EL2 CPU state to sane values. */ > +2: mov x29, x0 > + init_el2_state nvhe > + mov x0, x29 > + > + /* Enable MMU, set vectors and stack. */ > + bl ___kvm_hyp_init > + > + /* Load address of the C handler. */ > + ldr x1, =__kvm_hyp_psci_cpu_entry > + kimg_hyp_va x1, x2 > + > + /* Leave idmap. */ > + br x1 > +SYM_CODE_END(__kvm_hyp_cpu_on_entry) > + > SYM_CODE_START(__kvm_handle_stub_hvc) > cmp x0, #HVC_SOFT_RESTART > b.ne 1f > diff --git a/arch/arm64/kvm/hyp/nvhe/psci-relay.c b/arch/arm64/kvm/hyp/nvhe/psci-relay.c > index 7aa87ab7f5ce..39e507672e6e 100644 > --- a/arch/arm64/kvm/hyp/nvhe/psci-relay.c > +++ b/arch/arm64/kvm/hyp/nvhe/psci-relay.c > @@ -9,12 +9,17 @@ > #include > #include > #include > +#include > #include > #include > #include > > #include > > +extern char __kvm_hyp_cpu_on_entry[]; > + > +void __noreturn __host_enter(struct kvm_cpu_context *host_ctxt); > + > /* Config options set by the host. */ > u32 __ro_after_init kvm_host_psci_version; > u32 __ro_after_init kvm_host_psci_function_id[PSCI_FN_MAX]; > @@ -22,6 +27,19 @@ s64 __ro_after_init hyp_physvirt_offset; > > #define __hyp_pa(x) ((phys_addr_t)((x)) + hyp_physvirt_offset) > > +#define INVALID_CPU_ID UINT_MAX > + > +#define CPU_UNLOCKED 0 > +#define CPU_LOCKED 1 > + > +struct cpu_boot_args { > + unsigned long pc; > + unsigned long r0; > +}; > + > +static DEFINE_PER_CPU(atomic_t, cpu_on_lock) = ATOMIC_INIT(0); > +static DEFINE_PER_CPU(struct cpu_boot_args, cpu_on_args); > + > static u64 get_psci_func_id(struct kvm_cpu_context *host_ctxt) > { > DECLARE_REG(u64, func_id, host_ctxt, 0); > @@ -78,10 +96,99 @@ static __noreturn unsigned long psci_forward_noreturn(struct kvm_cpu_context *ho > hyp_panic(); /* unreachable */ > } > > +static unsigned int find_cpu_id(u64 mpidr) > +{ > + unsigned int i; > + > + /* Reject invalid MPIDRs */ > + if (mpidr & ~MPIDR_HWID_BITMASK) > + return INVALID_CPU_ID; > + > + for (i = 0; i < NR_CPUS; i++) { I may not have understood the flow correctly, so just asking: This is just called for secondaries on boot right ? And the cpumasks are setup by then ? Just trying to see if we can use cpu_possible_mask instead of running through all 256/1k/4k cpus(ofcourse based on NR_CPUS config) -- Regards, Sudeep