Received: by 2002:a05:6a10:f347:0:0:0:0 with SMTP id d7csp51275pxu; Tue, 5 Jan 2021 05:04:01 -0800 (PST) X-Google-Smtp-Source: ABdhPJwUuCdBU/byWoHDpukml2ipMgRVn4oEMX5eqyoMSIjBXx9gOML41zpL06bVZLZ0qqnRmsG7 X-Received: by 2002:aa7:d919:: with SMTP id a25mr73611122edr.81.1609851841629; Tue, 05 Jan 2021 05:04:01 -0800 (PST) ARC-Seal: i=1; a=rsa-sha256; t=1609851841; cv=none; d=google.com; s=arc-20160816; b=S2El1rlnanHZmHZiRc14EUo8tFgFg5f+meZ6L0bt/OxjM4d/8nUFCMmkLgURyzdK/l dUA1zEL3XtyVoR3UEQLD0ZefXWWuTHWr+QI9O6wD++ihL2SvHKyAQMmkDaQJ4PgOPxH+ +CdNwM2uJLIbGpFtEzZ4/wwwmG1feV32wG8DHTxqRfBamjrcNFW6uQYDu+zGIC0Io7BD b8NNpf0Gdam+TPkVjBPyQWikBi1ehyeLw5P9I8MRIqojGnepUzlWRehKaSHAC0awFVAV dlUXkQ/o85jLC1QtkUmsnPPXNrVGzDF1rGugVRZApH51GU50Ec/fGpT9Ji9ckdzHesv9 wCzQ== ARC-Message-Signature: i=1; a=rsa-sha256; c=relaxed/relaxed; d=google.com; s=arc-20160816; h=list-id:precedence:message-id:user-agent:references:in-reply-to :subject:cc:to:from:date:content-transfer-encoding:mime-version; bh=9QL6uX4X+7U+48DnrpjjTwgupKTJlY7aOF9/QuFFiLI=; b=PpEkLF8vFyJt/0Cmiz2pUO+pSDBSoK/qG4r6xoRD3flllbjmjS919gtohG25GJI8Oq SWDvrOHJWBMXnxGaQWatPZve0eiP+FuHuswOvy6EGGl1J2sZs16dR2lrimLGCKCXyTEi 5+UETTOv25zW6mKdNP2v4o/YGPMwIxU7/mJexdhNhYcIzBnPi7MWY1fMUwk/DPvpvJxp YSa3LHHFrLnWv+MKvcgGzZeUbj6nAqbzbNtgifFs3HPF63NIq6K2Vnzayi1rUv8ERJWC VGPQ6Y63+OZJZPr53L7j1CXJeMK+3aWpwF1Wh/fOOKzgciAVyiB1/5/V3h2gcCZ3c9mB 9KUg== ARC-Authentication-Results: i=1; mx.google.com; spf=pass (google.com: domain of linux-kernel-owner@vger.kernel.org designates 23.128.96.18 as permitted sender) smtp.mailfrom=linux-kernel-owner@vger.kernel.org; dmarc=pass (p=NONE sp=NONE dis=NONE) header.from=kernel.org Return-Path: Received: from vger.kernel.org (vger.kernel.org. [23.128.96.18]) by mx.google.com with ESMTP id dt21si21466945ejc.510.2021.01.05.05.03.38; Tue, 05 Jan 2021 05:04:01 -0800 (PST) Received-SPF: pass (google.com: domain of linux-kernel-owner@vger.kernel.org designates 23.128.96.18 as permitted sender) client-ip=23.128.96.18; Authentication-Results: mx.google.com; spf=pass (google.com: domain of linux-kernel-owner@vger.kernel.org designates 23.128.96.18 as permitted sender) smtp.mailfrom=linux-kernel-owner@vger.kernel.org; dmarc=pass (p=NONE sp=NONE dis=NONE) header.from=kernel.org Received: (majordomo@vger.kernel.org) by vger.kernel.org via listexpand id S1729751AbhAELk6 (ORCPT + 99 others); Tue, 5 Jan 2021 06:40:58 -0500 Received: from mail.kernel.org ([198.145.29.99]:43966 "EHLO mail.kernel.org" rhost-flags-OK-OK-OK-OK) by vger.kernel.org with ESMTP id S1729744AbhAELk5 (ORCPT ); Tue, 5 Jan 2021 06:40:57 -0500 Received: from disco-boy.misterjones.org (disco-boy.misterjones.org [51.254.78.96]) (using TLSv1.2 with cipher ECDHE-RSA-AES256-GCM-SHA384 (256/256 bits)) (No client certificate requested) by mail.kernel.org (Postfix) with ESMTPSA id 017BB22286; Tue, 5 Jan 2021 11:40:16 +0000 (UTC) Received: from disco-boy.misterjones.org ([51.254.78.96] helo=www.loen.fr) by disco-boy.misterjones.org with esmtpsa (TLS1.2) tls TLS_ECDHE_RSA_WITH_AES_128_GCM_SHA256 (Exim 4.94) (envelope-from ) id 1kwkhB-005Q61-Lt; Tue, 05 Jan 2021 11:40:14 +0000 MIME-Version: 1.0 Content-Type: text/plain; charset=UTF-8; format=flowed Content-Transfer-Encoding: 8bit Date: Tue, 05 Jan 2021 11:40:13 +0000 From: Marc Zyngier To: Shenming Lu Cc: Eric Auger , Will Deacon , linux-arm-kernel@lists.infradead.org, kvmarm@lists.cs.columbia.edu, kvm@vger.kernel.org, linux-kernel@vger.kernel.org, Alex Williamson , Cornelia Huck , Lorenzo Pieralisi , wanghaibin.wang@huawei.com, yuzenghui@huawei.com Subject: Re: [RFC PATCH v2 2/4] KVM: arm64: GICv4.1: Try to save hw pending state in save_pending_tables In-Reply-To: References: <20210104081613.100-1-lushenming@huawei.com> <20210104081613.100-3-lushenming@huawei.com> User-Agent: Roundcube Webmail/1.4.9 Message-ID: <6f09084b32e239176b3f9b4b00874a51@kernel.org> X-Sender: maz@kernel.org X-SA-Exim-Connect-IP: 51.254.78.96 X-SA-Exim-Rcpt-To: lushenming@huawei.com, eric.auger@redhat.com, will@kernel.org, linux-arm-kernel@lists.infradead.org, kvmarm@lists.cs.columbia.edu, kvm@vger.kernel.org, linux-kernel@vger.kernel.org, alex.williamson@redhat.com, cohuck@redhat.com, lorenzo.pieralisi@arm.com, wanghaibin.wang@huawei.com, yuzenghui@huawei.com X-SA-Exim-Mail-From: maz@kernel.org X-SA-Exim-Scanned: No (on disco-boy.misterjones.org); SAEximRunCond expanded to false Precedence: bulk List-ID: X-Mailing-List: linux-kernel@vger.kernel.org On 2021-01-05 09:13, Marc Zyngier wrote: > On 2021-01-04 08:16, Shenming Lu wrote: >> After pausing all vCPUs and devices capable of interrupting, in order >> to save the information of all interrupts, besides flushing the >> pending >> states in kvm’s vgic, we also try to flush the states of VLPIs in the >> virtual pending tables into guest RAM, but we need to have GICv4.1 and >> safely unmap the vPEs first. >> >> Signed-off-by: Shenming Lu >> --- >> arch/arm64/kvm/vgic/vgic-v3.c | 58 >> +++++++++++++++++++++++++++++++---- >> 1 file changed, 52 insertions(+), 6 deletions(-) >> >> diff --git a/arch/arm64/kvm/vgic/vgic-v3.c >> b/arch/arm64/kvm/vgic/vgic-v3.c >> index 9cdf39a94a63..a58c94127cb0 100644 >> --- a/arch/arm64/kvm/vgic/vgic-v3.c >> +++ b/arch/arm64/kvm/vgic/vgic-v3.c >> @@ -1,6 +1,8 @@ >> // SPDX-License-Identifier: GPL-2.0-only >> >> #include >> +#include >> +#include >> #include >> #include >> #include >> @@ -356,6 +358,38 @@ int vgic_v3_lpi_sync_pending_status(struct kvm >> *kvm, struct vgic_irq *irq) >> return 0; >> } >> >> +/* >> + * The deactivation of the doorbell interrupt will trigger the >> + * unmapping of the associated vPE. >> + */ >> +static void unmap_all_vpes(struct vgic_dist *dist) >> +{ >> + struct irq_desc *desc; >> + int i; >> + >> + if (!kvm_vgic_global_state.has_gicv4_1) >> + return; >> + >> + for (i = 0; i < dist->its_vm.nr_vpes; i++) { >> + desc = irq_to_desc(dist->its_vm.vpes[i]->irq); >> + irq_domain_deactivate_irq(irq_desc_get_irq_data(desc)); >> + } >> +} >> + >> +static void map_all_vpes(struct vgic_dist *dist) >> +{ >> + struct irq_desc *desc; >> + int i; >> + >> + if (!kvm_vgic_global_state.has_gicv4_1) >> + return; >> + >> + for (i = 0; i < dist->its_vm.nr_vpes; i++) { >> + desc = irq_to_desc(dist->its_vm.vpes[i]->irq); >> + irq_domain_activate_irq(irq_desc_get_irq_data(desc), false); >> + } >> +} >> + >> /** >> * vgic_v3_save_pending_tables - Save the pending tables into guest >> RAM >> * kvm lock and all vcpu lock must be held >> @@ -365,14 +399,18 @@ int vgic_v3_save_pending_tables(struct kvm *kvm) >> struct vgic_dist *dist = &kvm->arch.vgic; >> struct vgic_irq *irq; >> gpa_t last_ptr = ~(gpa_t)0; >> - int ret; >> + int ret = 0; >> u8 val; >> >> + /* As a preparation for getting any VLPI states. */ >> + unmap_all_vpes(dist); > > What if the VPEs are not mapped yet? Is it possible to snapshot a VM > that has not run at all? More questions: what happens to vSGIs that were mapped to the VPEs? Can they safely be restarted? The spec is not saying much on the subject. Once the unmap has taken place, it won't be possible to read their state via GICR_VSGIRPEND, and only the memory state can be used. This probably needs to be tracked as well. Thanks, M. -- Jazz is not dead. It just smells funny...