Received: by 2002:ad5:474a:0:0:0:0:0 with SMTP id i10csp1825349imu; Thu, 24 Jan 2019 02:34:27 -0800 (PST) X-Google-Smtp-Source: ALg8bN4V0dSl1g8Suj7Fcafz20Xx6bj0SWdT+qQ+tzjlZ10UZNAf9a0KfZd6nIeH9C8hRaljcrWA X-Received: by 2002:a63:8d44:: with SMTP id z65mr5452487pgd.57.1548326067372; Thu, 24 Jan 2019 02:34:27 -0800 (PST) ARC-Seal: i=1; a=rsa-sha256; t=1548326067; cv=none; d=google.com; s=arc-20160816; b=LCUiTQiBtO+dDIS+NaFv6Ky7YFvW7yVFLXiVqCDrCmGTYO3HzRdckyTD5LtnBmYI8l rybIlxNtr+9favP3ftPf607jOI+OAe3g5HImNfZFVtUUwMTMYqhzaQvK2yP8uo2rWUtY 0uoafdwBDd++m7QOTvTpz3pjq6fz/1gSKWzBf6cDOKn4bcGSzFB/G6c1dkSXmZnSSPDh boe2Hy/Gy7bduW3GN//LXhOxV+uMQMnb7dS820JaoBTwjO8wK+VHTQjZ94MMoxVVNesN r6O+YMuof2nd+mWQ3sokIrxOnXNG9wQUB2+Z7o4qKGHZdz9caL55Eum1laoKdmM5/aen K2Jw== ARC-Message-Signature: i=1; a=rsa-sha256; c=relaxed/relaxed; d=google.com; s=arc-20160816; h=list-id:precedence:sender:mime-version:organization:user-agent :references:in-reply-to:subject:cc:to:from:message-id:date; bh=aGdByG7BAI7XKWGUdtdEA5qdhqy/e0wxDwnA35nHzYM=; b=K8Ccts8SHQw7ykoqnqIiT5xVsa4Mi1mxFkB58yq+XLQ6qWX3luwVahtcdYvcEbhkuY Nt+DweKrN11uOlS/RlFBm5Ph5ddt7J0jX4jJvjxn2lgI3ZXRImTbRe4xrVpMrTj3EEUk dwl5e2eJs9M/4ujSZdACia+WaIthNeh7tzh/dq3pwJBWMnffBdN0eAqRNK8dN9ifLno1 pHKGV3wdgxLzZXWizYUfAixwyu7dJVSvKNRFN1y8wHQ4Qs5IfrbQQYlPQ1CG2hWRpA/U DAL9YQO8Lu6xRlXZ7gkUjYrGojKOw+brxULpNkvt96AVKZiE0BpwfAqdwgmA3ByzL9rp YURQ== ARC-Authentication-Results: i=1; mx.google.com; spf=pass (google.com: best guess record for domain of linux-kernel-owner@vger.kernel.org designates 209.132.180.67 as permitted sender) smtp.mailfrom=linux-kernel-owner@vger.kernel.org Return-Path: Received: from vger.kernel.org (vger.kernel.org. [209.132.180.67]) by mx.google.com with ESMTP id i5si21727125pgg.279.2019.01.24.02.34.11; Thu, 24 Jan 2019 02:34:27 -0800 (PST) Received-SPF: pass (google.com: best guess record for domain of linux-kernel-owner@vger.kernel.org designates 209.132.180.67 as permitted sender) client-ip=209.132.180.67; Authentication-Results: mx.google.com; spf=pass (google.com: best guess record for domain of linux-kernel-owner@vger.kernel.org designates 209.132.180.67 as permitted sender) smtp.mailfrom=linux-kernel-owner@vger.kernel.org Received: (majordomo@vger.kernel.org) by vger.kernel.org via listexpand id S1727409AbfAXKds (ORCPT + 99 others); Thu, 24 Jan 2019 05:33:48 -0500 Received: from foss.arm.com ([217.140.101.70]:54042 "EHLO foss.arm.com" rhost-flags-OK-OK-OK-OK) by vger.kernel.org with ESMTP id S1726986AbfAXKds (ORCPT ); Thu, 24 Jan 2019 05:33:48 -0500 Received: from usa-sjc-imap-foss1.foss.arm.com (unknown [10.72.51.249]) by usa-sjc-mx-foss1.foss.arm.com (Postfix) with ESMTP id 348A31596; Thu, 24 Jan 2019 02:33:48 -0800 (PST) Received: from big-swifty.misterjones.org (big-swifty.copenhagen.arm.com [10.32.148.139]) by usa-sjc-imap-foss1.foss.arm.com (Postfix) with ESMTPSA id 682813F614; Thu, 24 Jan 2019 02:33:41 -0800 (PST) Date: Thu, 24 Jan 2019 10:33:35 +0000 Message-ID: <86munq8hvk.wl-marc.zyngier@arm.com> From: Marc Zyngier To: Heyi Guo Cc: , Thomas Gleixner , Jason Cooper , , Subject: Re: [PATCH] irqchip/gic-v4: fix occasional VLPI drop In-Reply-To: <1548233948-7450-1-git-send-email-guoheyi@huawei.com> References: <1548233948-7450-1-git-send-email-guoheyi@huawei.com> User-Agent: Wanderlust/2.15.9 (Almost Unreal) SEMI-EPG/1.14.7 (Harue) FLIM/1.14.9 (=?UTF-8?B?R29qxY0=?=) APEL/10.8 EasyPG/1.0.0 Emacs/25.1 (aarch64-unknown-linux-gnu) MULE/6.0 (HANACHIRUSATO) Organization: ARM Ltd MIME-Version: 1.0 (generated by SEMI-EPG 1.14.7 - "Harue") Content-Type: text/plain; charset=US-ASCII Sender: linux-kernel-owner@vger.kernel.org Precedence: bulk List-ID: X-Mailing-List: linux-kernel@vger.kernel.org On Wed, 23 Jan 2019 08:59:08 +0000, Heyi Guo wrote: > > 1. In current implementation, every VLPI will temporarily be mapped to > the first CPU in system (normally CPU0) and then moved to the real > scheduled CPU later. > > 2. So there is a time window and a VLPI may be sent to CPU0 instead of > the real scheduled vCPU, in a multi-CPU virtual machine. > > 3. However, CPU0 may have not been scheduled as a virtual CPU after > system boots up, so the value of its GICR_VPROPBASER is unknown at > that moment. > > 4. If the INTID of VLPI is larger than 2^(GICR_VPROPBASER.IDbits+1), > while IDbits is also in unknown state, GIC will behave as if the VLPI > is out of range and simply drop it, which results in interrupt missing > in Guest. > > As no code will clear GICR_VPROPBASER at runtime, we can safely > initialize the IDbits field at boot time for each CPU to get rid of > this issue. > > We also clear Valid bit of GICR_VPENDBASER in case any ancient > programming gets left in and causes memory corrupting. > > Signed-off-by: Heyi Guo > Signed-off-by: Heyi Guo > --- > drivers/irqchip/irq-gic-v3-its.c | 25 +++++++++++++++++++++++++ > 1 file changed, 25 insertions(+) > > diff --git a/drivers/irqchip/irq-gic-v3-its.c b/drivers/irqchip/irq-gic-v3-its.c > index 3365d44..634c8a7 100644 > --- a/drivers/irqchip/irq-gic-v3-its.c > +++ b/drivers/irqchip/irq-gic-v3-its.c > @@ -2145,6 +2145,31 @@ static void its_cpu_init_lpis(void) > val |= GICR_CTLR_ENABLE_LPIS; > writel_relaxed(val, rbase + GICR_CTLR); > > + if (gic_rdists->has_vlpis) { > + void __iomem *vlpi_base = gic_data_rdist_vlpi_base(); > + > + /* > + * It's possible for CPU to receive VLPIs before it is > + * sheduled as a vPE, especially for the first CPU, and the > + * VLPI with INTID larger than 2^(IDbits+1) will be considered > + * as out of range and dropped by GIC. > + * So we initialize IDbits to known value to avoid VLPI drop. > + */ > + val = (LPI_NRBITS - 1) & GICR_VPROPBASER_IDBITS_MASK; > + pr_debug("GICv4: CPU%d: Init IDbits to 0x%llx for GICR_VPROPBASER\n", > + smp_processor_id(), val); > + gits_write_vpropbaser(val, vlpi_base + GICR_VPROPBASER); > + > + /* > + * Also clear Valid bit of GICR_VPENDBASER, in case some > + * ancient programming gets left in and has possibility of > + * corrupting memory. > + */ > + val = gits_read_vpendbaser(vlpi_base + GICR_VPENDBASER); > + val &= ~GICR_VPENDBASER_Valid; > + gits_write_vpendbaser(val, vlpi_base + GICR_VPENDBASER); I'm afraid it is not enough to just write back with the valid bit clear. You need to make sure Dirty has been cleared before continuing, which is why I was asking you to reuse some bits of its_vpe_deschedule. Something like this: diff --git a/drivers/irqchip/irq-gic-v3-its.c b/drivers/irqchip/irq-gic-v3-its.c index db20e992a40f..1808cb5f6a3e 100644 --- a/drivers/irqchip/irq-gic-v3-its.c +++ b/drivers/irqchip/irq-gic-v3-its.c @@ -2751,9 +2751,8 @@ static void its_vpe_schedule(struct its_vpe *vpe) gits_write_vpendbaser(val, vlpi_base + GICR_VPENDBASER); } -static void its_vpe_deschedule(struct its_vpe *vpe) +static u64 its_clear_vpend_valid(void __iomem *vlpi_base) { - void __iomem *vlpi_base = gic_data_rdist_vlpi_base(); u32 count = 1000000; /* 1s! */ bool clean; u64 val; @@ -2773,7 +2772,17 @@ static void its_vpe_deschedule(struct its_vpe *vpe) } } while (!clean && count); - if (unlikely(!clean && !count)) { + return val; +} + +static void its_vpe_deschedule(struct its_vpe *vpe) +{ + void __iomem *vlpi_base = gic_data_rdist_vlpi_base(); + u64 val; + + val = its_clear_vpend_valid(vlpi_base); + + if (unlikely(val & GICR_VPENDBASER_Dirty)) { pr_err_ratelimited("ITS virtual pending table not cleaning\n"); vpe->idai = false; vpe->pending_last = true; And you can the rewrite the above GICR_VPENDBASER_Valid clearing as: val = its_clear_vpend_valid(vlpi_base); WARN_ON(val & GICR_VPENDBASER_Dirty); We'll at least get a warning if something goes wrong. Thanks, M. -- Jazz is not dead, it just smell funny.