Return-Path: Received: (majordomo@vger.kernel.org) by vger.kernel.org via listexpand id S1752475AbdLUGta (ORCPT ); Thu, 21 Dec 2017 01:49:30 -0500 Received: from mail-lf0-f66.google.com ([209.85.215.66]:45498 "EHLO mail-lf0-f66.google.com" rhost-flags-OK-OK-OK-OK) by vger.kernel.org with ESMTP id S1751790AbdLUGt3 (ORCPT ); Thu, 21 Dec 2017 01:49:29 -0500 X-Google-Smtp-Source: ACJfBotcJ81ERX2MC0V3MXe74TQ/TmXleJZkjYhm1m9om6yGMD7TkIU+kcG8R/YEMTT6W5CpUzKqNxl82O5NHxriCM8= MIME-Version: 1.0 In-Reply-To: References: <20171220091544.4467-1-ganapatrao.kulkarni@cavium.com> From: Ganapatrao Kulkarni Date: Thu, 21 Dec 2017 12:19:27 +0530 Message-ID: Subject: Re: [PATCH] irqchip/gic-v3-its: Flush GICR caching for a cross node collection move of an irq To: Marc Zyngier Cc: Ganapatrao Kulkarni , linux-kernel@vger.kernel.org, linux-arm-kernel@lists.infradead.org, Thomas Gleixner , jason@lakedaemon.net, catalin.marinas@arm.com, Will Deacon , jnair@caviumnetworks.com, Robert Richter , Jan.Glauber@cavium.com Content-Type: text/plain; charset="UTF-8" Sender: linux-kernel-owner@vger.kernel.org List-ID: X-Mailing-List: linux-kernel@vger.kernel.org Content-Length: 3445 Lines: 79 On Wed, Dec 20, 2017 at 6:42 PM, Marc Zyngier wrote: > On 20/12/17 09:34, Ganapatrao Kulkarni wrote: >> Hi Marc, >> >> On Wed, Dec 20, 2017 at 2:56 PM, Marc Zyngier wrote: >>> On 20/12/17 09:15, Ganapatrao Kulkarni wrote: >>>> When an interrupt is moved, it is possible that an implementation that >>>> supports caching might still have cached data for a previous >>>> (no longer valid) mapping of the interrupt. In particular, in a distributed >>>> GIC implementation like multi-socket SoC platfroms. Hence it is necessary >>>> to flush cached entries after cross node collection migration. >>>> >>>> Signed-off-by: Ganapatrao Kulkarni >>>> --- >>>> drivers/irqchip/irq-gic-v3-its.c | 6 ++++++ >>>> 1 file changed, 6 insertions(+) >>>> >>>> diff --git a/drivers/irqchip/irq-gic-v3-its.c b/drivers/irqchip/irq-gic-v3-its.c >>>> index 4039e64..ea849a1 100644 >>>> --- a/drivers/irqchip/irq-gic-v3-its.c >>>> +++ b/drivers/irqchip/irq-gic-v3-its.c >>>> @@ -1119,6 +1119,12 @@ static int its_set_affinity(struct irq_data *d, const struct cpumask *mask_val, >>>> if (cpu != its_dev->event_map.col_map[id]) { >>>> target_col = &its_dev->its->collections[cpu]; >>>> its_send_movi(its_dev, target_col, id); >>>> + /* Issue INV for cross node collection move on >>>> + * multi socket systems. >>>> + */ >>>> + if (cpu_to_node(cpu) != >>>> + cpu_to_node(its_dev->event_map.col_map[id])) >>>> + its_send_inv(its_dev, id); >>>> its_dev->event_map.col_map[id] = cpu; >>>> irq_data_update_effective_affinity(d, cpumask_of(cpu)); >>>> } >>>> >>> >>> The MOVI command doesn't have any such requirement (it only mandates >>> synchronization), and doesn't say anything about distributed vs monolithic. >> >> GIC-v3 spec do mention to issue ITS INV command or a write to GICR_INVLPIR. >> pasting below snippet of MOVI command description. >> >> "When an interrupt is moved to a collection, it is possible that an >> implementation that supports speculative caching >> might still have cached data for a previous (no longer valid) mapping >> of the interrupt. Hence, implementations >> must take care to invalidate any data associated with an interrupt >> when it is moved. In particular, in a distributed >> implementation, the ITS must write to the appropriate GICR_* register >> to perform the invalidation in the redistributor." > > Doing some documentation archaeology, I found that this wording has been > dropped from the engineering specification in August 2014, and was never > included in the architecture specification. I suggest you start using a > slightly more up-to-date set of documentation... thanks Marc for digging in to archive. > > Now, back to your point: what it says in the bit of (confidential) > document that you quoted is that the *HW* must perform the invalidation > (that's what the words "implementations" and "ITS" refer to), not some > random bits of SW. > > If you know of an implementation that suffers from this, please resend a > patch that handles this as a quirk, with a proper erratum number. Sure, this is being discussed internally and will repost as errata fix at the earliest. > > Thanks, > > M. > -- > Jazz is not dead. It just smells funny... thanks Ganapat