Return-Path: Received: (majordomo@vger.kernel.org) by vger.kernel.org via listexpand id S932149AbdGCOx5 (ORCPT ); Mon, 3 Jul 2017 10:53:57 -0400 Received: from foss.arm.com ([217.140.101.70]:35478 "EHLO foss.arm.com" rhost-flags-OK-OK-OK-OK) by vger.kernel.org with ESMTP id S1756680AbdGCOxw (ORCPT ); Mon, 3 Jul 2017 10:53:52 -0400 Subject: Re: [PATCH] irqchip: gicv3-its: Use NUMA aware memory allocation for ITS tables To: shankerd@codeaurora.org, Ganapatrao Kulkarni References: <1498405569-463-1-git-send-email-shankerd@codeaurora.org> Cc: linux-kernel , linux-arm-kernel , Thomas Gleixner , Jason Cooper , Vikram Sethi , Jayachandran C , "ganapatrao.kulkarni@cavium.com" From: Marc Zyngier Organization: ARM Ltd Message-ID: <27b46938-ae23-9750-e0c7-09fa472d3297@arm.com> Date: Mon, 3 Jul 2017 15:53:49 +0100 User-Agent: Mozilla/5.0 (X11; Linux x86_64; rv:45.0) Gecko/20100101 Thunderbird/45.8.0 MIME-Version: 1.0 In-Reply-To: Content-Type: text/plain; charset=utf-8 Content-Transfer-Encoding: 7bit Sender: linux-kernel-owner@vger.kernel.org List-ID: X-Mailing-List: linux-kernel@vger.kernel.org Content-Length: 2038 Lines: 52 Hi Shanker, On 03/07/17 15:24, Shanker Donthineni wrote: > Hi Marc, > > On 06/30/2017 03:51 AM, Marc Zyngier wrote: >> On 30/06/17 04:01, Ganapatrao Kulkarni wrote: >>> On Fri, Jun 30, 2017 at 8:04 AM, Ganapatrao Kulkarni >>> wrote: >>>> Hi Shanker, >>>> >>>> On Sun, Jun 25, 2017 at 9:16 PM, Shanker Donthineni >>>> wrote: >>>>> The NUMA node information is visible to ITS driver but not being used >>>>> other than handling errata. This patch allocates the memory for ITS >>>>> tables from the corresponding NUMA node using the appropriate NUMA >>>>> aware functions. >>> >>> IMHO, the description would have been more constructive? >>> >>> "All ITS tables are mapped by default to NODE 0 memory. >>> Adding changes to allocate memory from respective NUMA NODES of ITS devices. >>> This will optimize tables access and avoids unnecessary inter-node traffic." >> >> But more importantly, I'd like to see figures showing the actual benefit >> of this per-node allocation. Given that both of you guys have access to >> such platforms, please show me the numbers! >> > > I'll share the actual results which shows the improvement whenever > available on our next chips. Current version of Qualcomm qdf2400 doesn't > support multi socket configuration to capture results and share with you. > > Do you see any other issues with this patch apart from the performance > improvements. I strongly believe this brings the noticeable improvement > in numbers on systems where it has multi node memory/CPU configuration. I agree that it *could* show an improvement, but it very much depends on how often the ITS misses in its caches. For this kind of patches, I want to see two things: 1) It brings a measurable benefit on NUMA platforms 2) it doesn't adversely impact non-NUMA systems I can deal with (2), but I have no way of evaluating (1), mostly for the lack of an infrastructure exercising multiple ITSs at the same time. Thanks, M. -- Jazz is not dead. It just smells funny...