Return-Path: Received: (majordomo@vger.kernel.org) by vger.kernel.org via listexpand id S1755227AbbEGKmD (ORCPT ); Thu, 7 May 2015 06:42:03 -0400 Received: from mail-am1on0065.outbound.protection.outlook.com ([157.56.112.65]:50176 "EHLO emea01-am1-obe.outbound.protection.outlook.com" rhost-flags-OK-OK-OK-FAIL) by vger.kernel.org with ESMTP id S1751820AbbEGKl5 (ORCPT ); Thu, 7 May 2015 06:41:57 -0400 Authentication-Results: spf=none (sender IP is 193.47.165.134) smtp.mailfrom=mellanox.com; vger.kernel.org; dkim=none (message not signed) header.d=none; Message-ID: <554B416C.6050701@mellanox.com> Date: Thu, 7 May 2015 13:41:48 +0300 From: Amir Vadai User-Agent: Mozilla/5.0 (Windows NT 6.1; WOW64; rv:31.0) Gecko/20100101 Thunderbird/31.6.0 MIME-Version: 1.0 To: Thomas Gleixner CC: Jiang Liu , Bjorn Helgaas , Benjamin Herrenschmidt , Ingo Molnar , "H. Peter Anvin" , "Rafael J. Wysocki" , Randy Dunlap , "Yinghai Lu" , Borislav Petkov , Ido Shamay , "David S. Miller" , Or Gerlitz , Eric Dumazet , Hadar Hen Zion , Eran Ben Elisha , Joe Perches , Saeed Mahameed , Matan Barak , Konrad Rzeszutek Wilk , "Tony Luck" , "x86@kernel.org" , "linux-kernel@vger.kernel.org" , "linux-pci@vger.kernel.org" , "linux-acpi@vger.kernel.org" , netdev , Amir Vadai Subject: Re: [RFC v1 07/11] net/mlx4: Cache irq_desc->affinity instead of irq_desc References: <1430709339-29083-1-git-send-email-jiang.liu@linux.intel.com> <1430709339-29083-8-git-send-email-jiang.liu@linux.intel.com> In-Reply-To: Content-Type: text/plain; charset="windows-1252" Content-Transfer-Encoding: 7bit X-Originating-IP: [172.16.1.240] X-EOPAttributedMessage: 0 X-Forefront-Antispam-Report: CIP:193.47.165.134;CTRY:IL;IPV:NLI;EFV:NLI;SFV:NSPM;SFS:(10009020)(6009001)(428002)(479174004)(51704005)(164054003)(52314003)(377454003)(189002)(24454002)(199003)(86362001)(33656002)(101416001)(65816999)(87266999)(105586002)(76176999)(54356999)(50986999)(47776003)(46102003)(87936001)(59896002)(6806004)(50466002)(19580405001)(19580395003)(77156002)(62966003)(93886004)(110136002)(107886002)(5001920100001)(92566002)(2950100001)(36756003)(106466001)(15975445007)(77096005)(189998001)(3940600001)(4001430100001);DIR:OUT;SFP:1101;SCL:1;SRVR:AMSPR05MB533;H:mtlcas13.mtl.com;FPR:;SPF:None;MLV:sfv;MX:1;A:1;LANG:en; X-Microsoft-Antispam: UriScan:;BCL:0;PCL:0;RULEID:;SRVR:AMSPR05MB533; X-Microsoft-Antispam-PRVS: X-Exchange-Antispam-Report-Test: UriScan:; X-Exchange-Antispam-Report-CFA-Test: BCL:0;PCL:0;RULEID:(601004)(5005006)(3002001);SRVR:AMSPR05MB533;BCL:0;PCL:0;RULEID:;SRVR:AMSPR05MB533; X-Forefront-PRVS: 056929CBB8 X-OriginatorOrg: Mellanox.com X-MS-Exchange-CrossTenant-OriginalArrivalTime: 07 May 2015 10:41:53.3452 (UTC) X-MS-Exchange-CrossTenant-Id: a652971c-7d2e-4d9b-a6a4-d149256f461b X-MS-Exchange-CrossTenant-OriginalAttributedTenantConnectingIp: TenantId=a652971c-7d2e-4d9b-a6a4-d149256f461b;Ip=[193.47.165.134];Helo=[mtlcas13.mtl.com] X-MS-Exchange-CrossTenant-FromEntityHeader: HybridOnPrem X-MS-Exchange-Transport-CrossTenantHeadersStamped: AMSPR05MB533 Sender: linux-kernel-owner@vger.kernel.org List-ID: X-Mailing-List: linux-kernel@vger.kernel.org Content-Length: 2742 Lines: 70 On 5/5/2015 5:53 PM, Thomas Gleixner wrote: > On Tue, 5 May 2015, Amir Vadai wrote: > >> On Mon, May 4, 2015 at 6:10 PM, Thomas Gleixner wrote: >>> On Mon, 4 May 2015, Amir Vadai wrote: >>> >>>> On Mon, May 4, 2015 at 6:15 AM, Jiang Liu wrote: >>>>> The field 'affinity' in irq_desc won't change once the irq_desc data >>>>> structure is created. So cache irq_desc->affinity instead of irq_desc. >>>>> This also helps to hide struct irq_desc from device drivers. >>>> >>>> Hi Jiang, >>>> >>>> I might not understand the new changes irq core, but up until now >>>> affinity was changed when the user changed it through >>>> /proc/irq//smp_affinity. >>>> This code is monitoring the affinity from the napi_poll context to >>>> detect affinity changes, and prevent napi from keep running on the >>>> wrong CPU. >>>> Therefore, the affinity can't be cached at the beginning. Please >>>> revert this caching. >>> >>> Sigh. All of this is completely wrong. Polling in the internals of >>> irq_desc is just crap. >> Hi, >> >>> >>> irq_set_affinity_notifier() has been added for exactly this purpose. >> rmap is already registered on irq_set_affinity_notifier(). >> Actually, I did post an extension to the affinity notifier, to use >> affinity chain which you rejected [1] (and BTW, convinced me that this >> is not the way to do it). >> Current code is according to your suggestion. >> >> [1] - https://lkml.org/lkml/2014/5/26/222 > > No, it's not. I wrote: > > "So what's the point of adding notifier call chain complexity, > ordering problems etc., if you can simply note the fact that the > affinity changed in the rmap itself and check that in the RX path?" > > I did not tell you to fiddle in irqdesc, really. > > Thanks, > > tglx > It is problematic, since I didn't want to link the napi polling affinity issue and CONFIG_CPU_RMAP being selected. That's why I CC'd you when submitted the patch [1]. Guess it was my bad that I didn't do it in a better way that would give you a chance to fully review it. Anyway, I understand now that I shouldn't access irqdesc attributes from a device driver. I will see if I can live with disabling this whole feature (migrate a currently looping NAPI when affinity is changed), when CONFIG_CPU_RMAP is not selected. If not, I might modify the rmap code to give me this information even when it is not selected. I assume that rmap notifier callback can access this information. Thanks, Amir -- To unsubscribe from this list: send the line "unsubscribe linux-kernel" in the body of a message to majordomo@vger.kernel.org More majordomo info at http://vger.kernel.org/majordomo-info.html Please read the FAQ at http://www.tux.org/lkml/