Received-SPF: pass (google.com: domain of linux-kernel-owner@vger.kernel.org designates 2620:137:e000::3:4 as permitted sender) client-ip=2620:137:e000::3:4;
Message-ID: <6431a069-6fc5-47ad-9519-868ae84b4a1a@intel.com>
Date:   Wed, 13 Dec 2023 12:23:43 +0100
User-Agent: Mozilla Thunderbird
Subject: Re: [Intel-wired-lan] [PATCH net-next v6 08/12] libie: add Rx buffer
 management (via Page Pool)
Content-Language: en-US
To:     Jakub Kicinski <kuba@kernel.org>
CC:     Paul Menzel <pmenzel@molgen.mpg.de>,
        Maciej Fijalkowski <maciej.fijalkowski@intel.com>,
        Jesper Dangaard Brouer <hawk@kernel.org>,
        Amritha Nambiar <amritha.nambiar@intel.com>,
        Larysa Zaremba <larysa.zaremba@intel.com>,
        <netdev@vger.kernel.org>, Alexander Duyck <alexanderduyck@fb.com>,
        Ilias Apalodimas <ilias.apalodimas@linaro.org>,
        "Eric Dumazet" <edumazet@google.com>,
        <linux-kernel@vger.kernel.org>,
        Yunsheng Lin <linyunsheng@huawei.com>,
        Michal Kubiak <michal.kubiak@intel.com>,
        <intel-wired-lan@lists.osuosl.org>,
        David Christensen <drc@linux.vnet.ibm.com>,
        Paolo Abeni <pabeni@redhat.com>,
        "David S. Miller" <davem@davemloft.net>
References: <20231207172010.1441468-1-aleksander.lobakin@intel.com>
 <20231207172010.1441468-9-aleksander.lobakin@intel.com>
 <1103fe8f-04c8-8cc4-8f1b-ff45cea22b54@huawei.com>
 <03d7e8b0-8766-4f59-afd4-15b592693a83@intel.com>
 <20231211112332.2abc94ae@kernel.org>
From:   Alexander Lobakin <aleksander.lobakin@intel.com>
In-Reply-To: <20231211112332.2abc94ae@kernel.org>
Content-Type: text/plain; charset="UTF-8"
Content-Transfer-Encoding: 8bit
MIME-Version: 1.0
X-MS-Exchange-AntiSpam-MessageData-ChunkCount: 1
X-MS-Exchange-AntiSpam-MessageData-0: =?utf-8?B?OHVsL3d4RmVpd1VFTnpyK1QzdXJpZk5kZzZpenhUb2JIK3JBbTRkZFMzemxq?=
 =?utf-8?B?V0szWWtEWmZlaysvU1hhVHdwamZra3I2eDlORXJCK0dIQ1pXeXc0NXZwZFVq?=
 =?utf-8?B?a1dDaFZjVy9DRENTY24zU3JYU2dFMTF2d1Ewd0lRbzdHQ0laUVg4eHhqOUNq?=
 =?utf-8?B?bEVsWWxjK1I0bEtsZ1R0em5WRmJVV1R5aHNzZHhSaU1Ka2VFS2lyVHJrY1dB?=
 =?utf-8?B?MGpWa01qS2N5T3l3Tk9DSkxSNmVGNlVEeUZ1K09rV2sxUE04NkllNTVnZnJE?=
 =?utf-8?B?TUlzenN2Z1JJd1ZlWjVwN01VRDVBdXNYem44OXBvYXFhVXJGanJGcm1zSllu?=
 =?utf-8?B?OGhHOTRaSmMyQnpJTzNvWkdTM3U4NFMwQmV1RE5PS3hkSk5tNGExTDNiZkVa?=
 =?utf-8?B?STUzdmxwU1BLSCs2NnAreURBTmdnbjJ2WFpJQ3lPZ3BncXZyUHoyVFdJOWxs?=
 =?utf-8?B?NEpFMDVPbGtwMjRzT0lGcU5Jcjh1c053M21jMVUwTWswVGFOT29STHhzVXNw?=
 =?utf-8?B?aXdhUU01YVkzTlB3WGN2Vk40dUVOeUpXOGRZeVh1RnhZNnc2NlNLNU9zUlF6?=
 =?utf-8?B?MDBaZkdaWnZzaWlVMWtNS3FCSlFuT1ZUdXdHaS9kU2wrSk1NeGwvMzNidXFW?=
 =?utf-8?B?WjVQQXBPejk3TUVOQXEyM0wzakFibFYrWUJiNXFBWVpuVGFYYXVmYzIrbThw?=
 =?utf-8?B?a3BiOVNicUtheFRqYXNIbDltV1VCaUpReGNoQ1NJQlNiRkdzZjR0TXROZFJG?=
 =?utf-8?B?R1VJWWJtSkRyNEJndjF5cnNQSDhBS2ZkN0NZMVFRZ3A3OVZGZllxRktpLzZZ?=
 =?utf-8?B?dkVDNUdCNlNCVUNMQ0RUSVdGdXVqdm53VDRHUXRoYlRZZkx0R3l5dzlFRSs5?=
 =?utf-8?B?Zm9RbHhRSTJ1VkdRRlR0aTRGaEpZOEVWdXRZNGRUa3hVRktlZlRYcWJXN24r?=
 =?utf-8?B?bEludTUxQVZnZldFWGtVT21FQTcyUy9pQzg3SE9sY25wL1FhU0ZDdlp3TS80?=
 =?utf-8?B?UFpUdkordHROTDBCMDBJb0VzWUl6N1RUMkRzNFJXdFhtTU9YaUsxZnYrV2dl?=
 =?utf-8?B?U0p5c0lhTnN6cFIrdmo0djdtWnovbVFCV1pXQnBTeDR4ZjhSSko4UjdBRmFG?=
 =?utf-8?B?bjdhWTljb2lJdTFDcFQxR083dGpwT2YxeEp5N2MyUkU5cTRFU3ZmUEI2b3F1?=
 =?utf-8?B?SGZKVDFHVEd0VU9mT3FYODYrYjBKdDhDZW1SVWdLdzkyL2thVGhyTlBFK0dO?=
 =?utf-8?B?d1JsYmkrSTZVeWIyOU83ZVJ1cC9ZYWdVWWZZU2NDQjhlK21JR0F5U3VhNHk1?=
 =?utf-8?B?UHZoenZNOVZFSnB5d2N6TDFGbmN0Z3RXa3dycmVTTnpFTCtPQ0pXTzlBOXpw?=
 =?utf-8?B?ck4zcVdGaEg0aE9CMlVRQ0MrNFluODJKREVYOFFCakVobG04WXU3bjlGMlVw?=
 =?utf-8?B?OEVzNDhqcnlYTEozTnBOTlBHMFdGbk1aT2hIeFI0YTQxUnlYa0NNNmdMb01q?=
 =?utf-8?B?TUVXSFVwZytOWjhsTmdidXFOMkZtdnhnNzFSSjk3TmVybWhlMG9CL25XM2NP?=
 =?utf-8?B?SGF3bTFqUGEwelgwTVYyVU1OM2Z1UHQxbXowNElMUHV2OER4V0tJTXIrYU9H?=
 =?utf-8?B?VlNWYVpUZzdrQW9SdGNRQWZQNnZLOUpCWFhxazE0OTVocW1aQkJsTEJZQnZp?=
 =?utf-8?B?UzQ4cTRkMFcrcXZJSEpkVFNONU9YcXJJT2ZDejdXVlZHaEF1c2ZIWUZGT2lk?=
 =?utf-8?B?NUFXL3NRQVVRK0h0ZllmMnV3SzBQTmxOTm4xRDIrWTY3VTBWV1dOVUh4YXZw?=
 =?utf-8?B?TlJlbU5QWm9jWU5mOGNEdi8zSWRLVmtRdmlTcDNoWmdEaWNqdHhlMEtPVVJn?=
 =?utf-8?B?OUV4Y05xdzEyTkh1MEJVM1V2Y2RGMFl4eEtwVnpjanl6QmZVNUVzV09STWVL?=
 =?utf-8?B?VDlBZHAzMjZSREc4ZTJCMDQrN00xbFBtbTkvTTY1eG1rbWhzVmVwbXBudkJo?=
 =?utf-8?B?RWNrTkpTckV0NzdqN1lqVG8zMkF1V2czc1hvQ2dSMkFoQlQ3Z2FOTU11WHNn?=
 =?utf-8?B?YkJWYjJacWszUER1VFVOSjY2ZWNYalpucUxEZDE5bVJmbUdDK2R0QUJSOUpR?=
 =?utf-8?B?U0FvUlU2OHZvOW5zMmh5c24yT0Q3L2lLS2cxWlRBRUkyM241d1AzV2R2RDFx?=
 =?utf-8?B?NWc9PQ==?=
X-MS-Exchange-CrossTenant-Network-Message-Id: 134237fa-f14c-43c4-c2c4-08dbfbce2c83
X-MS-Exchange-CrossTenant-AuthSource: DM6PR11MB3625.namprd11.prod.outlook.com
X-MS-Exchange-CrossTenant-AuthAs: Internal
X-MS-Exchange-CrossTenant-OriginalArrivalTime: 13 Dec 2023 11:25:13.3863
 (UTC)
X-MS-Exchange-CrossTenant-FromEntityHeader: Hosted
X-MS-Exchange-CrossTenant-Id: 46c98d88-e344-4ed4-8496-4ed7712e255d
X-MS-Exchange-CrossTenant-MailboxType: HOSTED
X-MS-Exchange-CrossTenant-UserPrincipalName: g35Zax1fuikIgOafaiVVnIKoRzaDoGWfOyl2/rVfzqzvk8fTRU33CD78TVhvWtGrc2Uu2vvCCwkYSJSF869pK+zVGlHmjWC9YxcqDw23mE4=
X-MS-Exchange-Transport-CrossTenantHeadersStamped: CYYPR11MB8429
X-OriginatorOrg: intel.com
X-Spam-Status: No, score=-0.9 required=5.0 tests=DKIMWL_WL_HIGH,DKIM_SIGNED,
	DKIM_VALID,DKIM_VALID_AU,HEADER_FROM_DIFFERENT_DOMAINS,
	MAILING_LIST_MULTI,SPF_HELO_NONE,SPF_PASS,T_SCC_BODY_TEXT_LINE
	autolearn=unavailable autolearn_force=no version=3.4.6
X-Spam-Checker-Version: SpamAssassin 3.4.6 (2021-04-09) on howler.vger.email
Precedence: bulk
List-ID: <linux-kernel.vger.kernel.org>
X-Mailing-List: linux-kernel@vger.kernel.org
X-Greylist: Sender passed SPF test, not delayed by milter-greylist-4.6.4 (howler.vger.email [0.0.0.0]); Wed, 13 Dec 2023 03:25:30 -0800 (PST)

From: Jakub Kicinski <kuba@kernel.org>
Date: Mon, 11 Dec 2023 11:23:32 -0800

> On Mon, 11 Dec 2023 11:16:20 +0100 Alexander Lobakin wrote:
>> Ideally, I'd like to pass a CPU ID this queue will be run on and use
>> cpu_to_node(), but currently there's no NUMA-aware allocations in the
>> Intel drivers and Rx queues don't get the corresponding CPU ID when
>> configuring. I may revisit this later, but for now NUMA_NO_NODE is the
>> most optimal solution here.
> 
> Hm, I've been wondering about persistent IRQ mappings. Drivers
> resetting IRQ mapping on reconfiguration is a major PITA in production
> clusters. You change the RSS hash and some NICs suddenly forget
> affinitization ????️
> 
> The connection with memory allocations changes the math on that a bit.
> 
> The question is really whether we add CPU <> NAPI config as a netdev
> Netlink API or build around the generic IRQ affinity API. The latter 
> is definitely better from "don't duplicate uAPI" perspective.
> But we need to reset the queues and reallocate their state when 
> the mapping is changed. And shutting down queues on 
> 
>   echo $cpu > /../smp_affinity_list
> 
> seems moderately insane. Perhaps some middle-ground exists.
> 
> Anyway, if you do find cycles to tackle this - pls try to do it
> generically not just for Intel? :)

Sounds good, adding to my fathomless backlog :>

Thanks,
Olek