Received: by 2002:a05:6358:e9c4:b0:b2:91dc:71ab with SMTP id hc4csp5907607rwb; Tue, 9 Aug 2022 06:12:05 -0700 (PDT) X-Google-Smtp-Source: AA6agR5FBEuHb6/w+9SRCkeL1Ck0ia+2GdWhF7sLYnxZ1PE4fqXt6lHkV4tMLrGYxwZ00QqFDaqf X-Received: by 2002:a17:907:6e12:b0:730:bc12:82e3 with SMTP id sd18-20020a1709076e1200b00730bc1282e3mr17423075ejc.36.1660050724995; Tue, 09 Aug 2022 06:12:04 -0700 (PDT) ARC-Seal: i=1; a=rsa-sha256; t=1660050724; cv=none; d=google.com; s=arc-20160816; b=K2SrgE7aN8wCOO7cQ2x0+TwhS1d2LHtmZYQkmuHGLgdccjk9V+QjKvdI0Uqpf06LVV g1GQjdg4FJUlxSkxXc9Ymm+M9f85F1wC/PMatqFb1vRH1KOEhnZjqZ+ikgqCWZ46M8Nx JzxXABwpzK5tONJPrjxFtzOfeSMzikagTek3Z3EFNDElEJGP/lehIWAzux4hyaUdxZGh egZBmYd99AttxJXQ/skaYsWFNIMuxFkFkQjTVo4F6ZkSvl5JrM3TLd53g7iugfz9/xyY fYXwsuQu/MxxEqwggMPP1fotn5x3Vo1DgP/8dCLu27N7U+3QEJewicn7ldX/PoUUkCj2 4xxA== ARC-Message-Signature: i=1; a=rsa-sha256; c=relaxed/relaxed; d=google.com; s=arc-20160816; h=list-id:precedence:mime-version:message-id:date:references :in-reply-to:subject:cc:to:from:dkim-signature; bh=rN2wh+nxvchYpu9hQV2LMnguLQFknovVkUYSriHC3N4=; b=c/Yo5FxjclWBcXi6uPkxwuh1H8jBi/237CGO3eiZAnSWKltySc8F8XksisIXMUIRtR gI49mxA3PQuDg5qa5krW8Wcvsy5lf4szhZ0OeZbrrRua8mLHMnwB0rHUjen9QXgvtBpC fsh/0P9dL3UL2Wa8dCt6R9zpoEqNt/3PT5evPCaIN3xrcHeShh4jyL6NpGE/o5yiizHG k4g8pSInKjquzlzH4QMDn9u4YmNck/SVX63PuHFg+C+MWwZCaGxpKoiLKDnOgni90K3h kCVyx2bYwwCk9zyjL6tcd27jmO8ud99RamtNT8itiAvZEaYvmE7Nh31X2F256WKpVsT5 /73w== ARC-Authentication-Results: i=1; mx.google.com; dkim=pass header.i=@redhat.com header.s=mimecast20190719 header.b=HU7ixmFA; spf=pass (google.com: domain of linux-kernel-owner@vger.kernel.org designates 2620:137:e000::1:20 as permitted sender) smtp.mailfrom=linux-kernel-owner@vger.kernel.org; dmarc=pass (p=NONE sp=NONE dis=NONE) header.from=redhat.com Return-Path: Received: from out1.vger.email (out1.vger.email. [2620:137:e000::1:20]) by mx.google.com with ESMTP id j23-20020a170906051700b007309383490esi1803550eja.122.2022.08.09.06.11.37; Tue, 09 Aug 2022 06:12:04 -0700 (PDT) Received-SPF: pass (google.com: domain of linux-kernel-owner@vger.kernel.org designates 2620:137:e000::1:20 as permitted sender) client-ip=2620:137:e000::1:20; Authentication-Results: mx.google.com; dkim=pass header.i=@redhat.com header.s=mimecast20190719 header.b=HU7ixmFA; spf=pass (google.com: domain of linux-kernel-owner@vger.kernel.org designates 2620:137:e000::1:20 as permitted sender) smtp.mailfrom=linux-kernel-owner@vger.kernel.org; dmarc=pass (p=NONE sp=NONE dis=NONE) header.from=redhat.com Received: (majordomo@vger.kernel.org) by vger.kernel.org via listexpand id S243220AbiHIMwW (ORCPT + 99 others); Tue, 9 Aug 2022 08:52:22 -0400 Received: from lindbergh.monkeyblade.net ([23.128.96.19]:46792 "EHLO lindbergh.monkeyblade.net" rhost-flags-OK-OK-OK-OK) by vger.kernel.org with ESMTP id S243189AbiHIMwV (ORCPT ); Tue, 9 Aug 2022 08:52:21 -0400 Received: from us-smtp-delivery-124.mimecast.com (us-smtp-delivery-124.mimecast.com [170.10.133.124]) by lindbergh.monkeyblade.net (Postfix) with ESMTP id 1585C12D28 for ; Tue, 9 Aug 2022 05:52:20 -0700 (PDT) DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=redhat.com; s=mimecast20190719; t=1660049539; h=from:from:reply-to:subject:subject:date:date:message-id:message-id: to:to:cc:cc:mime-version:mime-version:content-type:content-type: in-reply-to:in-reply-to:references:references; bh=rN2wh+nxvchYpu9hQV2LMnguLQFknovVkUYSriHC3N4=; b=HU7ixmFA2mebYw7myBdXKitTUCT/i1ykFuUvgN+phs8FcjZr47CxT67YslMl/BdMwisdce COlGB2fPBe1ayEHTQDaOekGeEqcDTIskKCJDO+QiXdcbrttk72muV3HWxURVgInKTXP8kz 3gYtJq+VOz3E4X8Kn3tzYmJknFRlA34= Received: from mail-wm1-f69.google.com (mail-wm1-f69.google.com [209.85.128.69]) by relay.mimecast.com with ESMTP with STARTTLS (version=TLSv1.2, cipher=TLS_ECDHE_RSA_WITH_AES_256_GCM_SHA384) id us-mta-600-gnTNjytmMN6-PfyVA-8aqQ-1; Tue, 09 Aug 2022 08:52:17 -0400 X-MC-Unique: gnTNjytmMN6-PfyVA-8aqQ-1 Received: by mail-wm1-f69.google.com with SMTP id u12-20020a05600c034c00b003a5124600c5so2283254wmd.4 for ; Tue, 09 Aug 2022 05:52:17 -0700 (PDT) X-Google-DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=1e100.net; s=20210112; h=mime-version:message-id:date:references:in-reply-to:subject:cc:to :from:x-gm-message-state:from:to:cc; bh=rN2wh+nxvchYpu9hQV2LMnguLQFknovVkUYSriHC3N4=; b=FWsH1rHulppjX8Fe5bxX374p04klhj35tv0Td6nEJprY9nRHapURKlpKBkanFO8bOh Aqjy6HSMKtGiKzxNDNeC1AbcbXA4FC6bzWt9vcmUsq7oQMisTQZnYQjs9RXLvjCjcmiX YzSB7rcW1Ea1KSbYAIQKppytMwvC17KsWkf30Y/oKTkpwqfciEjfn00RR0BWZvn8ZUuW savrZLZvpjkAg4o623VOt5nZBCd0pJGOnPAqBt3T35N+6M56FwKllj6A7/0stkPvpIT5 gS2ObeNR3yLsEpYqZ0dPhgINWEDNljtGcEMY/qock/96036TA8kIFCbZokdACihqgD5/ sDAg== X-Gm-Message-State: ACgBeo3IjznnNFwWKReclMb2QZyPxk6Uo0qfkxBc8QbI/17Oc8192ISH ygq+KGxzmXZcIcrS5p/6z4NBWW5Axg0u/5C0NponOzAvggVumBPySX8pvS1Ge1NLX6n6VFwCxXH BwtTdo/JFToZsZkLs9AqOxKICSIyPzxQ9+bKfFRXZZ+cxXoE+4YdD3agUZSkpg2yHOjdWW3SwjU mA X-Received: by 2002:a05:600c:4f95:b0:3a3:4612:6884 with SMTP id n21-20020a05600c4f9500b003a346126884mr20920895wmq.39.1660049535837; Tue, 09 Aug 2022 05:52:15 -0700 (PDT) X-Received: by 2002:a05:600c:4f95:b0:3a3:4612:6884 with SMTP id n21-20020a05600c4f9500b003a346126884mr20920870wmq.39.1660049535620; Tue, 09 Aug 2022 05:52:15 -0700 (PDT) Received: from vschneid.remote.csb ([185.11.37.247]) by smtp.gmail.com with ESMTPSA id 6-20020a1c1906000000b003a511e92abcsm17149747wmz.34.2022.08.09.05.52.13 (version=TLS1_3 cipher=TLS_AES_256_GCM_SHA384 bits=256/256); Tue, 09 Aug 2022 05:52:14 -0700 (PDT) From: Valentin Schneider To: Tariq Toukan , Tariq Toukan , "David S. Miller" , Saeed Mahameed , Jakub Kicinski , Ingo Molnar , Peter Zijlstra , Juri Lelli Cc: Eric Dumazet , Paolo Abeni , netdev@vger.kernel.org, Gal Pressman , Vincent Guittot , linux-kernel@vger.kernel.org Subject: Re: [PATCH net-next V4 1/3] sched/topology: Add NUMA-based CPUs spread API In-Reply-To: <12fd25f9-96fb-d0e0-14ec-3f08c01a5a4b@gmail.com> References: <20220728191203.4055-1-tariqt@nvidia.com> <20220728191203.4055-2-tariqt@nvidia.com> <12fd25f9-96fb-d0e0-14ec-3f08c01a5a4b@gmail.com> Date: Tue, 09 Aug 2022 13:52:13 +0100 Message-ID: MIME-Version: 1.0 Content-Type: text/plain X-Spam-Status: No, score=-2.7 required=5.0 tests=BAYES_00,DKIMWL_WL_HIGH, DKIM_SIGNED,DKIM_VALID,DKIM_VALID_AU,DKIM_VALID_EF,RCVD_IN_DNSWL_NONE, SPF_HELO_NONE,SPF_NONE,T_SCC_BODY_TEXT_LINE autolearn=ham autolearn_force=no version=3.4.6 X-Spam-Checker-Version: SpamAssassin 3.4.6 (2021-04-09) on lindbergh.monkeyblade.net Precedence: bulk List-ID: X-Mailing-List: linux-kernel@vger.kernel.org On 09/08/22 13:18, Tariq Toukan wrote: > On 8/9/2022 1:02 PM, Valentin Schneider wrote: >> >> Are there cases where we can't figure this out in advance? From what I grok >> out of the two callsites you patched, all vectors will be used unless some >> error happens, so compressing the CPUs in a single cpumask seemed >> sufficient. >> > > All vectors will be initialized to support the maximum number of traffic > rings. However, the actual number of traffic rings can be controlled and > set to a lower number N_actual < N. In this case, we'll be using only > N_actual instances and we want them to be the first/closest. Ok, that makes sense, thank you. In that case I wonder if we'd want a public-facing iterator for sched_domains_numa_masks[%i][node], rather than copy a portion of it. Something like the below (naming and implementation haven't been thought about too much). const struct cpumask *sched_numa_level_mask(int node, int level) { struct cpumask ***masks = rcu_dereference(sched_domains_numa_masks); if (node >= nr_node_ids || level >= sched_domains_numa_levels) return NULL; if (!masks) return NULL; return masks[level][node]; } EXPORT_SYMBOL_GPL(sched_numa_level_mask); #define for_each_numa_level_mask(node, lvl, mask) \ for (mask = sched_numa_level_mask(node, lvl); mask; \ mask = sched_numa_level_mask(node, ++lvl)) void foo(int node, int cpus[], int ncpus) { const struct cpumask *mask; int lvl = 0; int i = 0; int cpu; rcu_read_lock(); for_each_numa_level_mask(node, lvl, mask) { for_each_cpu(cpu, mask) { cpus[i] = cpu; if (++i == ncpus) goto done; } } done: rcu_read_unlock(); }