Received-SPF: pass (google.com: domain of linux-kernel-owner@vger.kernel.org designates 2620:137:e000::1:18 as permitted sender) client-ip=2620:137:e000::1:18;
Date:   Wed, 16 Mar 2022 20:10:07 -0700
From:   Jakub Kicinski <kuba@kernel.org>
To:     =?UTF-8?B?w43DsWlnbw==?= Huguet <ihuguet@redhat.com>
Cc:     davem@davemloft.net, edumazet@google.com, bigeasy@linutronix.de,
        atenart@kernel.org, imagedong@tencent.com, petrm@nvidia.com,
        arnd@arndb.de, netdev@vger.kernel.org,
        linux-kernel@vger.kernel.org, michael.chan@broadcom.com,
        Siva Reddy Kallam <siva.kallam@broadcom.com>,
        Prashant Sreedharan <prashant@broadcom.com>,
        Rasesh Mody <rmody@marvell.com>,
        Sunil Goutham <sgoutham@marvell.com>,
        Raju Rangoju <rajur@chelsio.com>,
        Ajit Khaparde <ajit.khaparde@broadcom.com>,
        Sriharsha Basavapatna <sriharsha.basavapatna@broadcom.com>,
        Somnath Kotur <somnath.kotur@broadcom.com>,
        Chris Lee <christopher.lee@cspi.com>,
        Jon Mason <jdmason@kudzu.us>,
        Simon Horman <simon.horman@corigine.com>,
        Ariel Elior <aelior@marvell.com>,
        Manish Chopra <manishc@marvell.com>,
        Jose Abreu <Jose.Abreu@synopsys.com>
Subject: Re: [PATCH net-next] net: set default rss queues num to physical
 cores / 2
Message-ID: <20220316201007.41aeb5f7@kicinski-fedora-pc1c0hjn.dhcp.thefacebook.com>
In-Reply-To: <20220315091832.13873-1-ihuguet@redhat.com>
References: <20220315091832.13873-1-ihuguet@redhat.com>
MIME-Version: 1.0
Content-Type: text/plain; charset=UTF-8
Content-Transfer-Encoding: quoted-printable
Precedence: bulk

On Tue, 15 Mar 2022 10:18:32 +0100 =C3=8D=C3=B1igo Huguet wrote:
> Network drivers can call to netif_get_num_default_rss_queues to get the
> default number of receive queues to use. Right now, this default number
> is min(8, num_online_cpus()).
>=20
> Instead, as suggested by Jakub, use the number of physical cores divided
> by 2 as a way to avoid wasting CPU resources and to avoid using both CPU
> threads, but still allowing to scale for high-end processors with many
> cores.
>=20
> As an exception, select 2 queues for processors with 2 cores, because
> otherwise it won't take any advantage of RSS despite being SMP capable.
>=20
> Tested: Processor Intel Xeon E5-2620 (2 sockets, 6 cores/socket, 2
> threads/core). NIC Broadcom NetXtreme II BCM57810 (10GBps). Ran some
> tests with `perf stat iperf3 -R`, with parallelisms of 1, 8 and 24,
> getting the following results:
> - Number of queues: 6 (instead of 8)
> - Network throughput: not affected
> - CPU usage: utilized 0.05-0.12 CPUs more than before (having 24 CPUs
>   this is only 0.2-0.5% higher)
> - Reduced the number of context switches by 7-50%, being more noticeable
>   when using a higher number of parallel threads.

Thanks for following up, Inigo!

Heads up for the maintainers of drivers which use
netif_get_num_default_rss_queues() today - please note the above -=20
the default number of Rx queues may change for you starting with=20
the 5.18 kernel.