Received-SPF: pass (google.com: domain of linux-kernel+bounces-84310-linux.lists.archive=gmail.com@vger.kernel.org designates 2604:1380:40f1:3f00::1 as permitted sender) client-ip=2604:1380:40f1:3f00::1;
Date: Tue, 27 Feb 2024 17:13:16 -0800
From: Jakub Kicinski <kuba@kernel.org>
To: "Lameter, Christopher" <cl@os.amperecomputing.com>
Cc: Eric Dumazet <edumazet@google.com>, Adam Li
 <adamli@os.amperecomputing.com>, corbet@lwn.net, davem@davemloft.net,
 pabeni@redhat.com, willemb@google.com, yangtiezhu@loongson.cn,
 atenart@kernel.org, kuniyu@amazon.com, wuyun.abel@bytedance.com,
 leitao@debian.org, alexander@mihalicyn.com, dhowells@redhat.com,
 paulmck@kernel.org, joel.granados@gmail.com, urezki@gmail.com,
 joel@joelfernandes.org, linux-doc@vger.kernel.org,
 linux-kernel@vger.kernel.org, netdev@vger.kernel.org,
 patches@amperecomputing.com, shijie@os.amperecomputing.com
Subject: Re: [PATCH] net: make SK_MEMORY_PCPU_RESERV tunable
Message-ID: <20240227171316.40fe9c35@kernel.org>
In-Reply-To: <bc168824-25dd-7541-1a34-38b1a3c00489@os.amperecomputing.com>
References: <20240226022452.20558-1-adamli@os.amperecomputing.com>
	<CANn89iLbA4_YdQrF+9Rmv2uVSb1HLhu0qXqCm923FCut1E78FA@mail.gmail.com>
	<bc168824-25dd-7541-1a34-38b1a3c00489@os.amperecomputing.com>
Precedence: bulk
MIME-Version: 1.0
Content-Type: text/plain; charset=US-ASCII
Content-Transfer-Encoding: 7bit

On Tue, 27 Feb 2024 15:08:18 -0800 (PST) Lameter, Christopher wrote:
> > This looks good, do you have any performance numbers to share ?
> >
> > On a host with 384 threads, 384*16 ->  6 GB of memory.  
> 
> Those things also come with corresponding memories of a couple of TB...

We have a lot of machines at Meta with more cores than gigabytes of
memory. Keying on amount of memory would make sense. Something like
max(1MB, sk_mem / cores / 8) comes to mind?

In fact it may be a better idea to have the sysctl control the divisor
(the 8 in my example above). I had issues in the past with people
"micro-optimizing" the absolute size, forgetting about it, moving
workload to a larger machine and then complaining TCP is choking :(