Received-SPF: pass (google.com: domain of linux-kernel-owner@vger.kernel.org designates 2620:137:e000::1:20 as permitted sender) client-ip=2620:137:e000::1:20;
Date:   Mon, 20 Jun 2022 11:31:39 -0700
From:   Luis Chamberlain <mcgrof@kernel.org>
To:     Aaron Lu <aaron.lu@intel.com>, Davidlohr Bueso <dave@stgolabs.net>
Cc:     Song Liu <song@kernel.org>, linux-kernel@vger.kernel.org,
        bpf@vger.kernel.org, linux-mm@kvack.org, ast@kernel.org,
        daniel@iogearbox.net, peterz@infradead.org,
        torvalds@linux-foundation.org, rick.p.edgecombe@intel.com,
        kernel-team@fb.com
Subject: Re: [PATCH v4 bpf-next 0/8] bpf_prog_pack followup
Message-ID: <YrC9CyOPamPneUOT@bombadil.infradead.org>
References: <20220520235758.1858153-1-song@kernel.org>
 <YrBV8darrlmUnrHR@ziqianlu-Dell-Optiplex7000>
MIME-Version: 1.0
Content-Type: text/plain; charset=us-ascii
Content-Disposition: inline
In-Reply-To: <YrBV8darrlmUnrHR@ziqianlu-Dell-Optiplex7000>
Sender: Luis Chamberlain <mcgrof@infradead.org>
Precedence: bulk

On Mon, Jun 20, 2022 at 07:11:45PM +0800, Aaron Lu wrote:
> Hi Song,
> 
> On Fri, May 20, 2022 at 04:57:50PM -0700, Song Liu wrote:
> 
> ... ...
> 
> > The primary goal of bpf_prog_pack is to reduce iTLB miss rate and reduce
> > direct memory mapping fragmentation. This leads to non-trivial performance
> > improvements.
> >
> > For our web service production benchmark, bpf_prog_pack on 4kB pages
> > gives 0.5% to 0.7% more throughput than not using bpf_prog_pack.
> > bpf_prog_pack on 2MB pages 0.6% to 0.9% more throughput than not using
> > bpf_prog_pack. Note that 0.5% is a huge improvement for our fleet. I
> > believe this is also significant for other companies with many thousand
> > servers.
> >
> 
> I'm evaluationg performance impact due to direct memory mapping
> fragmentation 

BTW how exactly are you doing this?

  Luis

> and seeing the above, I wonder: is the performance improve
> mostly due to prog pack and hugepage instead of less direct mapping
> fragmentation?
> 
> I can understand that when progs are packed together, iTLB miss rate will
> be reduced and thus, performance can be improved. But I don't see
> immediately how direct mapping fragmentation can impact performance since
> the bpf code are running from the module alias addresses, not the direct
> mapping addresses IIUC?
> 
> I appreciate it if you can shed some light on performance impact direct
> mapping fragmentation can cause, thanks.