Received-SPF: pass (google.com: domain of linux-kernel-owner@vger.kernel.org designates 23.128.96.18 as permitted sender) client-ip=23.128.96.18;
From:   Uladzislau Rezki <urezki@gmail.com>
Date:   Sat, 4 Jul 2020 00:22:26 +0200
To:     Matthew Wilcox <willy@infradead.org>
Cc:     Uladzislau Rezki <urezki@gmail.com>,
        Linus Torvalds <torvalds@linux-foundation.org>,
        peter enderborg <peter.enderborg@sony.com>,
        Linux Kernel Mailing List <linux-kernel@vger.kernel.org>,
        Linux-MM <linux-mm@kvack.org>,
        Andrew Morton <akpm@linux-foundation.org>,
        GregKroah-Hartmangregkh@linuxfoundation.org,
        Peter Zijlstra <peterz@infradead.org>
Subject: Re: nr_cpu_ids vs AMD 3970x(32 physical CPUs)
Message-ID: <20200703222226.GA7472@pc636>
References: <20200703155749.GA6255@pc636>
 <8a2a55e6-6087-e4bf-3d35-ed4b4c216369@sony.com>
 <20200703192807.GB5207@pc636>
 <CAHk-=wiagrzJs9OBe_6bHK+Cc6RdoCOV85CiJAd3MhYnc8idfw@mail.gmail.com>
 <20200703212047.GA6856@pc636>
 <20200703215157.GI25523@casper.infradead.org>
MIME-Version: 1.0
Content-Type: text/plain; charset=us-ascii
Content-Disposition: inline
In-Reply-To: <20200703215157.GI25523@casper.infradead.org>
User-Agent: Mutt/1.10.1 (2018-07-13)
Sender: linux-kernel-owner@vger.kernel.org
Precedence: bulk

On Fri, Jul 03, 2020 at 10:51:57PM +0100, Matthew Wilcox wrote:
> On Fri, Jul 03, 2020 at 11:20:47PM +0200, Uladzislau Rezki wrote:
> > Some background:
> > Actually i have been thinking about making vmalloc address space to
> > be per-CPU, i.e. divide it to per-CPU address space making an allocation
> > lock-less. It will eliminate a high lock contention. When i have done
> > a prototype i noticed and realized that there is a silly issue with
> > nr_cpu_ids on some systems.
> 
> vfree() may happen on a different CPU from the one which called vmalloc(),
> so I'm not sure you're going to get as large a win as you think you will.
>
Hmm.. According to my tests the difference is approximately 7x/10x but
i also need to say as of now those tests are draft. Indeed vfree() can
be done on different CPU, but i do not think it is a big issue. The main
goal is to make the vmalloc() to be scaled to number of CPUs in a system.
Because as number of CPUs increase as tight an allocation becomes.

Doing vfree() on another CPU would be kind of noise(critical section is
short), whereas other ones will be able to do progress because of their
own locks.

--
Vlad Rezki