Received-SPF: pass (google.com: domain of linux-kernel-owner@vger.kernel.org designates 23.128.96.18 as permitted sender) client-ip=23.128.96.18;
From:   Nadav Amit <nadav.amit@gmail.com>
Message-Id: <C42F16F8-646B-448D-A098-E1A98637E28D@gmail.com>
Content-Type: multipart/signed;
        boundary="Apple-Mail=_F337632E-750D-4B46-8772-8021968CA3B8";
        protocol="application/pgp-signature";
        micalg=pgp-sha256
Mime-Version: 1.0 (Mac OS X Mail 14.0 \(3654.60.0.2.21\))
Subject: Re: [RFC] NUMA balancing: reduce TLB flush via delaying mapping on
 hint page fault
Date:   Wed, 31 Mar 2021 09:36:04 -0700
In-Reply-To: <20210331131658.GV15768@suse.de>
Cc:     Linux-MM <linux-mm@kvack.org>,
        Andrew Morton <akpm@linux-foundation.org>,
        LKML <linux-kernel@vger.kernel.org>,
        Peter Zijlstra <peterz@infradead.org>,
        Peter Xu <peterx@redhat.com>,
        Johannes Weiner <hannes@cmpxchg.org>,
        Vlastimil Babka <vbabka@suse.cz>,
        Matthew Wilcox <willy@infradead.org>,
        Will Deacon <will@kernel.org>,
        Michel Lespinasse <walken@google.com>,
        Arjun Roy <arjunroy@google.com>,
        "Kirill A. Shutemov" <kirill.shutemov@linux.intel.com>,
        Mel Gorman <mgorman@suse.de>
To:     "Huang, Ying" <ying.huang@intel.com>
References: <20210329062651.2487905-1-ying.huang@intel.com>
 <20210330133310.GT15768@suse.de>
 <87a6qj8t92.fsf@yhuang6-desk1.ccr.corp.intel.com>
 <20210331131658.GV15768@suse.de>
Precedence: bulk


--Apple-Mail=_F337632E-750D-4B46-8772-8021968CA3B8
Content-Transfer-Encoding: quoted-printable
Content-Type: text/plain;
	charset=utf-8


> On Mar 31, 2021, at 6:16 AM, Mel Gorman <mgorman@suse.de> wrote:
>=20
> On Wed, Mar 31, 2021 at 07:20:09PM +0800, Huang, Ying wrote:
>> Mel Gorman <mgorman@suse.de> writes:
>>=20
>>> On Mon, Mar 29, 2021 at 02:26:51PM +0800, Huang Ying wrote:
>>>> For NUMA balancing, in hint page fault handler, the faulting page =
will
>>>> be migrated to the accessing node if necessary.  During the =
migration,
>>>> TLB will be shot down on all CPUs that the process has run on
>>>> recently.  Because in the hint page fault handler, the PTE will be
>>>> made accessible before the migration is tried.  The overhead of TLB
>>>> shooting down is high, so it's better to be avoided if possible.  =
In
>>>> fact, if we delay mapping the page in PTE until migration, that can =
be
>>>> avoided.  This is what this patch doing.
>>>>=20
>>>=20
>>> Why would the overhead be high? It was previously inaccessibly so =
it's
>>> only parallel accesses making forward progress that trigger the need
>>> for a flush.
>>=20
>> Sorry, I don't understand this.  Although the page is inaccessible, =
the
>> threads may access other pages, so TLB flushing is still necessary.
>>=20
>=20
> You assert the overhead of TLB shootdown is high and yes, it can be
> very high but you also said "the benchmark score has no visible =
changes"
> indicating the TLB shootdown cost is not a major problem for the =
workload.
> It does not mean we should ignore it though.

If you are looking for a benchmark that is negatively affected by NUMA
balancing, then IIRC Parsec=E2=80=99s dedup is such a workload. [1]

[1] https://parsec.cs.princeton.edu/

--Apple-Mail=_F337632E-750D-4B46-8772-8021968CA3B8
Content-Transfer-Encoding: 7bit
Content-Disposition: attachment;
	filename=signature.asc
Content-Type: application/pgp-signature;
	name=signature.asc
Content-Description: Message signed with OpenPGP

-----BEGIN PGP SIGNATURE-----

iQIzBAEBCAAdFiEESJL3osl5Ymx/w9I1HaAqSabaD1oFAmBkpPQACgkQHaAqSaba
D1pAvBAAjzpeqjamVJvvm0jAmkjIyImbd6zd8mCnwDXmKVaGuhPVi329I8qadJ+u
/eq0a+xVd1Rm2FbhEi4LT7LoccrIDe8959f7thT0TDIdaUGNuO83qJRIHQQZiyEL
dxwFWIfwHHvoXjGgRZZ/yHttCGNhbyh/J9RToOrEURQJm8EniBo1Ik2X2BdP2bMc
eLWzHMiXn8flthpiIrNXwKz9oThlVXg9U3VJEVzNOP5y8L9yTkg1iwhzDxSOZlNH
78/4EXTYx8PS0UD+0KLlO3d3J2sFiYnPRmBAD/oBxDBYGfprcV9CHsaJUlEh6F8V
+joOQfMvpKZNah74gw1A4JsE/8fEJGubfzZ3g+QQV40dA03xP7+mQIX5llo2h2a1
+u3rLDygT1wLzZK8jsmG6GEFcL+4JhCEMOlBq/CEtb2nAourfi9einvbWanS9bCq
Ytw5sMsc/At6KWg4uQiPuM75edvPdUjgokaE/dVZRQwesUt98dj9mjLNR44mu8N2
QI5fU5J4IDr1jR2SQrAfMRYgQQU3eftaOVf6Quvh63h2rXRv9YpmG/C2pqZcYF/Q
CA0PiqV1jB6Z+yxvTLoUuJXnCC0pl5vEhdPR3BmEDdbkDuTNmJkTiD9Pk5jEWTuR
5qnebPt+zsIjgWrO9sppk8GuYHOVAtRQuE73Nhj/d4Bxi42m4GE=
=i6Am
-----END PGP SIGNATURE-----

--Apple-Mail=_F337632E-750D-4B46-8772-8021968CA3B8--