Received-SPF: pass (google.com: best guess record for domain of linux-kernel-owner@vger.kernel.org designates 209.132.180.67 as permitted sender) client-ip=209.132.180.67;
Message-ID: <18c5c378db98f223a0663034baa9fd6ce42f1ec7.camel@linux.ibm.com>
Subject: Re: [PATCH v2 11/11] powerpc/mm/book3s64/pgtable: Uses counting
 method to skip serializing
From:   Leonardo Bras <leonardo@linux.ibm.com>
To:     John Hubbard <jhubbard@nvidia.com>, linuxppc-dev@lists.ozlabs.org,
        linux-kernel@vger.kernel.org, Linux-MM <linux-mm@kvack.org>
Cc:     Jason Gunthorpe <jgg@ziepe.ca>,
        Thomas Gleixner <tglx@linutronix.de>,
        Arnd Bergmann <arnd@arndb.de>,
        Greg Kroah-Hartman <gregkh@linuxfoundation.org>,
        YueHaibing <yuehaibing@huawei.com>,
        Keith Busch <keith.busch@intel.com>,
        Nicholas Piggin <npiggin@gmail.com>,
        Mike Rapoport <rppt@linux.ibm.com>,
        Mahesh Salgaonkar <mahesh@linux.vnet.ibm.com>,
        Richard Fontana <rfontana@redhat.com>,
        Paul Mackerras <paulus@samba.org>,
        "Aneesh Kumar K.V" <aneesh.kumar@linux.ibm.com>,
        Ganesh Goudar <ganeshgr@linux.ibm.com>,
        Andrew Morton <akpm@linux-foundation.org>,
        Ira Weiny <ira.weiny@intel.com>,
        Dan Williams <dan.j.williams@intel.com>,
        Allison Randal <allison@lohutok.net>
Date:   Mon, 23 Sep 2019 14:25:51 -0300
In-Reply-To: <4ea26ffb-ad03-bdff-7893-95332b22a5fd@nvidia.com>
References: <20190920195047.7703-1-leonardo@linux.ibm.com>
         <20190920195047.7703-12-leonardo@linux.ibm.com>
         <1b39eaa7-751d-40bc-d3d7-41aaa15be42a@nvidia.com>
         <24863d8904c6e05e5dd48cab57db4274675ae654.camel@linux.ibm.com>
         <4ea26ffb-ad03-bdff-7893-95332b22a5fd@nvidia.com>
Content-Type: multipart/signed; micalg="pgp-sha256";
        protocol="application/pgp-signature"; boundary="=-QkhmEMJu3huKrEpzj07X"
User-Agent: Evolution 3.30.5 (3.30.5-1.fc29) 
MIME-Version: 1.0
Sender: linux-kernel-owner@vger.kernel.org
Precedence: bulk


--=-QkhmEMJu3huKrEpzj07X
Content-Type: text/plain; charset="UTF-8"
Content-Transfer-Encoding: quoted-printable

On Fri, 2019-09-20 at 17:48 -0700, John Hubbard wrote:
>=20
[...]
> So it seems that full memory barriers (not just compiler barriers) are re=
quired.
> If the irq enable/disable somehow provides that, then your new code just =
goes
> along for the ride and Just Works. (You don't have any memory barriers in
> start_lockless_pgtbl_walk() / end_lockless_pgtbl_walk(), just the compile=
r
> barriers provided by the atomic inc/dec.)
>=20
> So it's really a pre-existing question about the correctness of the gup_f=
ast()
> irq disabling approach.

I am not experienced in other archs, and I am still pretty new to
Power, but by what I could understand, this behavior is better
explained in serialize_against_pte_lookup.=20

What happens here is that, before doing a THP split/collapse, the
function does a update of the pmd and a serialize_against_pte_lookup,
in order do avoid a invalid output on a lockless pagetable walk.

Serialize basically runs a do_nothing in every cpu related to the
process, and wait for it to return.=20

This running depends on interrupt being enabled, so disabling it before
gup_pgd_range() and re-enabling after the end, makes the THP
split/collapse wait for gup_pgd_range() completion in every cpu before
continuing. (here happens the lock)

(As told before, every gup_pgd_range() that occurs after it uses a
updated pmd, so no problem.)

I am sure other archs may have a similar mechanism using
local_irq_{disable,enable}.

Did it answer your questions?

Best regards,

Leonardo Bras

--=-QkhmEMJu3huKrEpzj07X
Content-Type: application/pgp-signature; name="signature.asc"
Content-Description: This is a digitally signed message part
Content-Transfer-Encoding: 7bit

-----BEGIN PGP SIGNATURE-----

iQIzBAABCAAdFiEEMdeUgIzgjf6YmUyOlQYWtz9SttQFAl2JAB8ACgkQlQYWtz9S
ttRr1Q//T4jMYmkfmx27+Bg69sKonvlgnqSo2pOluY9vzcehO8orfWtP3QQO2WNv
ruCYAfCtzUh9x6sByQylolwbt1e/8dI82HBybU4Cy0lgP7ZfDTe+YiWJzMUiUh1l
Mm36hPPdVyN3EC0UgWQ7nnfnk2xR7UDOfw701rOzvIsK9P568qP65Iryf7uu0V3n
jvZ5JhVJX9eit1OMfEfVBvZigAuv4eAMvu0LK4ko2uTRUgRdkgZa+Pgf9N38Ok8/
iaHvDIcbUaZM/PiHkMSdh1iypyAnkGEYb9zpu0He41DiMOsjSkspvcJPRv19/N7X
p71AztGoVXZd02F6tAMkO+794GqIT2/sixX3gUPCCY7d1uhae2d/u5lMhvXK+sCS
ktivTMvPqwVfAOGpMheFkugQAu7GhGkUvhslC+YXcZUZJDShK1exMfpbF9slvpEy
x8/IkF0EGNWei1jw9iW5ic6b9Abk0WoxwbAAjCgZIkbSliZp9O5eD5+woyMn6DRT
c9CSagzNt6ztFsoAMFbtGHq0bgyW8THVNt9Oq1mijpPEwmfGoa7yfMRmt4+KU4bh
n6NHbeStFsqhe+59pFTpLRRFkTbMoTQHWsNCLVRlG6fTVZOp0kurs2IjywGqVV8X
KyFpkCPlUtX8N69af3sVNbUHG11DDvFF4F+FPPjpRnCpqiyRs3E=
=BjCh
-----END PGP SIGNATURE-----

--=-QkhmEMJu3huKrEpzj07X--