Received-SPF: pass (google.com: domain of linux-kernel-owner@vger.kernel.org designates 2620:137:e000::1:20 as permitted sender) client-ip=2620:137:e000::1:20;
From:   "Huang, Ying" <ying.huang@intel.com>
To:     Nadav Amit <namit@vmware.com>
Cc:     Andrew Morton <akpm@linux-foundation.org>,
        "linux-mm@kvack.org" <linux-mm@kvack.org>,
        "linux-kernel@vger.kernel.org" <linux-kernel@vger.kernel.org>,
        kernel test robot <yujie.liu@intel.com>,
        "Mel Gorman" <mgorman@techsingularity.net>,
        Hugh Dickins <hughd@google.com>,
        Matthew Wilcox <willy@infradead.org>,
        David Hildenbrand <david@redhat.com>
Subject: Re: [PATCH] mm,unmap: avoid flushing TLB in batch if PTE is
 inaccessible
References: <20230410075224.827740-1-ying.huang@intel.com>
        <402A3E9D-5136-4747-91FF-C3AA2D557784@vmware.com>
        <87zg7f19xu.fsf@yhuang6-desk2.ccr.corp.intel.com>
        <D432368D-7E3F-47C8-8BE3-A0D11BC6EA2D@vmware.com>
        <87sfd5zx5b.fsf@yhuang6-desk2.ccr.corp.intel.com>
        <03BCE979-33B1-486F-A969-0475A35DEBB5@vmware.com>
Date:   Tue, 18 Apr 2023 11:17:52 +0800
In-Reply-To: <03BCE979-33B1-486F-A969-0475A35DEBB5@vmware.com> (Nadav Amit's
        message of "Wed, 12 Apr 2023 17:00:49 +0000")
Message-ID: <87a5z5vpy7.fsf@yhuang6-desk2.ccr.corp.intel.com>
User-Agent: Gnus/5.13 (Gnus v5.13) Emacs/27.1 (gnu/linux)
MIME-Version: 1.0
Content-Type: text/plain; charset=utf-8
Content-Transfer-Encoding: quoted-printable
Precedence: bulk

Nadav Amit <namit@vmware.com> writes:

>> On Apr 11, 2023, at 6:50 PM, Huang, Ying <ying.huang@intel.com> wrote:
>>=20
>> !! External Email
>>=20
>> Nadav Amit <namit@vmware.com> writes:
>>=20
>>>> On Apr 10, 2023, at 6:31 PM, Huang, Ying <ying.huang@intel.com> wrote:
>>>>=20
>>>> !! External Email
>>>>=20
>>>> Hi, Amit,
>>>>=20
>>>> Thank you very much for review!
>>>>=20
>>>> Nadav Amit <namit@vmware.com> writes:
>>>>=20
>>>>>> On Apr 10, 2023, at 12:52 AM, Huang Ying <ying.huang@intel.com> wrot=
e:
>>>>>>=20
>>>>>> 0Day/LKP reported a performance regression for commit
>>>>>> 7e12beb8ca2a ("migrate_pages: batch flushing TLB"). In the commit, t=
he
>>>>>> TLB flushing during page migration is batched.  So, in
>>>>>> try_to_migrate_one(), ptep_clear_flush() is replaced with
>>>>>> set_tlb_ubc_flush_pending().  In further investigation, it is found
>>>>>> that the TLB flushing can be avoided in ptep_clear_flush() if the PTE
>>>>>> is inaccessible.  In fact, we can optimize in similar way for the
>>>>>> batched TLB flushing too to improve the performance.
>>>>>>=20
>>>>>> So in this patch, we check pte_accessible() before
>>>>>> set_tlb_ubc_flush_pending() in try_to_unmap/migrate_one().  Tests sh=
ow
>>>>>> that the benchmark score of the anon-cow-rand-mt test case of
>>>>>> vm-scalability test suite can improve up to 2.1% with the patch on a
>>>>>> Intel server machine.  The TLB flushing IPI can reduce up to 44.3%.
>>>>>=20
>>>>> LGTM.
>>>>=20
>>>> Thanks!
>>>>=20
>>>>> I know it=E2=80=99s meaningless for x86 (but perhaps ARM would use th=
is infra
>>>>> too): do we need smp_mb__after_atomic() after ptep_get_and_clear() and
>>>>> before pte_accessible()?
>>>>=20
>>>> Why do we need the memory barrier?  IIUC, the PTL is locked, so PTE
>>>> value will not be changed under us.  Anything else?
>>>=20
>>> I was thinking about the ordering with respect to
>>> atomic_read(&mm->tlb_flush_pending), which is not protected by the PTL.
>>> I guess you can correctly argue that because of other control-flow
>>> dependencies, the barrier is not necessary.
>>=20
>> For ordering between ptep_get_and_clear() and
>> atomic_read(&mm->tlb_flush_pending), I think PTL has provided the
>> necessary protection already.  The code path to write
>> mm->tlb_flush_pending is,
>>=20
>>  tlb_gather_mmu
>>    inc_tlb_flush_pending       a)
>>  lock PTL
>>  change PTE                    b)
>>  unlock PTL
>>  tlb_finish_mmu
>>    dec_tlb_flush_pending       c)
>>=20
>> While code path of try_to_unmap/migrate_one is,
>>=20
>>  lock PTL
>>  read and change PTE           d)
>>  read mm->tlb_flush_pending    e)
>>  unlock PTL
>>=20
>> Even if e) occurs before d), they cannot occur at the same time of b).
>> Do I miss anything?
>
> You didn=E2=80=99t miss anything. I went over the comment on
> inc_tlb_flush_pending() and you follow the scheme.

Thanks!  Can I get your acked-by or reviewed-by for this patch?

Best Regards,
Huang, Ying