Received-SPF: pass (google.com: domain of linux-kernel-owner@vger.kernel.org designates 2620:137:e000::1:20 as permitted sender) client-ip=2620:137:e000::1:20;
Content-Type: text/plain;
        charset=us-ascii
Mime-Version: 1.0 (Mac OS X Mail 16.0 \(3696.120.41.1.1\))
Subject: Re: [PATCH v3 5/7] mm: Remember young/dirty bit for page migrations
From:   Nadav Amit <nadav.amit@gmail.com>
In-Reply-To: <YvqcGq44oonHNyCO@xz-m1.local>
Date:   Mon, 15 Aug 2022 13:52:49 -0700
Cc:     "Huang, Ying" <ying.huang@intel.com>, linux-mm@kvack.org,
        linux-kernel@vger.kernel.org, Minchan Kim <minchan@kernel.org>,
        David Hildenbrand <david@redhat.com>,
        Andrew Morton <akpm@linux-foundation.org>,
        Hugh Dickins <hughd@google.com>,
        Vlastimil Babka <vbabka@suse.cz>,
        Andrea Arcangeli <aarcange@redhat.com>,
        Andi Kleen <andi.kleen@intel.com>,
        "Kirill A . Shutemov" <kirill@shutemov.name>
Content-Transfer-Encoding: quoted-printable
Message-Id: <5B21352C-2BE6-4070-BB6B-C1B7A5D4D225@gmail.com>
References: <20220809220100.20033-1-peterx@redhat.com>
 <20220809220100.20033-6-peterx@redhat.com> <YvUeB0jc6clz59z5@xz-m1.local>
 <87pmh6dwdr.fsf@yhuang6-desk2.ccr.corp.intel.com>
 <YvqcGq44oonHNyCO@xz-m1.local>
To:     Peter Xu <peterx@redhat.com>
Precedence: bulk

On Aug 15, 2022, at 12:18 PM, Peter Xu <peterx@redhat.com> wrote:

> On Fri, Aug 12, 2022 at 10:32:48AM +0800, Huang, Ying wrote:
>> Peter Xu <peterx@redhat.com> writes:
>>=20
>>> On Tue, Aug 09, 2022 at 06:00:58PM -0400, Peter Xu wrote:
>>>> diff --git a/mm/migrate_device.c b/mm/migrate_device.c
>>>> index 27fb37d65476..699f821b8443 100644
>>>> --- a/mm/migrate_device.c
>>>> +++ b/mm/migrate_device.c
>>>> @@ -221,6 +221,10 @@ static int migrate_vma_collect_pmd(pmd_t =
*pmdp,
>>>> 			else
>>>> 				entry =3D make_readable_migration_entry(
>>>> 							=
page_to_pfn(page));
>>>> +			if (pte_young(pte))
>>>> +				entry =3D =
make_migration_entry_young(entry);
>>>> +			if (pte_dirty(pte))
>>>> +				entry =3D =
make_migration_entry_dirty(entry);
>>>> 			swp_pte =3D swp_entry_to_pte(entry);
>>>> 			if (pte_present(pte)) {
>>>> 				if (pte_soft_dirty(pte))
>>>=20
>>> This change needs to be wrapped with pte_present() at least..
>>>=20
>>> I also just noticed that this change probably won't help anyway =
because:
>>>=20
>>>  (1) When ram->device, the pte will finally be replaced with a =
device
>>>      private entry, and device private entry does not yet support =
A/D, it
>>>      means A/D will be dropped again,
>>>=20
>>>  (2) When device->ram, we are missing information on either A/D =
bits, or
>>>      even if device private entries start to suport A/D, it's still =
not
>>>      clear whether we should take device read/write into =
considerations
>>>      too on the page A/D bits to be accurate.
>>>=20
>>> I think I'll probably keep the code there for completeness, but I =
think it
>>> won't really help much until more things are done.
>>=20
>> It appears that there are more issues.  Between "pte =3D *ptep" and =
pte
>> clear, CPU may set A/D bit in PTE, so we may need to update pte when
>> clearing PTE.
>=20
> Agreed, I didn't see it a huge problem with current code, but it =
should be
> better in that way.
>=20
>> And I don't find the TLB is flushed in some cases after PTE is =
cleared.
>=20
> I think it's okay to not flush tlb if pte not present.  But maybe =
you're
> talking about something else?

I think Huang refers to situation in which the PTE is cleared, still not
flushed, and then A/D is being set by the hardware.

At least on x86, the hardware is not supposed to do so. The only case I
remember (and sometimes misremembers) is with KNL erratum, which perhaps
needs to be considered:

https://lore.kernel.org/all/20160708001911.9A3FD2B6@viggo.jf.intel.com/