Received-SPF: pass (google.com: domain of linux-kernel+bounces-82067-linux.lists.archive=gmail.com@vger.kernel.org designates 2604:1380:45e3:2400::1 as permitted sender) client-ip=2604:1380:45e3:2400::1;
Date: Mon, 26 Feb 2024 10:32:38 -0700
From: Alex Williamson <alex.williamson@redhat.com>
To: Yisheng Xie <ethan.xys@linux.alibaba.com>, akpm@linux-foundation.org
Cc: kvm@vger.kernel.org, linux-kernel@vger.kernel.org, linux-mm@kvack.org
Subject: Re: [PATCH] vfio/type1: unpin PageReserved page
Message-ID: <20240226103238.75ad4b24.alex.williamson@redhat.com>
In-Reply-To: <e10ace3f-78d3-4843-8028-a0e1cd107c15@linux.alibaba.com>
References: <20240226160106.24222-1-ethan.xys@linux.alibaba.com>
	<20240226091438.1fc37957.alex.williamson@redhat.com>
	<e10ace3f-78d3-4843-8028-a0e1cd107c15@linux.alibaba.com>
Precedence: bulk
MIME-Version: 1.0
Content-Type: text/plain; charset=UTF-8
Content-Transfer-Encoding: quoted-printable

On Tue, 27 Feb 2024 01:14:54 +0800
Yisheng Xie <ethan.xys@linux.alibaba.com> wrote:

> =E5=9C=A8 2024/2/27 00:14, Alex Williamson =E5=86=99=E9=81=93:
> > On Tue, 27 Feb 2024 00:01:06 +0800
> > Yisheng Xie<ethan.xys@linux.alibaba.com>  wrote:
> > =20
> >> We meet a warning as following:
> >>   WARNING: CPU: 99 PID: 1766859 at mm/gup.c:209 try_grab_page.part.0+0=
xe8/0x1b0
> >>   CPU: 99 PID: 1766859 Comm: qemu-kvm Kdump: loaded Tainted: GOE  5.10=
134-008.2.x86_64 #1 =20
> >                                                                     ^^^=
^^^^^
> >
> > Does this issue reproduce on mainline?  Thanks, =20
>=20
> I have check the code of mainline, the logical seems the same as my=20
> version.
>=20
> so I think it can reproduce if i understand correctly.

I obviously can't speak to what's in your 5.10.134-008.2 kernel, but I
do know there's a very similar issue resolved in v6.0 mainline and
included in v5.10.146 of the stable tree.  Please test.  Thanks,

Alex

> >>   Hardware name: Foxconn AliServer-Thor-04-12U-v2/Thunder2, BIOS 1.0.P=
L.FC.P.031.00 05/18/2022
> >>   RIP: 0010:try_grab_page.part.0+0xe8/0x1b0
> >>   Code: b9 00 04 00 00 83 e6 01 74 ca 48 8b 32 b9 00 04 00 00 f7 c6 00=
 00 01 00 74 ba eb 91 8b 57 34 48 89 f8 85 d2 0f 8f 48 ff ff ff <0f> 0b 31 =
c0 c3 48 89 fa 48 8b 0a f7 c1 00 00 01 00 0f 85 5c ff ff
> >>   RSP: 0018:ffffc900b1a63b98 EFLAGS: 00010282
> >>   RAX: ffffea00000e4580 RBX: 0000000000052202 RCX: ffffea00000e4580
> >>   RDX: 0000000080000001 RSI: 0000000000052202 RDI: ffffea00000e4580
> >>   RBP: ffff88efa5d3d860 R08: 0000000000000000 R09: 0000000000000002
> >>   R10: 0000000000000008 R11: ffff89403fff7000 R12: ffff88f589165818
> >>   R13: 00007f1320600000 R14: ffffea0181296ca8 R15: ffffea00000e4580
> >>   FS:  00007f1324f93e00(0000) GS:ffff893ebfb80000(0000) knlGS:00000000=
00000000
> >>   CS:  0010 DS: 0000 ES: 0000 CR0: 0000000080050033
> >>   CR2: 00007f1321694070 CR3: 0000006046014004 CR4: 00000000007726e0
> >>   DR0: 0000000000000000 DR1: 0000000000000000 DR2: 0000000000000000
> >>   DR3: 0000000000000000 DR6: 00000000fffe0ff0 DR7: 0000000000000400
> >>   PKRU: 55555554
> >>   Call Trace:
> >>    follow_page_pte+0x64b/0x800
> >>    __get_user_pages+0x228/0x560
> >>    __gup_longterm_locked+0xa0/0x2f0
> >>    vaddr_get_pfns+0x67/0x100 [vfio_iommu_type1]
> >>    vfio_pin_pages_remote+0x30b/0x460 [vfio_iommu_type1]
> >>    vfio_pin_map_dma+0xd4/0x2e0 [vfio_iommu_type1]
> >>    vfio_dma_do_map+0x21e/0x340 [vfio_iommu_type1]
> >>    vfio_iommu_type1_ioctl+0xdd/0x170 [vfio_iommu_type1]
> >>    ? __fget_files+0x79/0xb0
> >>    ksys_ioctl+0x7b/0xb0
> >>    ? ksys_write+0xc4/0xe0
> >>    __x64_sys_ioctl+0x16/0x20
> >>    do_syscall_64+0x2d/0x40
> >>    entry_SYSCALL_64_after_hwframe+0x44/0xa9
> >>
> >> After add dumppage, it shows that it is a PageReserved page(e.g. zero =
page),
> >> whoes refcount is just overflow:
> >>   page:00000000b0504535 refcount:-2147483647 mapcount:0 mapping:000000=
0000000000 index:0x0 pfn:0x3916
> >>   flags: 0xffffc000001002(referenced|reserved)
> >>   raw: 00ffffc000001002 ffffea00000e4588 ffffea00000e4588 000000000000=
0000
> >>   raw: 0000000000000000 0000000000000000 80000001ffffffff 000000000000=
0000
> >>
> >> gup will _pin_ a page which is PageReserved, however, put_pfn in vfio =
will
> >> skip unpin page which is PageReserved. So use pfn_valid in put_pfn
> >> instead of !is_invalid_reserved_pfn to unpin PageReserved page.
> >>
> >> Signed-off-by: Yisheng Xie<ethan.xys@linux.alibaba.com>
> >> ---
> >>   drivers/vfio/vfio_iommu_type1.c | 2 +-
> >>   1 file changed, 1 insertion(+), 1 deletion(-)
> >>
> >> diff --git a/drivers/vfio/vfio_iommu_type1.c b/drivers/vfio/vfio_iommu=
_type1.c
> >> index b2854d7939ce..12775bab27ee 100644
> >> --- a/drivers/vfio/vfio_iommu_type1.c
> >> +++ b/drivers/vfio/vfio_iommu_type1.c
> >> @@ -461,7 +461,7 @@ static bool is_invalid_reserved_pfn(unsigned long =
pfn)
> >>  =20
> >>   static int put_pfn(unsigned long pfn, int prot)
> >>   {
> >> -	if (!is_invalid_reserved_pfn(pfn)) {
> >> +	if (pfn_valid(pfn)) {
> >>   		struct page *page =3D pfn_to_page(pfn);
> >>  =20
> >>   		unpin_user_pages_dirty_lock(&page, 1, prot & IOMMU_WRITE); =20