Received-SPF: pass (google.com: domain of linux-kernel-owner@vger.kernel.org designates 23.128.96.18 as permitted sender) client-ip=23.128.96.18;
Subject: Re: [regression] Re: [PATCH v6 06/12] mm/gup: track FOLL_PIN pages
To:     Alex Williamson <alex.williamson@redhat.com>,
        LKML <linux-kernel@vger.kernel.org>
CC:     Andrew Morton <akpm@linux-foundation.org>,
        Al Viro <viro@zeniv.linux.org.uk>,
        Christoph Hellwig <hch@infradead.org>,
        "Dan Williams" <dan.j.williams@intel.com>,
        Dave Chinner <david@fromorbit.com>,
        "Ira Weiny" <ira.weiny@intel.com>, Jan Kara <jack@suse.cz>,
        Jason Gunthorpe <jgg@ziepe.ca>,
        Jonathan Corbet <corbet@lwn.net>,
        =?UTF-8?B?SsOpcsO0bWUgR2xpc3Nl?= <jglisse@redhat.com>,
        "Kirill A . Shutemov" <kirill@shutemov.name>,
        Michal Hocko <mhocko@suse.com>,
        Mike Kravetz <mike.kravetz@oracle.com>,
        Shuah Khan <shuah@kernel.org>,
        Vlastimil Babka <vbabka@suse.cz>,
        Matthew Wilcox <willy@infradead.org>,
        <linux-doc@vger.kernel.org>, <linux-fsdevel@vger.kernel.org>,
        <linux-kselftest@vger.kernel.org>, <linux-rdma@vger.kernel.org>,
        <linux-mm@kvack.org>,
        "Kirill A . Shutemov" <kirill.shutemov@linux.intel.com>
References: <20200211001536.1027652-1-jhubbard@nvidia.com>
 <20200211001536.1027652-7-jhubbard@nvidia.com>
 <20200424121846.5ee2685f@w520.home>
From:   John Hubbard <jhubbard@nvidia.com>
Message-ID: <5b901542-d949-8d7e-89c7-f8d5ee20f6e9@nvidia.com>
Date:   Fri, 24 Apr 2020 12:20:03 -0700
User-Agent: Mozilla/5.0 (X11; Linux x86_64; rv:68.0) Gecko/20100101
 Thunderbird/68.7.0
MIME-Version: 1.0
In-Reply-To: <20200424121846.5ee2685f@w520.home>
Content-Type: text/plain; charset="utf-8"; format=flowed
Content-Language: en-US
Content-Transfer-Encoding: 7bit
Sender: linux-kernel-owner@vger.kernel.org
Precedence: bulk

On 2020-04-24 11:18, Alex Williamson wrote:
...
> Hi John,
> 
> I'm seeing a regression bisected back to this commit (3faa52c03f44
> mm/gup: track FOLL_PIN pages).  I've attached some vfio-pci test code
> that reproduces this by mmap'ing a page of MMIO space of a device and
> then tries to map that through the IOMMU, so this should be attempting
> a gup/pin of a PFNMAP page.  Previously this failed gracefully (-EFAULT),
> but now results in:


Hi Alex,

Thanks for this report, and especially for source code to test it, 
seeing as how I can't immediately spot the problem just from the crash
data so far.  I'll get set up and attempt a repro.

Actually this looks like it should be relatively easier than the usual 
sort of "oops, we leaked a pin_user_pages() or unpin_user_pages() call,
good luck finding which one" report that I fear the most. :) This one 
looks more like a crash that happens directly, when calling into the 
pin_user_pages_remote() code. Which should be a lot easier to solve...

btw, if you are set up for it, it would be nice to know what source file 
and line number corresponds to the RIP (get_pfnblock_flags_mask+0x22) 
below. But if not, no problem, because I've likely got to do the repro 
in any case.

thanks,
-- 
John Hubbard
NVIDIA
> 
> BUG: unable to handle page fault for address: ffffae5cbfe5e938
> #PF: supervisor read access in kernel mode
> #PF: error_code(0x0000) - not-present page
> PGD 0 P4D 0
> Oops: 0000 [#1] SMP NOPTI
> CPU: 18 PID: 3365 Comm: vfio-pci-dma-ma Tainted: G           OE     5.6.0+ #6
> Hardware name: AMD Corporation Diesel/Diesel, BIOS TDL100CB 03/17/2020
> RIP: 0010:get_pfnblock_flags_mask+0x22/0x70
> Code: c3 0f 1f 80 00 00 00 00 0f 1f 44 00 00 48 8b 05 bc e1 d9 01 48 89 f7 49 89 c8 48 c1 ef 0f 48 85 c0 74 48 48 89 f1 48 c1 e9 17 <48> 8b 04 c8 48 85 c0 74 0b 40 0f b6 ff 48 c1 e7 04 48 01 f8 48 c1
> RSP: 0018:ffffb55289b3fcc8 EFLAGS: 00010216
> RAX: ffff9e5cbff50000 RBX: 0000000000000001 RCX: 000001fffffe1d27
> RDX: 0000000000000002 RSI: ffffff0e93acd633 RDI: 0001fffffe1d2759
> RBP: ffffb55289b3fd88 R08: 0000000000000007 R09: ffff9e48a52476a8
> R10: 0000000000000000 R11: 0000000000000000 R12: 0000000000000001
> R13: 0000000000000000 R14: 0000000000000001 R15: ffff9e48ab358cc0
> FS:  00007f4ef7269740(0000) GS:ffff9e48afa80000(0000) knlGS:0000000000000000
> CS:  0010 DS: 0000 ES: 0000 CR0: 0000000080050033
> CR2: ffffae5cbfe5e938 CR3: 0000000c61eda000 CR4: 00000000003406e0
> Call Trace:
>   __gup_longterm_locked+0x274/0x620
>   vaddr_get_pfn+0x74/0x110 [vfio_iommu_type1]
>   vfio_pin_pages_remote+0x6e/0x370 [vfio_iommu_type1]
>   vfio_iommu_type1_ioctl+0x8e5/0xaac [vfio_iommu_type1]
>   ksys_ioctl+0x86/0xc0
>   __x64_sys_ioctl+0x16/0x20
>   do_syscall_64+0x5b/0x1f0
>   entry_SYSCALL_64_after_hwframe+0x44/0xa9
> RIP: 0033:0x7f4ef6d7d307
> Code: 44 00 00 48 8b 05 69 1b 2d 00 64 c7 00 26 00 00 00 48 c7 c0 ff ff ff ff c3 66 2e 0f 1f 84 00 00 00 00 00 b8 10 00 00 00 0f 05 <48> 3d 01 f0 ff ff 73 01 c3 48 8b 0d 39 1b 2d 00 f7 d8 64 89 01 48
> RSP: 002b:00007fff76ada738 EFLAGS: 00000213 ORIG_RAX: 0000000000000010
> RAX: ffffffffffffffda RBX: 0000000000000000 RCX: 00007f4ef6d7d307
> RDX: 00007fff76ada760 RSI: 0000000000003b71 RDI: 0000000000000003
> RBP: 00007fff76ada930 R08: 0000000000000005 R09: 0000000000000000
> R10: 0000000000000001 R11: 0000000000000213 R12: 0000000000400950
> R13: 00007fff76adaa10 R14: 0000000000000000 R15: 0000000000000000
> Modules linked in: vfio_pci(OE) vfio_virqfd(OE) vfio_iommu_type1(OE) vfio(OE) amd64_edac_mod edac_mce_amd kvm_amd kvm rfkill sunrpc ipmi_ssif vfat irqbypass fat ipmi_si crct10dif_pclmul crc32_pclmul sp5100_tco ghash_clmulni_intel ipmi_devintf pcspkr joydev ccp i2c_piix4 k10temp ipmi_msghandler pinctrl_amd acpi_cpufreq ip_tables nouveau ast video mxm_wmi drm_vram_helper wmi drm_ttm_helper i2c_algo_bit drm_kms_helper cec ttm drm i40e e1000e crc32c_intel
> CR2: ffffae5cbfe5e938
> ---[ end trace a384ab7cc8e37d46 ]---
> RIP: 0010:get_pfnblock_flags_mask+0x22/0x70
> Code: c3 0f 1f 80 00 00 00 00 0f 1f 44 00 00 48 8b 05 bc e1 d9 01 48 89 f7 49 89 c8 48 c1 ef 0f 48 85 c0 74 48 48 89 f1 48 c1 e9 17 <48> 8b 04 c8 48 85 c0 74 0b 40 0f b6 ff 48 c1 e7 04 48 01 f8 48 c1
> RSP: 0018:ffffb55289b3fcc8 EFLAGS: 00010216
> RAX: ffff9e5cbff50000 RBX: 0000000000000001 RCX: 000001fffffe1d27
> RDX: 0000000000000002 RSI: ffffff0e93acd633 RDI: 0001fffffe1d2759
> RBP: ffffb55289b3fd88 R08: 0000000000000007 R09: ffff9e48a52476a8
> R10: 0000000000000000 R11: 0000000000000000 R12: 0000000000000001
> R13: 0000000000000000 R14: 0000000000000001 R15: ffff9e48ab358cc0
> FS:  00007f4ef7269740(0000) GS:ffff9e48afa80000(0000) knlGS:0000000000000000
> CS:  0010 DS: 0000 ES: 0000 CR0: 0000000080050033
> CR2: ffffae5cbfe5e938 CR3: 0000000c61eda000 CR4: 00000000003406e0
> 
> Thanks,
> Alex
>