Received-SPF: pass (google.com: domain of linux-kernel-owner@vger.kernel.org designates 23.128.96.18 as permitted sender) client-ip=23.128.96.18;
Subject: Re: vfio-pci: protect remap_pfn_range() from simultaneous calls
To:     Bharat Bhushan <bbhushan2@marvell.com>,
        "alex.williamson@redhat.com" <alex.williamson@redhat.com>
Cc:     "linux-kernel@vger.kernel.org" <linux-kernel@vger.kernel.org>,
        Sunil Kovvuri Goutham <sgoutham@marvell.com>
References: <MWHPR18MB152034C6BAE8CC6D503DDEBEE3D09@MWHPR18MB1520.namprd18.prod.outlook.com>
From:   Ankur Arora <ankur.a.arora@oracle.com>
Message-ID: <f33979f5-0e89-4fa2-c48e-06d353379f23@oracle.com>
Date:   Wed, 6 Jan 2021 10:13:58 -0800
User-Agent: Mozilla/5.0 (X11; Linux x86_64; rv:78.0) Gecko/20100101
 Thunderbird/78.4.0
MIME-Version: 1.0
In-Reply-To: <MWHPR18MB152034C6BAE8CC6D503DDEBEE3D09@MWHPR18MB1520.namprd18.prod.outlook.com>
Content-Type: text/plain; charset=utf-8; format=flowed
Content-Language: en-US
Content-Transfer-Encoding: 7bit
Precedence: bulk

On 2021-01-06 8:17 a.m., Bharat Bhushan wrote:
> Hi Ankur,
> 
> We are observing below BUG_ON() with latest kernel
> 
>     [10011.321645] ------------[ cut here ]------------
>     [10011.322262] kernel BUG at mm/memory.c:1816!
>     [10011.323793] Internal error: Oops - BUG: 0 [#1] PREEMPT SMP
>     [10011.326108] CPU: 2 PID: 1147 Comm: odp_l2fwd Not tainted 5.4.74-05938-gb9598e49fe61 #15
>     [10011.328272] Hardware name: Marvell CN106XX board (DT)
>     [10011.330328] pstate: 80400009 (Nzcv daif +PAN -UAO)
>     [10011.332402] pc : remap_pfn_range+0x1a4/0x260
>     [10011.334383] lr : remap_pfn_range+0x14c/0x260
>     [10011.335911] sp : ffff8000156afc10
>     [10011.337360] x29: ffff8000156afc10 x28: ffffffdffa240000
>     [10011.339671] x27: ffff00014a241000 x26: 0000002182000000
>     [10011.341984] x25: ffff0001489fbe00 x24: 0000002182040000
>     [10011.344279] x23: 0000002182040000 x22: 0068000000000fc3
>     [10011.346539] x21: 0000002182040000 x20: ffff000149d70860
>     [10011.348846] x19: 0000000000000041 x18: 0000000000000000
>     [10011.351064] x17: 0000000000000000 x16: 0000000000000000
>     [10011.353304] x15: 0000000000000000 x14: 0000000000000000
>     [10011.355519] x13: 0000000000000000 x12: 0000000000000000
>     [10011.357812] x11: 0000000000000000 x10: ffffffdfffe00000
>     [10011.360136] x9 : 0000000000000000 x8 : 0000000000000000
>     [10011.362414] x7 : 0000000000000000 x6 : 0000042182000000
>     [10011.364773] x5 : 0001000000000000 x4 : 0000000000000000
>     [10011.367103] x3 : ffffffe000328928 x2 : 016800017c240fc3
>     [10011.369462] x1 : 0000000000000000 x0 : ffffffe000328928
>     [10011.371694] Call trace:
>     [10011.373510]  remap_pfn_range+0x1a4/0x260
>     [10011.375386]  vfio_pci_mmap_fault+0x9c/0x114
>     [10011.377346]  __do_fault+0x38/0x100
>     [10011.379253]  __handle_mm_fault+0x81c/0xce4
>     [10011.381247]  handle_mm_fault+0xb4/0x17c
>     [10011.383220]  do_page_fault+0x110/0x430
>     [10011.385188]  do_translation_fault+0x80/0x90
>     [10011.387069]  do_mem_abort+0x3c/0xa0
>     [10011.388852]  el0_da+0x20/0x24
>     [10011.391239] Code: eb1a02ff 54000080 f9400362 b4fffe42 (d4210000)
>     [10011.393306] ---[ end trace ae8b75b32426d53c ]---
>     [10011.395140] note: odp_l2fwd[1147] exited with preempt_count 2
> 
> This is observed after patch "vfio-pci: Fault mmaps to enable vma tracking" where actual mapping delayed on page fault.
> When address of same page accessed by multiple threads at/around same time by threads running on different cores causes page fault for same page on multiple cores at same time. One of the fault hander creates mapping while second hander find that page-table mapping already exists and leads to above kernel BUG_ON().

Yeah, that's what my fix addressed as well.

> 
> While article  https://lwn.net/Articles/828536/ suggest that you have already faced and fixed this issue
>         "- vfio-pci: protect remap_pfn_range() from simultaneous calls (Ankur  Arora) [Orabug: 31663628] {CVE-2020-12888} {CVE-2020-12888}"
> 
> But I do not see any patch submitted or under review in upstream, hopefully I did not missed some discussion. Please let us know in case you already submitted or planning to submit fix or someone else fixed same.

No you haven't missed a discussion on this. For upstream this was more of
a theoretical race so I dallied a bit before sending the patch upstream.

I'll submit a patch soon. Also, would you mind if I ask you to run this
failing test before submission?

Thanks
Ankur

> 
> Thanks
> -Bharat
>