Received-SPF: pass (google.com: domain of linux-kernel-owner@vger.kernel.org designates 2620:137:e000::1:20 as permitted sender) client-ip=2620:137:e000::1:20;
Date:   Mon, 8 May 2023 13:48:30 -0300
From:   Jason Gunthorpe <jgg@nvidia.com>
To:     Yan Zhao <yan.y.zhao@intel.com>
Cc:     kvm@vger.kernel.org, linux-kernel@vger.kernel.org,
        alex.williamson@redhat.com, kevin.tian@intel.com,
        yishaih@nvidia.com, shameerali.kolothum.thodi@huawei.com
Subject: Re: [PATCH] vfio/pci: take mmap write lock for io_remap_pfn_range
Message-ID: <ZFkn3q45RUJXMS+P@nvidia.com>
References: <20230508125842.28193-1-yan.y.zhao@intel.com>
Content-Type: text/plain; charset=us-ascii
Content-Disposition: inline
In-Reply-To: <20230508125842.28193-1-yan.y.zhao@intel.com>
MIME-Version: 1.0
X-MS-Exchange-AntiSpam-MessageData-ChunkCount: 1
X-MS-Exchange-AntiSpam-MessageData-0: =?us-ascii?Q?pdQ/h1l2CUn8tbI0zlrVSkJjDd831yl5xWHhgsCW1GkqXyuPdJb7qbA09V9f?=
 =?us-ascii?Q?cewAitGtjiaUPndLnV4/oDig0EwfAos5gOKneS4cJz03KifynGyHFY0dsZ5v?=
 =?us-ascii?Q?so3RASLRNhkeNBIYz3Epw96d6PDOk4f0WNfBFt48E4qrlduBIPBg66T/GOvh?=
 =?us-ascii?Q?jyeALV5xMAg3hfxmgHrRZjld4tGEu2ZQTJ9evcZQzlwq/PZsRW+HmS0fYsz7?=
 =?us-ascii?Q?jqcBQg1n6ricmfaM77RwDq2xfBDrXlm4F9M9QJ7WKVf4jgO8/JbDZEsxnJhb?=
 =?us-ascii?Q?s49W3hDsws3/PJ457xZDBP59jfgxzL3iuCJkTH/bUpJefD/lFF5e/MaaIRVW?=
 =?us-ascii?Q?xKw61sFuKc1LrByDOiVr2nXFhzOfSaak/YaVSpQfmkSFuvBJHku0VdmFxc3t?=
 =?us-ascii?Q?M2HBeadjZQRgOS1ZTjTNxE/kjM2NpsMerdCDF+pnj/RmZaGI4oE6Q1nYByXs?=
 =?us-ascii?Q?QOxfRYDeah+ljbtAUDiYyTkeFcTfQNEd/e9bZElMss/MopD6R7E/IRSut0C9?=
 =?us-ascii?Q?uICDVXEaGV1h9zfIY9imdU0QlIinkskGXRTs6yDCxfPtk2da+kjmuAexn9xC?=
 =?us-ascii?Q?Gx+zw86vtbfZFkWbpzoZbaDDFmaAupqt7/ibf/HxedT7CUrqn4+A3FJPRQ9q?=
 =?us-ascii?Q?oddIZX0MI1SF1F9qqZeMAnWebGt6hS8xq4ADQoNUtRKy6amB8PFRqCKRpjnU?=
 =?us-ascii?Q?9TJqJwJUf+l04safet0mYE/SjPvmksmXeOSdSoVE7kG3EZc3DFwrShcN9CEk?=
 =?us-ascii?Q?BMz0/KBVL9PWScZoutotz+HtTHKibtcW7WTbU5Y9LNE1uQ+Q5Er2Md2+oFlV?=
 =?us-ascii?Q?6ZydBy1eiAow7CLEBE8DOPBMhmvuGikFTKQ5dSw5++UD2aEqm5/1OY1hV7PF?=
 =?us-ascii?Q?pDwpSkfWribg3WzT3RMf3wnoXJMNz6lsk8Sq0m/4C1/uLfftfmCYilICvUI/?=
 =?us-ascii?Q?pMdnglTR/7yYpzwOmDJ4y5B2xXiBrIHeEfSt84tEK3qqCvdNND/ysiITSTj4?=
 =?us-ascii?Q?wH5GcRG7C8w5iSDR7wX6/JgX6JHF4uj+AKhhJPP4uH3L0jAIG8g4l5sBpPWM?=
 =?us-ascii?Q?k8MJdtmreUDm4R2rSHzCZdqkHntfG5AtV9Le3hdo96kvUS6oIKomNQXObmpd?=
 =?us-ascii?Q?fotOiljpI9hAD2fwOfu08DEp+HSpCd1UNeRgDw/rMCgKAUBohi5fAChHGsZp?=
 =?us-ascii?Q?gvvSv0lZGA8j3DPODK4U3ikScZ3azN7PitA/vJ+2e5e5BKqcUCXQx9lSMv0Y?=
 =?us-ascii?Q?aec706p+WBG4Yb5tOerXiyhcciTWNOgk5cCX4RnucTAtRLhyAIxpTJ0HFTRQ?=
 =?us-ascii?Q?rOEhvP8+x2qOHf5ZyYMJvQB2SmDxoME5Io9nqSQ+OLuuVKMO98q7q4CMUC/X?=
 =?us-ascii?Q?YhIEV7NME0d/kce22iOxjtw78XFO2Rf6K1Hkh4JV50GWL+H07EBYhX7noukj?=
 =?us-ascii?Q?3LnMv7Yhsemy2WKJNGLls87S6gO3CyCLzmHlQfAOGpthufFQVMMo5SbWE2D/?=
 =?us-ascii?Q?ZpKw0NCRB5O0Puo34Kh8g5+JOgd83yhfqRZDupHpik1W/Pte15t5TIvqz1aL?=
 =?us-ascii?Q?vOPibgSuEnof4bJx0HI=3D?=
X-OriginatorOrg: Nvidia.com
X-MS-Exchange-CrossTenant-Network-Message-Id: 89a6653f-5746-4842-f14d-08db4fe40f27
X-MS-Exchange-CrossTenant-AuthSource: LV2PR12MB5869.namprd12.prod.outlook.com
X-MS-Exchange-CrossTenant-AuthAs: Internal
X-MS-Exchange-CrossTenant-OriginalArrivalTime: 08 May 2023 16:48:33.0246
 (UTC)
X-MS-Exchange-CrossTenant-FromEntityHeader: Hosted
X-MS-Exchange-CrossTenant-Id: 43083d15-7273-40c1-b7db-39efd9ccc17a
X-MS-Exchange-CrossTenant-MailboxType: HOSTED
X-MS-Exchange-CrossTenant-UserPrincipalName: XTrjR0TKDL6iUNl0wly0O+xRZnywA+9c6Z9XMQbhBSFYFsDUdJY3n226rK5QlNTg
X-MS-Exchange-Transport-CrossTenantHeadersStamped: IA1PR12MB6163
X-Spam-Status: No, score=-1.1 required=5.0 tests=BAYES_00,DKIMWL_WL_HIGH,
        DKIM_SIGNED,DKIM_VALID,DKIM_VALID_AU,DKIM_VALID_EF,FORGED_SPF_HELO,
        RCVD_IN_DNSWL_NONE,RCVD_IN_MSPIKE_H2,SPF_HELO_PASS,SPF_NONE,
        T_SCC_BODY_TEXT_LINE autolearn=no autolearn_force=no version=3.4.6
X-Spam-Checker-Version: SpamAssassin 3.4.6 (2021-04-09) on
        lindbergh.monkeyblade.net
Precedence: bulk
List-ID: <linux-kernel.vger.kernel.org>
X-Mailing-List: linux-kernel@vger.kernel.org

On Mon, May 08, 2023 at 08:58:42PM +0800, Yan Zhao wrote:
> In VFIO type1, vaddr_get_pfns() will try fault in MMIO PFNs after
> pin_user_pages_remote() returns -EFAULT.
> 
> follow_fault_pfn
>  fixup_user_fault
>   handle_mm_fault
>    handle_mm_fault
>     do_fault
>      do_shared_fault
>       do_fault
>        __do_fault
>         vfio_pci_mmap_fault
>          io_remap_pfn_range
>           remap_pfn_range
>            track_pfn_remap
>             vm_flags_set         ==> mmap_assert_write_locked(vma->vm_mm)
>            remap_pfn_range_notrack
>             vm_flags_set         ==> mmap_assert_write_locked(vma->vm_mm)
> 
> As io_remap_pfn_range() will call vm_flags_set() to update vm_flags [1],
> holding of mmap write lock is required.
> So, update vfio_pci_mmap_fault() to drop mmap read lock and take mmap
> write lock.
> 
> [1] https://lkml.kernel.org/r/20230126193752.297968-3-surenb@google.com
> commit bc292ab00f6c ("mm: introduce vma->vm_flags wrapper functions")
> commit 1c71222e5f23
> ("mm: replace vma->vm_flags direct modifications with modifier calls")
> 
> Signed-off-by: Yan Zhao <yan.y.zhao@intel.com>
> ---
>  drivers/vfio/pci/vfio_pci_core.c | 17 +++++++++++++++++
>  1 file changed, 17 insertions(+)
> 
> diff --git a/drivers/vfio/pci/vfio_pci_core.c b/drivers/vfio/pci/vfio_pci_core.c
> index a5ab416cf476..5082f89152b3 100644
> --- a/drivers/vfio/pci/vfio_pci_core.c
> +++ b/drivers/vfio/pci/vfio_pci_core.c
> @@ -1687,6 +1687,12 @@ static vm_fault_t vfio_pci_mmap_fault(struct vm_fault *vmf)
>  	struct vfio_pci_mmap_vma *mmap_vma;
>  	vm_fault_t ret = VM_FAULT_NOPAGE;
>  
> +	mmap_assert_locked(vma->vm_mm);
> +	mmap_read_unlock(vma->vm_mm);
> +
> +	if (mmap_write_lock_killable(vma->vm_mm))
> +		return VM_FAULT_RETRY;

Certainly not..

I'm not sure how to resolve this properly, set the flags in advance?

The address space conversion?

Jason