Received-SPF: pass (google.com: domain of linux-kernel-owner@vger.kernel.org designates 2620:137:e000::1:20 as permitted sender) client-ip=2620:137:e000::1:20;
Date:   Mon, 16 Jan 2023 11:03:42 -0400
From:   Jason Gunthorpe <jgg@nvidia.com>
To:     Matthew Rosato <mjrosato@linux.ibm.com>
Cc:     alex.williamson@redhat.com, pbonzini@redhat.com, cohuck@redhat.com,
        farman@linux.ibm.com, pmorel@linux.ibm.com,
        borntraeger@linux.ibm.com, frankja@linux.ibm.com,
        imbrenda@linux.ibm.com, david@redhat.com, akrowiak@linux.ibm.com,
        jjherne@linux.ibm.com, pasic@linux.ibm.com,
        zhenyuw@linux.intel.com, zhi.a.wang@intel.com, seanjc@google.com,
        linux-s390@vger.kernel.org, kvm@vger.kernel.org,
        intel-gvt-dev@lists.freedesktop.org,
        intel-gfx@lists.freedesktop.org, linux-kernel@vger.kernel.org
Subject: Re: [PATCH v4] vfio: fix potential deadlock on vfio group lock
Message-ID: <Y8VnTu2cNfoYmbv4@nvidia.com>
References: <20230114000351.115444-1-mjrosato@linux.ibm.com>
Content-Type: text/plain; charset=us-ascii
Content-Disposition: inline
In-Reply-To: <20230114000351.115444-1-mjrosato@linux.ibm.com>
MIME-Version: 1.0
X-MS-Exchange-AntiSpam-MessageData-ChunkCount: 1
X-MS-Exchange-AntiSpam-MessageData-0: =?us-ascii?Q?IWkBziMeN8byZG1pSF7lPuTyu767LEnzc0/jjSMcC2dvis76mWL3/BmKdRl4?=
 =?us-ascii?Q?2cYIYgdT9sCvuvvDeMW6eSyMV47QLjn0ftFb2hJiugGINtIlFGdhUbPRHh1N?=
 =?us-ascii?Q?8v3kdII+V8UJymlHsmvgl/qhGcxDuVthEAKS2FTyns2HeuEo256dyLcujjYM?=
 =?us-ascii?Q?WRxvbYQyLlG2XYEoCQs4rC1OV18dL2Yd/ay0lVgcqX6RVv81KGeEEM3um8i+?=
 =?us-ascii?Q?7B2DKFCWZop4nFHY8iXa+XctUyKFEp1PKYtDwYJNa8QqyoCEXAHPXL3AqYkn?=
 =?us-ascii?Q?H2sFGJ/hTbIBdsiaTKqFcQX/12SeVP95eIXbMmNm7qoyrDUQNsLG07rAKc5/?=
 =?us-ascii?Q?+xrMPesokzlZbWsdXV79/GKxeEEySM3JQUryIsv46UY6ILTzdtcXRJ1KKXTX?=
 =?us-ascii?Q?Uq1bHuUKNZ47GCUu1FPYmuci0634hIZKdzgeJ5uIiO/k//5J9HPESR46V1OM?=
 =?us-ascii?Q?YLMsJKdsL7aA8XhGB/V016g3fyMT9sxyMSl36q7nlTl5m8vIoKe6VMe5MsEh?=
 =?us-ascii?Q?aHIWhLORWgqvSuJgH+/nbZWxjihBi6z60DA5VZCxtzuuTf8XV0qC0vsJxQa3?=
 =?us-ascii?Q?hfZQXJZSvLCroeV/q4M5JMLEwkgoFYJ2y2E10cc4473mZTilnPsb4W3GDhRr?=
 =?us-ascii?Q?vd0+8h7vRMCVTj1pDswv2Q1xKEY6OmxTc/Lhgi/kk/n8sBQXdYXHmCFIWn6I?=
 =?us-ascii?Q?QHsi2QolezqlNPLYnKstxOy872/OBfmilITfgAfnL8Nyt5sUG9J3PYQYTYNF?=
 =?us-ascii?Q?YWRsa5pDVRl3lbRubSFra37QdZ9ppIZFazw0MJjb1PvjpYDhuBSM01AAlQWx?=
 =?us-ascii?Q?tURULoBJKn5sKB4D+Bn5p5dxFJYQJLckEeteR7nl7qOKShF2lrB0CoCSDDLr?=
 =?us-ascii?Q?GpOKPcjQ8JDnwhAjw2iA4DRnm3VOk/P0r8CTc0aJvTbhSb+wgmaYjuazn7y7?=
 =?us-ascii?Q?h5cu1lV3vMWvi251uQOvh7PGwHR9HUiaGHqhCVHLjgzVErvbsao9U8WOv4Hk?=
 =?us-ascii?Q?ls4UXTBVqfGSPDijJaKmS6HPkk/xX32O7osFOmHgFXfjIMQo9X0aJWtfksbX?=
 =?us-ascii?Q?KDQMYgje0TGHSsDXLEcepGs+PPw+iywtJ/Ks1vg4R5YJC+B3Ys3/nxZrolRg?=
 =?us-ascii?Q?/vLtDL2Hobmrn9F8Q9dHKZ1G0x3CL36sxltWMZU/Vb7oyvHBKZd0gLQVG8Vc?=
 =?us-ascii?Q?+vNfhvbWUhiBlpWuutAmfaKwuVd2gL0FnY9k0S+6taWNIqd7tcuoWEtJy4SI?=
 =?us-ascii?Q?iWOOZEsSxJcNE+yz5ibbBqe6/93KIE+9elacAC7X5bIzdTDVcRBMqkeGJWIA?=
 =?us-ascii?Q?VacZ11RXcjdw4vs4wIiL9Fd6l+25+h93utPdEx04cKsYbyNbbzAMcAs9psSl?=
 =?us-ascii?Q?pTYrD3zEUUsVY/kDcbE0711pxh0RA5x5PEg1v1jymKzivW14mIRVKh6gIDfC?=
 =?us-ascii?Q?LSKRDDgtosUZzTfS8CjAU54QM63nIXAg6BU0xvae+L0dD7szoDBoxhiBntsi?=
 =?us-ascii?Q?I1WNZ8lR3B+E6Nitpaf4BO+/YFRRiRH8eS3npGr69nO5S5b0U2R3jsEdNmes?=
 =?us-ascii?Q?lH0r3qrirn6DA3liULc+4NNGyM00/03mXIXE7geb?=
X-OriginatorOrg: Nvidia.com
X-MS-Exchange-CrossTenant-Network-Message-Id: 5fd8a3d6-e982-4974-6cfd-08daf7d2dc49
X-MS-Exchange-CrossTenant-AuthSource: LV2PR12MB5869.namprd12.prod.outlook.com
X-MS-Exchange-CrossTenant-AuthAs: Internal
X-MS-Exchange-CrossTenant-OriginalArrivalTime: 16 Jan 2023 15:03:43.9525
 (UTC)
X-MS-Exchange-CrossTenant-FromEntityHeader: Hosted
X-MS-Exchange-CrossTenant-Id: 43083d15-7273-40c1-b7db-39efd9ccc17a
X-MS-Exchange-CrossTenant-MailboxType: HOSTED
X-MS-Exchange-CrossTenant-UserPrincipalName: Ch14lyJN3wPHgumvuSAzxoXPv7FO9UT1ZqXIX9hCC3VuOpYQ1IPybnEsP06U3IRp
X-MS-Exchange-Transport-CrossTenantHeadersStamped: IA0PR12MB8325
X-Spam-Status: No, score=-1.1 required=5.0 tests=BAYES_00,DKIMWL_WL_HIGH,
        DKIM_SIGNED,DKIM_VALID,DKIM_VALID_AU,DKIM_VALID_EF,FORGED_SPF_HELO,
        RCVD_IN_DNSWL_NONE,RCVD_IN_MSPIKE_H2,SPF_HELO_PASS,SPF_NONE
        autolearn=no autolearn_force=no version=3.4.6
X-Spam-Checker-Version: SpamAssassin 3.4.6 (2021-04-09) on
        lindbergh.monkeyblade.net
Precedence: bulk
List-ID: <linux-kernel.vger.kernel.org>
X-Mailing-List: linux-kernel@vger.kernel.org

On Fri, Jan 13, 2023 at 07:03:51PM -0500, Matthew Rosato wrote:
> Currently it is possible that the final put of a KVM reference comes from
> vfio during its device close operation.  This occurs while the vfio group
> lock is held; however, if the vfio device is still in the kvm device list,
> then the following call chain could result in a deadlock:
> 
> kvm_put_kvm
>  -> kvm_destroy_vm
>   -> kvm_destroy_devices
>    -> kvm_vfio_destroy
>     -> kvm_vfio_file_set_kvm
>      -> vfio_file_set_kvm
>       -> group->group_lock/group_rwsem
> 
> Avoid this scenario by having vfio core code acquire a KVM reference
> the first time a device is opened and hold that reference until right
> after the group lock is released after the last device is closed.
> 
> Fixes: 421cfe6596f6 ("vfio: remove VFIO_GROUP_NOTIFY_SET_KVM")
> Reported-by: Alex Williamson <alex.williamson@redhat.com>
> Signed-off-by: Matthew Rosato <mjrosato@linux.ibm.com>
> ---

Reviewed-by: Jason Gunthorpe <jgg@nvidia.com>

Jason