Received-SPF: pass (google.com: domain of linux-crypto+bounces-2217-linux.lists.archive=gmail.com@vger.kernel.org designates 2604:1380:45d1:ec00::1 as permitted sender) client-ip=2604:1380:45d1:ec00::1;
Date: Wed, 21 Feb 2024 09:18:56 -0400
From: Jason Gunthorpe <jgg@nvidia.com>
To: "Zeng, Xin" <xin.zeng@intel.com>
Cc: Yishai Hadas <yishaih@nvidia.com>,
	"herbert@gondor.apana.org.au" <herbert@gondor.apana.org.au>,
	"alex.williamson@redhat.com" <alex.williamson@redhat.com>,
	"shameerali.kolothum.thodi@huawei.com" <shameerali.kolothum.thodi@huawei.com>,
	"Tian, Kevin" <kevin.tian@intel.com>,
	"linux-crypto@vger.kernel.org" <linux-crypto@vger.kernel.org>,
	"kvm@vger.kernel.org" <kvm@vger.kernel.org>,
	qat-linux <qat-linux@intel.com>, "Cao, Yahui" <yahui.cao@intel.com>
Subject: Re: [PATCH 10/10] vfio/qat: Add vfio_pci driver for Intel QAT VF
 devices
Message-ID: <20240221131856.GS13330@nvidia.com>
References: <20240201153337.4033490-11-xin.zeng@intel.com>
 <20240206125500.GC10476@nvidia.com>
 <DM4PR11MB550222F7A5454DF9DBEE7FEC884B2@DM4PR11MB5502.namprd11.prod.outlook.com>
 <20240209121045.GP10476@nvidia.com>
 <e740d9ec-6783-4777-b984-98262566974c@nvidia.com>
 <DM4PR11MB550274B713F6AE416CDF7FDB88532@DM4PR11MB5502.namprd11.prod.outlook.com>
 <20240220132459.GM13330@nvidia.com>
 <DM4PR11MB5502BE3CC8BD098584F31E8D88502@DM4PR11MB5502.namprd11.prod.outlook.com>
 <20240220170315.GO13330@nvidia.com>
 <DM4PR11MB550223E2A68FA6A95970873888572@DM4PR11MB5502.namprd11.prod.outlook.com>
Content-Type: text/plain; charset=us-ascii
Content-Disposition: inline
In-Reply-To: <DM4PR11MB550223E2A68FA6A95970873888572@DM4PR11MB5502.namprd11.prod.outlook.com>
Precedence: bulk
MIME-Version: 1.0
X-MS-Exchange-AntiSpam-MessageData-ChunkCount: 1
X-MS-Exchange-AntiSpam-MessageData-0:
	=?us-ascii?Q?JLBGBC1fG8jU9BeaX0vuWxgwyaGCw4/C14JfTPog29AQOiVnP/VyeoSSCLEa?=
 =?us-ascii?Q?xBXZ2Q+pyxtSRG6AMSKKpWUVwZ0tvih2qoHncu0Ps+Y9WycGAz6HG2xbm/JI?=
 =?us-ascii?Q?hhJcmbIKmduJvm1KBl4LloBackftJDvrMGXX1T8QxhZIgGymAyh9IHl2DDOd?=
 =?us-ascii?Q?CB9DCqNUPAw9uRtjwSgmb9Ta7C0BXV+CEoAQY4FkiR+pzm+dI1/V4BmZQ8N0?=
 =?us-ascii?Q?zSghKhLjCplCV0NeeD/Pp0RbbbXlSLclJEatTxpkycnOOFLwB9Iyzuli8xwS?=
 =?us-ascii?Q?hNp0kKTPqVcdIJDzAFce2HECIKrw/Z0LNvmoZd5V4BliE4b7m7LQLHRWHXk9?=
 =?us-ascii?Q?qyrvVrO/AY7/0hO/SXr9ggb/6dW9GaogktEA46uc4RVeMmEc+oZxDNOh2Ogk?=
 =?us-ascii?Q?2wCDLfdjHNJQozCc/g08YUzbLXsvZwJ6/hnEWJmewwoit47++9C2x/v5425B?=
 =?us-ascii?Q?hHj0Ie3HMCGgEJ/+sG4SVPuHZXKbYwhrMX9xzxuN4RZmMjinySJ3GnHY7YhO?=
 =?us-ascii?Q?3MpNgy8JtJT7OZz/IGPoqLwqGzYR4J4k5OiOaK7i7upaROJQdEwlWlizMo5C?=
 =?us-ascii?Q?XXwQdNAV8dGNJ+yTXf57zfOHigNEHLAdHxGg24DJ5T/UcPif2CsYNK3ltnYT?=
 =?us-ascii?Q?jOCfF+XOwRCK+tN87ZkCYmLNuTGT5RTJLRaiD3YW73WXQmFoGOL26Adaxn9D?=
 =?us-ascii?Q?TkRTIQLROmZQiDgAUCDJUgd7INoAYTH66n4AowZtgCSvd3BVT8W9XSpGXCuL?=
 =?us-ascii?Q?Aq+rNseEuFQX6H5xciXooQUQUXMxksnitvhr9VJ4rZpw7fSRbvdkDlTIcLd3?=
 =?us-ascii?Q?sYZ+CBmWSBCcCJnsDCjip8P0DAz50WwSYRWR293E1pIvwM5M+1I3nHcgLopK?=
 =?us-ascii?Q?4ErsMnMddK3txTsZsXGlEvRkx+agHZQzmS1JvY1IXwUanU5s3UXqQS/C0IjK?=
 =?us-ascii?Q?5mEkGa/tLnOwpDXV8ZdpKVjm9/zgZgs2feaEKGycZjlwbw15fAAcbDzeKN3F?=
 =?us-ascii?Q?gXUNsFFaiKXqAQgqjxciCydTf0+zLk2V7TNKNs4sdLk+QOFhnciqJM7Z3D9+?=
 =?us-ascii?Q?srmeDhpFecEp4txWYuxU3cSi8zflYeVrLXoAwTNlWUx6cJA8WIGDgjpTF/MP?=
 =?us-ascii?Q?/jdSB2WXpX5byssNUgGatQQQ1vzYx9eLTIMKIHvXxEjc2qFKGdbdfjMBiYrh?=
 =?us-ascii?Q?pGA5o38RpCkSXJPdQHkr/Nk32ZDTykGRlQCnvSYj0S4axVWK6OBUVtOFhaz9?=
 =?us-ascii?Q?6LyJl6He67pJa26eynSK+HE4fA1pG/i/pUYAawy2vMiyJkJRFOIL+QvMOfHv?=
 =?us-ascii?Q?prNveib3rKvKDB8N8AC9Exy0BxTPQ0Xcr/cEKCs98pqjzbhs13mS4Au1RjEC?=
 =?us-ascii?Q?s30fxSdxhnzaYyCAf+xv/aOW4FU2utbKOGWy0r+BcWSfcfHtgXb2tyDSH5Zi?=
 =?us-ascii?Q?L8cJVsncC0Yy5mSVvS/i+9B5bvj3I6bH2LCwfTKUtrFqfmwQp1/YkLzBouNo?=
 =?us-ascii?Q?uT/amoT2p5O+TDvq3seQ8QOqodCw7UNY3ZrhssWJ8iA9GzPmWrNfG/6SLUFC?=
 =?us-ascii?Q?bmnJacwWi+Oxjl6vMdRF3lYbdZnrWUSmyiiHZaoh?=
X-OriginatorOrg: Nvidia.com
X-MS-Exchange-CrossTenant-Network-Message-Id: 701bf1aa-9b09-44e0-2d7c-08dc32dfa897
X-MS-Exchange-CrossTenant-AuthSource: LV2PR12MB5869.namprd12.prod.outlook.com
X-MS-Exchange-CrossTenant-AuthAs: Internal
X-MS-Exchange-CrossTenant-OriginalArrivalTime: 21 Feb 2024 13:18:56.8239
 (UTC)
X-MS-Exchange-CrossTenant-FromEntityHeader: Hosted
X-MS-Exchange-CrossTenant-Id: 43083d15-7273-40c1-b7db-39efd9ccc17a
X-MS-Exchange-CrossTenant-MailboxType: HOSTED
X-MS-Exchange-CrossTenant-UserPrincipalName: 7vlSFWeicXATSEchL9ey7HHxmbial3AzuUAUCklCCclIt2sLhTlPXVBAxl3rcBd2
X-MS-Exchange-Transport-CrossTenantHeadersStamped: LV8PR12MB9205

On Wed, Feb 21, 2024 at 08:44:31AM +0000, Zeng, Xin wrote:
> On Wednesday, February 21, 2024 1:03 AM, Jason Gunthorpe wrote:
> > On Tue, Feb 20, 2024 at 03:53:08PM +0000, Zeng, Xin wrote:
> > > On Tuesday, February 20, 2024 9:25 PM, Jason Gunthorpe wrote:
> > > > To: Zeng, Xin <xin.zeng@intel.com>
> > > > Cc: Yishai Hadas <yishaih@nvidia.com>; herbert@gondor.apana.org.au;
> > > > alex.williamson@redhat.com; shameerali.kolothum.thodi@huawei.com;
> > Tian,
> > > > Kevin <kevin.tian@intel.com>; linux-crypto@vger.kernel.org;
> > > > kvm@vger.kernel.org; qat-linux <qat-linux@intel.com>; Cao, Yahui
> > > > <yahui.cao@intel.com>
> > > > Subject: Re: [PATCH 10/10] vfio/qat: Add vfio_pci driver for Intel QAT VF
> > devices
> > > >
> > > > On Sat, Feb 17, 2024 at 04:20:20PM +0000, Zeng, Xin wrote:
> > > >
> > > > > Thanks for this information, but this flow is not clear to me why it
> > > > > cause deadlock. From this flow, CPU0 is not waiting for any resource
> > > > > held by CPU1, so after CPU0 releases mmap_lock, CPU1 can continue
> > > > > to run. Am I missing something?
> > > >
> > > > At some point it was calling copy_to_user() under the state
> > > > mutex. These days it doesn't.
> > > >
> > > > copy_to_user() would nest the mm_lock under the state mutex which is
> > a
> > > > locking inversion.
> > > >
> > > > So I wonder if we still have this problem now that the copy_to_user()
> > > > is not under the mutex?
> > >
> > > In protocol v2, we still have the scenario in precopy_ioctl where
> > copy_to_user is
> > > called under state_mutex.
> > 
> > Why? Does mlx5 do that? It looked Ok to me:
> > 
> >         mlx5vf_state_mutex_unlock(mvdev);
> >         if (copy_to_user((void __user *)arg, &info, minsz))
> >                 return -EFAULT;
> 
> Indeed, thanks, Jason. BTW, is there any reason why was "deferred_reset" mode
> still implemented in mlx5 driver given this deadlock condition has been avoided
> with migration protocol v2 implementation.

I do not remember. Yishai? 

Jason