Received-SPF: pass (google.com: domain of linux-crypto+bounces-2204-linux.lists.archive=gmail.com@vger.kernel.org designates 2604:1380:45d1:ec00::1 as permitted sender) client-ip=2604:1380:45d1:ec00::1;
Date: Tue, 20 Feb 2024 13:03:15 -0400
From: Jason Gunthorpe <jgg@nvidia.com>
To: "Zeng, Xin" <xin.zeng@intel.com>
Cc: Yishai Hadas <yishaih@nvidia.com>,
	"herbert@gondor.apana.org.au" <herbert@gondor.apana.org.au>,
	"alex.williamson@redhat.com" <alex.williamson@redhat.com>,
	"shameerali.kolothum.thodi@huawei.com" <shameerali.kolothum.thodi@huawei.com>,
	"Tian, Kevin" <kevin.tian@intel.com>,
	"linux-crypto@vger.kernel.org" <linux-crypto@vger.kernel.org>,
	"kvm@vger.kernel.org" <kvm@vger.kernel.org>,
	qat-linux <qat-linux@intel.com>, "Cao, Yahui" <yahui.cao@intel.com>
Subject: Re: [PATCH 10/10] vfio/qat: Add vfio_pci driver for Intel QAT VF
 devices
Message-ID: <20240220170315.GO13330@nvidia.com>
References: <20240201153337.4033490-1-xin.zeng@intel.com>
 <20240201153337.4033490-11-xin.zeng@intel.com>
 <20240206125500.GC10476@nvidia.com>
 <DM4PR11MB550222F7A5454DF9DBEE7FEC884B2@DM4PR11MB5502.namprd11.prod.outlook.com>
 <20240209121045.GP10476@nvidia.com>
 <e740d9ec-6783-4777-b984-98262566974c@nvidia.com>
 <DM4PR11MB550274B713F6AE416CDF7FDB88532@DM4PR11MB5502.namprd11.prod.outlook.com>
 <20240220132459.GM13330@nvidia.com>
 <DM4PR11MB5502BE3CC8BD098584F31E8D88502@DM4PR11MB5502.namprd11.prod.outlook.com>
Content-Type: text/plain; charset=us-ascii
Content-Disposition: inline
In-Reply-To: <DM4PR11MB5502BE3CC8BD098584F31E8D88502@DM4PR11MB5502.namprd11.prod.outlook.com>
Precedence: bulk
MIME-Version: 1.0
X-MS-Exchange-AntiSpam-MessageData-ChunkCount: 1
X-MS-Exchange-AntiSpam-MessageData-0:
	=?us-ascii?Q?ZZBB7jgT1JORmH19iJ+tQBTinmfmnfr5XScTJMgjmbd6r5aaOLJ/ghoTsKES?=
 =?us-ascii?Q?uA9SVnXwGXYw1usQPQjG7T0M+0YVwtkdkcQTrO533jRPC3Aizcl0DYevQppD?=
 =?us-ascii?Q?HfGafOg+YqR+b2h0lc/jI0/x1IKRQsIhcNWc29sFr31Vj0Ky32Ri6sHRfGNP?=
 =?us-ascii?Q?MdlW+Z+FoeaN3GEdHEPaaAD2ECvaAm+EwVRkbd8KdMZEMDDlDt8LgzIhqfLs?=
 =?us-ascii?Q?cNFtNk6YUsAUq07OUtJiqrcCGuASu6Lk6+afr2AWfdSypVHNZK4YUVg9xnFd?=
 =?us-ascii?Q?4figmZMyW+s9I7oHSRXcDS+n6yA4e+TJrUU0tPCEvXtY+rTXUnyjBI+4my/1?=
 =?us-ascii?Q?GquPPG+4G8m0fKkWjnGxz/C8wxXfh39roa0bktiOPxhBRobXjNQ3tZARzgmv?=
 =?us-ascii?Q?gLvJGz64NhBZb4WOU4efqEW6IQcKUknLi4v+l5wKl7ILERa73iVRixX6VFyS?=
 =?us-ascii?Q?fF/K5NRZulknKslm6g3BZlM0JpOh6lQTqPzKZjTkuQCky8MoGtR/hiqNbPuY?=
 =?us-ascii?Q?Aa3hdlTzBxDzizm6TZ/4XSt6SkLoANIzUegdGBeE9tlGDrsbSeB+A6YZcdSn?=
 =?us-ascii?Q?uBdjzdbts6PUFjAzHInaO5Yc2G5VibKwcVCF0AHz6lRLpydXzAwJgxeb0o1Q?=
 =?us-ascii?Q?xmfnKl0/vgiHoDxD5uzY3lpbt6pAM8qh/hf4BH1IVO7Ke3y9OLZNpqeDnBWM?=
 =?us-ascii?Q?z9y/mCNXuc1MshuMu+OGggmyvFr86NTp7JRAtNGiy7eThA4ra2f1Q9pIEYh+?=
 =?us-ascii?Q?A+WT/2f36A9F80nKvfhP3UB/VFyfMU5ZRO2r1Vl/RyW8cd4WFXrFigFix6vL?=
 =?us-ascii?Q?OKx1HfsYMuWgdg9Pl4R0tALM3LOhVyFDFAAAgHxXtxayH2hz9f9V1V5/Avh0?=
 =?us-ascii?Q?VLwaY0X1HWeg1DSqVn0p4Hcfoh5BV5ofnZAQvSUijtcjjFOC7xcHOzOf9Qe1?=
 =?us-ascii?Q?5vQf6TZpP6zweN4H8QrGOefDppNDzyICyua49d+TAPyXglMxUPHzqLHOwl7w?=
 =?us-ascii?Q?iFJ+HxiuFreBDwxbwWgFYP3KB1lbXVtDaIeRtPiUkfdDDDhWrqb8aiWheqUW?=
 =?us-ascii?Q?/ZO/8mynGjn6/3Vd/Y0hvvylXnqPAtCc9FOJh/E4nmlax+6tKjHSzcBUrqGp?=
 =?us-ascii?Q?/KDKXbxdUozjPATf8M2QXsOMt7sTHBrqq/eUz4C/85wSqz0FShUKvBDk9iNP?=
 =?us-ascii?Q?57CZ4AGpT0FkksRdxldDWs1tmgztR7CQeGar14aE5D50UO5230Hel0XcTwyC?=
 =?us-ascii?Q?/z45HIQEbpNfKWMevK5VLeRnoAMqHBK/xjs/ZQKAGuES1X6/Iy5jNnC5/ZLw?=
 =?us-ascii?Q?TmVS001KKQqXKWqCpKE3660asrijKcQ4Vm4pncHsjtJH3Wwzx/pHrKI0n0/7?=
 =?us-ascii?Q?kWMJPnjdjahAdb2IRZF3AuNBlLG3xShRIorJpFU8KjyL75bRZNgtJwDfvwlo?=
 =?us-ascii?Q?On2cbHReIRk3XRjI711QQjVW02YNyQLHtTbvpoxQxxFDgB9G2KPzRl43L2qA?=
 =?us-ascii?Q?O1g3gW66BL2KH8bTDmLDKZTH0owHmmHf0l1jDuV4aQPV6iamR2LqHTWiY2w+?=
 =?us-ascii?Q?GgIaqUxxWrPKhFm3qWRgqCxyPh3VfIL3a1vuWBlo?=
X-OriginatorOrg: Nvidia.com
X-MS-Exchange-CrossTenant-Network-Message-Id: 69c4d28f-dc78-4e13-ce6b-08dc3235d4c2
X-MS-Exchange-CrossTenant-AuthSource: LV2PR12MB5869.namprd12.prod.outlook.com
X-MS-Exchange-CrossTenant-AuthAs: Internal
X-MS-Exchange-CrossTenant-OriginalArrivalTime: 20 Feb 2024 17:03:16.6965
 (UTC)
X-MS-Exchange-CrossTenant-FromEntityHeader: Hosted
X-MS-Exchange-CrossTenant-Id: 43083d15-7273-40c1-b7db-39efd9ccc17a
X-MS-Exchange-CrossTenant-MailboxType: HOSTED
X-MS-Exchange-CrossTenant-UserPrincipalName: sMyhgDbH3RDQnY4Kf0nDWSSP5zbwuacoU63M2+OBEtWYO78HbkCqR7I6xVIO2aYl
X-MS-Exchange-Transport-CrossTenantHeadersStamped: DM4PR12MB8521

On Tue, Feb 20, 2024 at 03:53:08PM +0000, Zeng, Xin wrote:
> On Tuesday, February 20, 2024 9:25 PM, Jason Gunthorpe wrote:
> > To: Zeng, Xin <xin.zeng@intel.com>
> > Cc: Yishai Hadas <yishaih@nvidia.com>; herbert@gondor.apana.org.au;
> > alex.williamson@redhat.com; shameerali.kolothum.thodi@huawei.com; Tian,
> > Kevin <kevin.tian@intel.com>; linux-crypto@vger.kernel.org;
> > kvm@vger.kernel.org; qat-linux <qat-linux@intel.com>; Cao, Yahui
> > <yahui.cao@intel.com>
> > Subject: Re: [PATCH 10/10] vfio/qat: Add vfio_pci driver for Intel QAT VF devices
> > 
> > On Sat, Feb 17, 2024 at 04:20:20PM +0000, Zeng, Xin wrote:
> > 
> > > Thanks for this information, but this flow is not clear to me why it
> > > cause deadlock. From this flow, CPU0 is not waiting for any resource
> > > held by CPU1, so after CPU0 releases mmap_lock, CPU1 can continue
> > > to run. Am I missing something?
> > 
> > At some point it was calling copy_to_user() under the state
> > mutex. These days it doesn't.
> > 
> > copy_to_user() would nest the mm_lock under the state mutex which is a
> > locking inversion.
> > 
> > So I wonder if we still have this problem now that the copy_to_user()
> > is not under the mutex?
> 
> In protocol v2, we still have the scenario in precopy_ioctl where copy_to_user is
> called under state_mutex.

Why? Does mlx5 do that? It looked Ok to me:

        mlx5vf_state_mutex_unlock(mvdev);
        if (copy_to_user((void __user *)arg, &info, minsz))
                return -EFAULT;

Jason