Received-SPF: pass (google.com: domain of linux-crypto+bounces-3692-linux.lists.archive=gmail.com@vger.kernel.org designates 147.75.199.223 as permitted sender) client-ip=147.75.199.223;
From: "Tian, Kevin" <kevin.tian@intel.com>
To: Alex Williamson <alex.williamson@redhat.com>, "Zeng, Xin"
	<xin.zeng@intel.com>
CC: "herbert@gondor.apana.org.au" <herbert@gondor.apana.org.au>,
	"jgg@nvidia.com" <jgg@nvidia.com>, "yishaih@nvidia.com" <yishaih@nvidia.com>,
	"shameerali.kolothum.thodi@huawei.com"
	<shameerali.kolothum.thodi@huawei.com>, "linux-crypto@vger.kernel.org"
	<linux-crypto@vger.kernel.org>, "kvm@vger.kernel.org" <kvm@vger.kernel.org>,
	qat-linux <qat-linux@intel.com>, "Cao, Yahui" <yahui.cao@intel.com>
Subject: RE: [PATCH v6 1/1] vfio/qat: Add vfio_pci driver for Intel QAT SR-IOV
 VF devices
Thread-Topic: [PATCH v6 1/1] vfio/qat: Add vfio_pci driver for Intel QAT
 SR-IOV VF devices
Thread-Index: AQHakNVVVTUb/4qbx0yNW3Gu0hQhQbFupRgAgAA6STA=
Date: Fri, 19 Apr 2024 05:23:30 +0000
Message-ID: <BN9PR11MB52767D5F7FF5D6498C974B388C0D2@BN9PR11MB5276.namprd11.prod.outlook.com>
References: <20240417143141.1909824-1-xin.zeng@intel.com>
	<20240417143141.1909824-2-xin.zeng@intel.com>
 <20240418165434.1da52cf0.alex.williamson@redhat.com>
In-Reply-To: <20240418165434.1da52cf0.alex.williamson@redhat.com>
Accept-Language: en-US
Content-Language: en-US
x-ms-exchange-antispam-messagedata-chunkcount: 1
x-ms-exchange-antispam-messagedata-0: =?us-ascii?Q?6FDsQkb6Kh/C4JkU7dlFJC/6g1Q4Iha9yq0dJ6WJ7+hNuDoEuUwmFgxeS1Zd?=
 =?us-ascii?Q?ewbgomr06HE3cuhymu1xIN+kqVRFM3Nhfe2PbuDZKvOXp05CxBipkEjymNAC?=
 =?us-ascii?Q?dug8Fd5pJHXlIo3SnTdnqWuRvJ5N504nlQpAcoh4geAsLIkh4iWFnzYtq7BG?=
 =?us-ascii?Q?inD93hmLVdFCVXhXsSp/Z6AMV7PAYKcCPVWyhOVJzfTuFZvzcRdRzcRwX8u0?=
 =?us-ascii?Q?NrCBrlR30xDPtnw4WbR8nVeyVlEZ9msSdwBlqZaZIbgDmU8TEy5ACK0ZqQKh?=
 =?us-ascii?Q?X6tlgBahU+MFF9nq0Ey3Jzqam33BtrXSukUwXO+IxC/WEXKl/OztysGRPFtG?=
 =?us-ascii?Q?CDt1TnZovaUq88KnOhxDpSEBTcLHa6hV7BOY7C8NBvdjvgx5dYsP1WDEFGTb?=
 =?us-ascii?Q?C+KNvL/US8Zb6AoJgW5kwnWKMRaaRhBO15ZFj+O+s0zoUcRKKJHsF2Ou7urn?=
 =?us-ascii?Q?vy9+T8Zm0sZ4bgiX/0AaINABEmt7259gHVv3Y2KC6burD2WqwVs0UhbsAInq?=
 =?us-ascii?Q?gp5WALgSs2npbh2CUOUHjBfKSDAiBtsIWMt/UlapMK1KaBtZVvM8zTW1v+CH?=
 =?us-ascii?Q?iwRlpYfn35vxiKCWS/lAR3dhJFsVPkoxF5KIOqkI7trn7q6Ru3WIv515yi2X?=
 =?us-ascii?Q?Pl7wtLl6d2mvCCQKDXsgzZdSoe4Itq5VwVGlt8YNOUmpj1iuVRLmPIEZUy8a?=
 =?us-ascii?Q?g/aonw+JdU51wL0xb9g+953musNaw5bYoIggBryGQi479sCv77yoUCIIviu4?=
 =?us-ascii?Q?/y6MFEAJ1qWyVBYX0eCs+5LZ+3TVZtHm12SNJ+G1hR4dGg1Zm4MUrkFDRzqV?=
 =?us-ascii?Q?HwkQruhKF18IRn9/aDpYnO3DxPqX8XCo9CMkA3AYxf+AW3Ke5t52hdedsOPY?=
 =?us-ascii?Q?iz5+i8qt3DK/ZHt5VDgZD8ChWvQbZxpOHJIZzGzrdWgBznXheb6LzylKW56v?=
 =?us-ascii?Q?z6WDXRFf54zlPzHQ8L2pi/uAUIQr2Xmx+1NGOzfUnO5A1xdHLCw1dRTim/tS?=
 =?us-ascii?Q?5JKEtDHAwWw+yTRBslumEM6WnVrpvZGimDlH7Yf3QiNoO+c6d+tfzU3FHPgQ?=
 =?us-ascii?Q?4u/4hm1bd42a5L/FR1Xt9clTLp/hceE1wM6Gskbx9DCG9O3G6xyiLOIgbD6+?=
 =?us-ascii?Q?NA8E5XQewAJqTkbrZ3JdUjoNPNxX5+5EAm2RkzwejgleFqzWIjPI2aLqi+y0?=
 =?us-ascii?Q?cvJ22Lub3mu1/P9N2uk14ANPxNN5OZ+302hVG8/h2piheKQhznqJi96xvqrq?=
 =?us-ascii?Q?n+F8+c7faIM/a87zz4TrSSafyUDJBbQr5bMhyEKoZGsg7U74jrHqbFdEE/+K?=
 =?us-ascii?Q?i761xA7WBnxqJ9xa8IJhn7cH+ehBroyXGnHLc7Q8K9p4n0jAZSEBQvavLEVS?=
 =?us-ascii?Q?ftafJqd/jWBpQu4KFx2fzZ5iCExLqkuOzKcasgzR3AU9lhWVZixjqB60gIdR?=
 =?us-ascii?Q?JYuaGf/YJUzYUBhXBrpoEaml45yNM2FgmLa7v+Xk7kOz5GTC34ulg2WnBicZ?=
 =?us-ascii?Q?iPuYiO3W0YttphDlNWLE0ujIPbB7y4dsKq0983thFSddRU4LhYc1AXjbISbA?=
 =?us-ascii?Q?IYsXqH0LkIjXmkv5TIZWYgNn/99tHHYEWP6ArC9R?=
Content-Type: text/plain; charset="us-ascii"
Content-Transfer-Encoding: quoted-printable
Precedence: bulk
X-Mailing-List: linux-crypto@vger.kernel.org
List-Id: <linux-crypto.vger.kernel.org>
List-Subscribe: <mailto:linux-crypto+subscribe@vger.kernel.org>
List-Unsubscribe: <mailto:linux-crypto+unsubscribe@vger.kernel.org>
MIME-Version: 1.0
X-MS-Exchange-CrossTenant-AuthAs: Internal
X-MS-Exchange-CrossTenant-AuthSource: BN9PR11MB5276.namprd11.prod.outlook.com
X-MS-Exchange-CrossTenant-Network-Message-Id: e5dc8324-4630-4f11-4608-08dc6030d9cb
X-MS-Exchange-CrossTenant-originalarrivaltime: 19 Apr 2024 05:23:30.8331
 (UTC)
X-MS-Exchange-CrossTenant-fromentityheader: Hosted
X-MS-Exchange-CrossTenant-id: 46c98d88-e344-4ed4-8496-4ed7712e255d
X-MS-Exchange-CrossTenant-mailboxtype: HOSTED
X-MS-Exchange-CrossTenant-userprincipalname: Hke1lFTgFfaiAUqPDAcCqSLVcKRAzfFbUQsHCQPNWkUHKcqgMBM9BqsKMxX3j6HsQ7vz+cnP49qak/vheFZfkQ==
X-MS-Exchange-Transport-CrossTenantHeadersStamped: MW4PR11MB6886
X-OriginatorOrg: intel.com

> From: Alex Williamson <alex.williamson@redhat.com>
> Sent: Friday, April 19, 2024 6:55 AM
>=20
> On Wed, 17 Apr 2024 22:31:41 +0800
> Xin Zeng <xin.zeng@intel.com> wrote:
>=20
> > +
> > +	/*
> > +	 * As the device is not capable of just stopping P2P DMAs, suspend
> the
> > +	 * device completely once any of the P2P states are reached.
> > +	 * On the opposite direction, resume the device after transiting from
> > +	 * the P2P state.
> > +	 */
> > +	if ((cur =3D=3D VFIO_DEVICE_STATE_RUNNING && new =3D=3D
> VFIO_DEVICE_STATE_RUNNING_P2P) ||
> > +	    (cur =3D=3D VFIO_DEVICE_STATE_PRE_COPY && new =3D=3D
> VFIO_DEVICE_STATE_PRE_COPY_P2P)) {
> > +		ret =3D qat_vfmig_suspend(qat_vdev->mdev);
> > +		if (ret)
> > +			return ERR_PTR(ret);
> > +		return NULL;
> > +	}
>=20
> This doesn't appear to be a valid way to support P2P, the P2P states
> are defined as running states.  The guest driver may legitimately
> access and modify the device state during P2P states.=20

yes it's a conceptual violation of the definition of the P2P states.

same issue also exists on an incoming P2P request from another=20
device, as once the device is fully stopped likely that P2P request
will be aborted leading to functional breakage.=20

A device in RUNNING_P2P is supposed to either complete the P2P
request as usual (if handling the request doesn't further initiate
outgoing P2P DMA) or pend the request (so later becomes part
of the device state when the device is fully stopped) to be
resumed on the target.

> Should this device be advertising support for P2P?

Jason suggests all new migration drivers must support P2P state.
In an old discussion [1] we thought what qat did is Okay if there
is no extra internal operation to be stopped in RUNNING_P2P->STOP.

But obviously we overlooked that by definition RUNNING_P2P is
a running state so could still see state changed from either CPU
or other devices.

With that I agree it sounds cleaner to stick to the fact i.e.=20
only advertising P2P support only if the device actually supports it.

Jason, I forgot the original intention why you suggest a must
support of P2P in migration drivers. Can you elaboreate?

[1] https://lore.kernel.org/intel-wired-lan/20231013140744.GT3952@nvidia.co=
m/