From:   "Tian, Kevin" <kevin.tian@intel.com>
To:     Jason Gunthorpe <jgg@nvidia.com>
CC:     Thomas Gleixner <tglx@linutronix.de>,
        "Jiang, Dave" <dave.jiang@intel.com>,
        Logan Gunthorpe <logang@deltatee.com>,
        LKML <linux-kernel@vger.kernel.org>,
        Bjorn Helgaas <helgaas@kernel.org>,
        "Marc Zygnier" <maz@kernel.org>,
        Alex Williamson <alex.williamson@redhat.com>,
        "Dey, Megha" <megha.dey@intel.com>,
        "Raj, Ashok" <ashok.raj@intel.com>,
        "linux-pci@vger.kernel.org" <linux-pci@vger.kernel.org>,
        Greg Kroah-Hartman <gregkh@linuxfoundation.org>,
        Jon Mason <jdmason@kudzu.us>, Allen Hubbe <allenbh@gmail.com>,
        "linux-ntb@googlegroups.com" <linux-ntb@googlegroups.com>,
        "linux-s390@vger.kernel.org" <linux-s390@vger.kernel.org>,
        Heiko Carstens <hca@linux.ibm.com>,
        "Christian Borntraeger" <borntraeger@de.ibm.com>,
        "x86@kernel.org" <x86@kernel.org>, Joerg Roedel <jroedel@suse.de>,
        "iommu@lists.linux-foundation.org" <iommu@lists.linux-foundation.org>
Subject: RE: [patch 21/32] NTB/msi: Convert to msi_on_each_desc()
Thread-Topic: [patch 21/32] NTB/msi: Convert to msi_on_each_desc()
Thread-Index: AQHX4y1VNaiO68omHEet2J9DWrVARKwa1YEAgAAp0QCAABreAIAAIi2AgAE8MoCAAAd1gIAACyoAgAAPcQCAADDDAIAAp1uAgABn9QCAACTxgIAAAdGAgAAbd4CAAA94gIAABpSAgAt9SoCAADlJAIAAgb+AgABGMACAAAc8gIAAE+oAgAAmHACAAFJ4oIAAem8AgAE/1RA=
Date:   Sat, 11 Dec 2021 07:52:59 +0000
Message-ID: <BN9PR11MB5276B2584F928B4BFD4573428C729@BN9PR11MB5276.namprd11.prod.outlook.com>
References: <f4cc305b-a329-6d27-9fca-b74ebc9fa0c1@intel.com>
 <878rx480fk.ffs@tglx>
 <BN9PR11MB52765F2EF8420C60FD5945D18C709@BN9PR11MB5276.namprd11.prod.outlook.com>
 <87sfv2yy19.ffs@tglx> <20211209162129.GS6385@nvidia.com>
 <878rwtzfh1.ffs@tglx> <20211209205835.GZ6385@nvidia.com>
 <8735n1zaz3.ffs@tglx> <87sfv1xq3b.ffs@tglx>
 <BN9PR11MB527619B099061B3814EB40408C719@BN9PR11MB5276.namprd11.prod.outlook.com>
 <20211210123938.GF6385@nvidia.com>
In-Reply-To: <20211210123938.GF6385@nvidia.com>
Accept-Language: en-US
Content-Language: en-US
x-ms-exchange-antispam-messagedata-chunkcount: 1
x-ms-exchange-antispam-messagedata-0: =?us-ascii?Q?f1eOO76T/evD/DN8ZkMUIqzFFqfVGVq9ths4MAAPgcXAddMcXCDHrovKfTJr?=
 =?us-ascii?Q?/rVJdoFV8Q+miurzd5hbhl4yFmRQSmFZbN6nG4qTQC9T+kfLuStEOgSHbuh5?=
 =?us-ascii?Q?oeG5IN38lizm4jcj/bnF41cYVNo3rf8iMJUorFydQttz9lMEOkqLJkyfUxFG?=
 =?us-ascii?Q?6hvQnudptOB+i2L/UulauClMxVndmdlksX1oQtlSurmD4MXLr1KIOUJsvSIm?=
 =?us-ascii?Q?OnQLYHxxebL5OQCAEZSvJXFwCBI5XYApn0OE3U9K51InAAD6T13aSg4N6z7D?=
 =?us-ascii?Q?0KohewMNN3Xk5K2cdMVHR84ZRr4ZgTuKuV+r2/pe4/+NUV2oINoTr1nO3Fy5?=
 =?us-ascii?Q?SniAWDxLVEtcWKjzell14UPRwECVRRlAa+uvmGFH1HgDS5TUQ9ZktDfmj2Dw?=
 =?us-ascii?Q?jtDlHizjdi1wMQ4mZlS8SNbdsT0lhz4uhnUnNAtxXXEoDQutQAjrKo5lhYwp?=
 =?us-ascii?Q?JMH2JAllVYSBZJFjGmnRQ3jMS06l+DHc18YyJ+TD1kzR4+mK5qckBL5FXKQ6?=
 =?us-ascii?Q?feDWkvdX+ztjcoYY8KKcEZDJq9/QLgfTeauBpq7i3U9b5pR4DY/Jl/ejsXkB?=
 =?us-ascii?Q?ihBKZSKr2AyfydkXz190EwRPFh/CXzNF1hUasaviv+ZQGupQ1Q+F5mWdBnIL?=
 =?us-ascii?Q?GLmrsQadA/Dm/eWFUFF0z83PJuk1ZsV6wOWdKrDQTz2VeovkGW9Iujvv61/U?=
 =?us-ascii?Q?W0Y4NZIZbfJ/7r+zSVDtCm4xEYLChFC2cHCKhhItSJDvKGTkyFYTwRFbQuzi?=
 =?us-ascii?Q?7SgMC5/blGWkISEGg4BAxZwMs2vUM2Dbr7F7xbEd+XDl814TEcK20XosIV6c?=
 =?us-ascii?Q?EPbsSGX4KgBJeY+4gohhJ5RUDCJN/Tyvap0/DbxiFaj0fbLpgLnQOhjxeXvE?=
 =?us-ascii?Q?JfSt0XLjLF8dRf5iOSsLpDD9P4k2y7II3OKUvmChUPUm/2/RimoCwHAgaGOG?=
 =?us-ascii?Q?L7XMGq0OiOPSdOW//E3/ro35MpMiPEEsfI7fi1n9lfQCtrvKkftBvVcLVRqN?=
 =?us-ascii?Q?HPpetiODQXEuQonqOkikyKpOnHerVM+qQ/A/zlSFZ7bQ8SKUNn0X+/f5kG/w?=
 =?us-ascii?Q?zTfsp2K34oPEjTIab5OSZQ28ytkBRxLdpt8HjITswot5ZSlN7nBx6NLN+Q1g?=
 =?us-ascii?Q?aW5Jb1chvJgZFikbEAH5S4edGzU3x0sCBoHYkjEN7W6IZQs8uvQyvGDjah/7?=
 =?us-ascii?Q?Ca69aZRSjLjkOHNqq8JKX5AiyUVjw44J0aEnxeMe+SllDN/RNKi8LnG+MTbH?=
 =?us-ascii?Q?VzYln/7bDb7heuCvZqeQqVSPnt7UKqwZRMuVORuuZxeBd/xiS5d7SVoVkxtT?=
 =?us-ascii?Q?zaIHCjmTZ4jkbeqVgUCMruMVl8b6/KZkK4m+Vdua9KbeurhuVEp7RAy45ZnH?=
 =?us-ascii?Q?YUr8W+/tYVXFc9V8ImSCVsVRgT+2OMsXj1EDmttbPMf+gcCnUOg8SxxsPMep?=
 =?us-ascii?Q?K0rLTss5gOyVLq/4yvpTFhD6B6XLr4JgpjkK4UDxTctYQUDRYayAallS4FvD?=
 =?us-ascii?Q?NmRSQlPfq7KYRZvUFJp9kH8vKt+8rwFBdWfdoWKxrquhcyrtUM/SH5jfWTDg?=
 =?us-ascii?Q?vjUw+POu7a+azAbigv06g4GgSZdVDwTlUD7vnQlTaRiiBIxHuC1VVyrtoBZq?=
 =?us-ascii?Q?Ag=3D=3D?=
Content-Type: text/plain; charset="us-ascii"
Content-Transfer-Encoding: quoted-printable
MIME-Version: 1.0
X-MS-Exchange-CrossTenant-AuthAs: Internal
X-MS-Exchange-CrossTenant-AuthSource: BN9PR11MB5276.namprd11.prod.outlook.com
X-MS-Exchange-CrossTenant-Network-Message-Id: f28959de-a94f-486d-b0b5-08d9bc7b4073
X-MS-Exchange-CrossTenant-originalarrivaltime: 11 Dec 2021 07:52:59.6266
 (UTC)
X-MS-Exchange-CrossTenant-fromentityheader: Hosted
X-MS-Exchange-CrossTenant-id: 46c98d88-e344-4ed4-8496-4ed7712e255d
X-MS-Exchange-CrossTenant-mailboxtype: HOSTED
X-MS-Exchange-CrossTenant-userprincipalname: TfkaBKHcM9RCCkfJf+mjfXy3FiKFR9h0uYeU8+xHigwv/zCMi38XXQmGvHGX4Aq0yjOmv7PSymYmwrprE/mJFQ==
X-MS-Exchange-Transport-CrossTenantHeadersStamped: BN6PR11MB3860
X-OriginatorOrg: intel.com
Precedence: bulk
List-ID: <linux-kernel.vger.kernel.org>
X-Mailing-List: linux-kernel@vger.kernel.org

> From: Jason Gunthorpe <jgg@nvidia.com>
> Sent: Friday, December 10, 2021 8:40 PM
>=20
>=20
> > Then Qemu needs to find out the GSI number for the vIRTE handle.
> > Again Qemu doesn't have such information since it doesn't know
> > which MSI[-X] entry points to this handle due to no trap.
>=20
> No this is already going wrong. qemu *cannot* know the MSI information
> because there is no MSI information for IMS.

I haven't thought of IMS at this step. The IR approach applies to
all types of interrupt storages, thus I'm more interested in how it
affect the storages which are already virtualized today (MSI[-X]=20
in my thought practice).

>=20
> All qemu should get is the origin device information and data about
> how the guest wants the interrupt setup.

yes, this is what I concluded in previous mail. The hypercall should
provide those information to the host and then get addr/data pair
back from the host.

>=20
> Forget about guests and all of this complexity, design how to make
> VFIO work with IMS in pure userspace like DPDK.
>=20
> We must have a VFIO ioctl to acquire a addr/data pair and link it to
> an event fd.

No question on it. Still need more thought down this road.

Thanks
Kevin