From:   "Tian, Kevin" <kevin.tian@intel.com>
To:     Thomas Gleixner <tglx@linutronix.de>,
        Jason Gunthorpe <jgg@nvidia.com>
CC:     "Jiang, Dave" <dave.jiang@intel.com>,
        Logan Gunthorpe <logang@deltatee.com>,
        LKML <linux-kernel@vger.kernel.org>,
        Bjorn Helgaas <helgaas@kernel.org>,
        Marc Zygnier <maz@kernel.org>,
        Alex Williamson <alex.williamson@redhat.com>,
        "Dey, Megha" <megha.dey@intel.com>,
        "Raj, Ashok" <ashok.raj@intel.com>,
        "linux-pci@vger.kernel.org" <linux-pci@vger.kernel.org>,
        Greg Kroah-Hartman <gregkh@linuxfoundation.org>,
        Jon Mason <jdmason@kudzu.us>, Allen Hubbe <allenbh@gmail.com>,
        "linux-ntb@googlegroups.com" <linux-ntb@googlegroups.com>,
        "linux-s390@vger.kernel.org" <linux-s390@vger.kernel.org>,
        Heiko Carstens <hca@linux.ibm.com>,
        Christian Borntraeger <borntraeger@de.ibm.com>,
        "x86@kernel.org" <x86@kernel.org>, "Rodel, Jorg" <jroedel@suse.de>,
        "iommu@lists.linux-foundation.org" <iommu@lists.linux-foundation.org>
Subject: RE: [patch 21/32] NTB/msi: Convert to msi_on_each_desc()
Thread-Topic: [patch 21/32] NTB/msi: Convert to msi_on_each_desc()
Thread-Index: AQHX4y1VNaiO68omHEet2J9DWrVARKwa1YEAgAAp0QCAABreAIAAIi2AgAE8MoCAAAd1gIAACyoAgAAPcQCAADDDAIAAp1uAgABn9QCAACTxgIAAAdGAgAAbd4CAAA94gIAABpSAgAt9SoCAADlJAIAAgb+AgABGMACAAAc8gIAAE+oAgAAmHACAAFJ4oIAAem8AgABqWwCAANPscIAAWxoAgADV7aA=
Date:   Sun, 12 Dec 2021 01:56:31 +0000
Message-ID: <BL1PR11MB5271326D39DAB692F07587768C739@BL1PR11MB5271.namprd11.prod.outlook.com>
References: <f4cc305b-a329-6d27-9fca-b74ebc9fa0c1@intel.com>
 <878rx480fk.ffs@tglx>
 <BN9PR11MB52765F2EF8420C60FD5945D18C709@BN9PR11MB5276.namprd11.prod.outlook.com>
 <87sfv2yy19.ffs@tglx> <20211209162129.GS6385@nvidia.com>
 <878rwtzfh1.ffs@tglx> <20211209205835.GZ6385@nvidia.com>
 <8735n1zaz3.ffs@tglx> <87sfv1xq3b.ffs@tglx>
 <BN9PR11MB527619B099061B3814EB40408C719@BN9PR11MB5276.namprd11.prod.outlook.com>
 <20211210123938.GF6385@nvidia.com> <87fsr0xp31.ffs@tglx>
 <BN9PR11MB527625E8A9BB854F3C0D19AE8C729@BN9PR11MB5276.namprd11.prod.outlook.com>
 <875yrvwavf.ffs@tglx>
In-Reply-To: <875yrvwavf.ffs@tglx>
Accept-Language: en-US
Content-Language: en-US
x-ms-exchange-antispam-messagedata-chunkcount: 1
x-ms-exchange-antispam-messagedata-0: =?us-ascii?Q?DUMXxiXhM3FFRaKsJzHdBEXeSe4LS5TxQ9cAhVIhmKMVs2WF1z48TBZba1nG?=
 =?us-ascii?Q?oLhOp6HW6bRcC3RxPr9t5t/8Vfsm4AQ1ILN0uGe41WYTvet3qdk6SJYuS077?=
 =?us-ascii?Q?rP4pTUIWJJBIgQtMMIkWAM57WHByxMxuDJ7qIqpLWOAfTqqWJIur5pQr5Rh0?=
 =?us-ascii?Q?KF/PpdfvK4PPsmun0Hhoc4CqWhsxoOJnam8NS3Kn8egDqKP2U/yJBNub2ZfH?=
 =?us-ascii?Q?5fB+3NnusRBpGuRWJ8rTIPioDc8hr9mA/XVGD18FvL4HOch5IUG4RiUS93ON?=
 =?us-ascii?Q?NQuyXnEKRu//vpmI/ncQZA/jhzBupsAiCxjwKMncfIV3FXlZaZ3Nekp2HWAf?=
 =?us-ascii?Q?/qHYhkOr7Mi7QXeNb8RwHo8/Sa+IbDasNXrSi64UDD/Noin8ORlo4xwqiO6g?=
 =?us-ascii?Q?IGv3Q/9xcC/lScG4SURGbxSjjJ/ErOx4k/Plg9Zs9Fn89TfJ5y3Ui3SbJ1Bn?=
 =?us-ascii?Q?aiUpl3V0+N8Z/jPSYrijJ8POMjtjZwdtc3nG1L+Mm2x0pJv1D7c4tk3o01jI?=
 =?us-ascii?Q?GFQN+6MLp0FpYgakcKg6a+51hu/apN8VWlE9DG8NWKl3eEpXA+Uiq5PcziWB?=
 =?us-ascii?Q?nI0gGx9RJJBCgY2KAOQH1HAf/We3wrsBPoiujbfi+vVyx4aOl1/9Dgb6vw0l?=
 =?us-ascii?Q?dQcLIXwe2uHN6OHoSwspF2K+eOZ5nseoCy9ZpP7v9qrSHrK4S6oZZo2EDvKY?=
 =?us-ascii?Q?+z3HYnyTQ2jJpAbgL2q1ijsxTmXnCFibcAxvxHJvGKThZ5KCQ8l8jGY5GROI?=
 =?us-ascii?Q?cUB361Jus3pHWvhfqyh9mwtmu1MCva4k45FfL3llgOXspix2jd3M4BLe5ebZ?=
 =?us-ascii?Q?iSwM0vfMaSyF6lis9vdluNMWYFPyC3CaIfZdeQ7pEIgW1rwA11NsHasJQbCR?=
 =?us-ascii?Q?cWk0lz9MnCe+LChnIUPRyxvzFL8iZHEw0NErgmQe8bGnpwPW9suzO7ObkKv3?=
 =?us-ascii?Q?wfzxZgFc7AONHgsKBocdjepHjste4CxL5pVreRFteiavbCx6dSw+2HtvHyU/?=
 =?us-ascii?Q?DGKNwlTlhtMM3pYNrSrz1uEE2DsiHoNAXnLqAl4fYlqA9NXubb3gJ7trLMLY?=
 =?us-ascii?Q?MkRANAKvSNntt1XYpOl45hQU6/jQ/y+ViYTDjiA+nqy73FJony9DLiDsEai/?=
 =?us-ascii?Q?6vppnteZkH2U9EFswPYzFXbY6cH4pmyrKpihV51I53G5C03uRswnyj9Ysgbb?=
 =?us-ascii?Q?kqy2wIfxh9GluthMj8t2Meb+9giwlC5jGHM8KQO5nsf1er07gwN04oeR76kd?=
 =?us-ascii?Q?wG1VI1m39yzmrRri0W47pwDASpgkp8kjH+XOXWQWs5BqpnYZmrKUTRVF4X0c?=
 =?us-ascii?Q?YtE6jq7vGE4aZrCiewF/mPtOaOHA2ZwjgsMYuLveBUfioFgCFdkHjM5CrBR2?=
 =?us-ascii?Q?38//u7NTef2/GI6ryqdcmRXQ3eIK3Bew2RhhKTbwV/ymuIqYTG37+TYWacCb?=
 =?us-ascii?Q?rGjoFUzzrCQ68Nj8lopHshf1haHCVTqn4GP++7wrgYrCm3PTzSuA9kmqlxa1?=
 =?us-ascii?Q?Y5Xt1RkAFC4fz5ygs6R4K2gWHcVyQEgsAjj30Kay8oqmtzdMWAvys2tWcxQx?=
 =?us-ascii?Q?ptpKt5zy+KK60rzIqmNapTnuOENfAsmULWEz6G7A43cLCtnC+LB+1itYZHyd?=
 =?us-ascii?Q?ZXGsGM2sar9pn5C09HQyI4g=3D?=
Content-Type: text/plain; charset="us-ascii"
Content-Transfer-Encoding: quoted-printable
MIME-Version: 1.0
X-MS-Exchange-CrossTenant-AuthAs: Internal
X-MS-Exchange-CrossTenant-AuthSource: BL1PR11MB5271.namprd11.prod.outlook.com
X-MS-Exchange-CrossTenant-Network-Message-Id: 5382d6d1-7740-4aba-a189-08d9bd129e59
X-MS-Exchange-CrossTenant-originalarrivaltime: 12 Dec 2021 01:56:31.2265
 (UTC)
X-MS-Exchange-CrossTenant-fromentityheader: Hosted
X-MS-Exchange-CrossTenant-id: 46c98d88-e344-4ed4-8496-4ed7712e255d
X-MS-Exchange-CrossTenant-mailboxtype: HOSTED
X-MS-Exchange-CrossTenant-userprincipalname: HuIDCOJdFjarZFqQ8DWCUKqEi2Z/iX9oNs3EtMUjetsSeGkizPHBnXJReUIlzBRzYsaUzYfHs5FwA/ZUzI0rZw==
X-MS-Exchange-Transport-CrossTenantHeadersStamped: MN2PR11MB3871
X-OriginatorOrg: intel.com
Precedence: bulk
List-ID: <linux-kernel.vger.kernel.org>
X-Mailing-List: linux-kernel@vger.kernel.org

> From: Thomas Gleixner <tglx@linutronix.de>
> Sent: Saturday, December 11, 2021 9:05 PM
>=20
> Kevin,
>=20
> On Sat, Dec 11 2021 at 07:44, Kevin Tian wrote:
> >> From: Thomas Gleixner <tglx@linutronix.de>
> >> On Fri, Dec 10 2021 at 08:39, Jason Gunthorpe wrote:
> >> > It is clever, we don't have an vIOMMU that supplies vIR today, so by
> >> > definition all guests are excluded and only bare metal works.
> >>
> >> Dammit. Now you spilled the beans. :)
> >
> > Unfortunately we do have that today. Qemu supports IR for
> > both AMD and Intel vIOMMU.
>=20
> can you point me to the code?
>=20
> All I can find is drivers/iommu/virtio-iommu.c but I can't find anything
> vIR related there.
>=20

Well, virtio-iommu is a para-virtualized vIOMMU implementations.

In reality there are also fully emulated vIOMMU implementations (e.g.
Qemu fully emulates Intel/AMD/ARM IOMMUs). In those configurations
the IR logic in existing iommu drivers just apply:

	drivers/iommu/intel/irq_remapping.c
	drivers/iommu/amd/iommu.c
	...

As I replied in another mail, the 1st vIR implementation was introduced
to Qemu back to 2016:

commit 1121e0afdcfa0cd40e36bd3acff56a3fac4f70fd
Author: Peter Xu <peterx@redhat.com>
Date:   Thu Jul 14 13:56:13 2016 +0800

    x86-iommu: introduce "intremap" property

    Adding one property for intel-iommu devices to specify whether we shoul=
d
    support interrupt remapping. By default, IR is disabled. To enable it,
    we should use (take Intel IOMMU as example):

      -device intel_iommu,intremap=3Don

    This property can be shared by Intel and future AMD IOMMUs.

    Signed-off-by: Peter Xu <peterx@redhat.com>
    Reviewed-by: Michael S. Tsirkin <mst@redhat.com>
    Signed-off-by: Michael S. Tsirkin <mst@redhat.com>

Thanks
Kevin