Received-SPF: pass (google.com: domain of linux-kernel-owner@vger.kernel.org designates 2620:137:e000::1:20 as permitted sender) client-ip=2620:137:e000::1:20;
From:   "Tian, Kevin" <kevin.tian@intel.com>
To:     Jason Gunthorpe <jgg@nvidia.com>
CC:     Thomas Gleixner <tglx@linutronix.de>,
        "Chatre, Reinette" <reinette.chatre@intel.com>,
        "Jiang, Dave" <dave.jiang@intel.com>,
        "Logan Gunthorpe" <logang@deltatee.com>,
        LKML <linux-kernel@vger.kernel.org>,
        "Bjorn Helgaas" <helgaas@kernel.org>,
        Marc Zygnier <maz@kernel.org>,
        Alex Williamson <alex.williamson@redhat.com>,
        "Dey, Megha" <megha.dey@intel.com>,
        "Raj, Ashok" <ashok.raj@intel.com>,
        "linux-pci@vger.kernel.org" <linux-pci@vger.kernel.org>,
        Greg Kroah-Hartman <gregkh@linuxfoundation.org>,
        Jon Mason <jdmason@kudzu.us>, Allen Hubbe <allenbh@gmail.com>,
        "linux-ntb@googlegroups.com" <linux-ntb@googlegroups.com>,
        "linux-s390@vger.kernel.org" <linux-s390@vger.kernel.org>,
        Heiko Carstens <hca@linux.ibm.com>,
        Christian Borntraeger <borntraeger@de.ibm.com>,
        "x86@kernel.org" <x86@kernel.org>, "Rodel, Jorg" <jroedel@suse.de>,
        "iommu@lists.linux-foundation.org" <iommu@lists.linux-foundation.org>
Subject: RE: [patch 21/32] NTB/msi: Convert to msi_on_each_desc()
Thread-Topic: [patch 21/32] NTB/msi: Convert to msi_on_each_desc()
Thread-Index: AQHX4y1VNaiO68omHEet2J9DWrVARKwa1YEAgAAp0QCAABreAIAAIi2AgAE8MoCAAAd1gIAACyoAgAAPcQCAADDDAIAAp1uAgABn9QCAACTxgIAAAdGAgAAbd4CAAA94gIAABpSAgAt9SoCAADlJAIAAgb+AgABGMACAAAc8gIAAE+oAgAAmHACAAFJ4oIAAem8AgABqWwCAANPscIAAWxoAgADV7aCAAT/pAIGycJswgAhPdQCAASQawIAAV3mAgAERgACAAHc2gIAAq/pA
Date:   Thu, 22 Sep 2022 22:42:46 +0000
Message-ID: <BN9PR11MB5276D6337528F356B44F18498C4E9@BN9PR11MB5276.namprd11.prod.outlook.com>
References: <87fsr0xp31.ffs@tglx>
 <BN9PR11MB527625E8A9BB854F3C0D19AE8C729@BN9PR11MB5276.namprd11.prod.outlook.com>
 <875yrvwavf.ffs@tglx>
 <BL1PR11MB5271326D39DAB692F07587768C739@BL1PR11MB5271.namprd11.prod.outlook.com>
 <87fsqxv8zf.ffs@tglx>
 <BN9PR11MB5276961D838169BF237928E18C499@BN9PR11MB5276.namprd11.prod.outlook.com>
 <YynJqID/E5dFCakg@nvidia.com>
 <BN9PR11MB5276CAB439EE27557FC17B1A8C4F9@BN9PR11MB5276.namprd11.prod.outlook.com>
 <YysIEUbxjS328TAX@nvidia.com>
 <BN9PR11MB52768936DF8C7F19D5997A0C8C4E9@BN9PR11MB5276.namprd11.prod.outlook.com>
 <YyxRf0UEEyT79oE5@nvidia.com>
In-Reply-To: <YyxRf0UEEyT79oE5@nvidia.com>
Accept-Language: en-US
Content-Language: en-US
x-ms-exchange-antispam-messagedata-chunkcount: 1
x-ms-exchange-antispam-messagedata-0: =?us-ascii?Q?rCOztX62mBk9EbTUucYFwYkIeoFATrOJ0dNh9lDB2e4R/CIYAmDBN+bjJijh?=
 =?us-ascii?Q?TAd3tN6kIv7PuIvqhtgZ58+GZ4dHe/ifMOY3S+qD38aWREbJgUUYoICJpjt4?=
 =?us-ascii?Q?XtwpMelQWQxv7JiygzjzqHIPfHJOof/Qya8b527Fdb0s5X/1GwEWuThwQEOk?=
 =?us-ascii?Q?nWnKh/v3by1GmIz/J3QR+fCvHkwNNbf9C/OSmiGg0Y8lLaqbIoXb6UtlpcHk?=
 =?us-ascii?Q?15YDiCu3Hhkqk2tKKx9uRVZS4YBBYTv3Dhfl5qE0TKfYlLIekqvdD12rpUjy?=
 =?us-ascii?Q?M6pJ3HyAKaGEsFWUPQ2dnTi03TTsVaGNenxcF+wJ2DbRF1/O0DgqxRLYB8Uw?=
 =?us-ascii?Q?0pj4Jfv1m8mnDZzYL9uc+qNIRtO/S/qm3tmBkq4ORW26eq7imosrqBrGIeoF?=
 =?us-ascii?Q?C1mTxN2LHV/oopQaA6KRTvKfjmsxjXvDDrUVM04TpG5FQqdep26x65gUc4Fs?=
 =?us-ascii?Q?wZkr9PIcA7faLa3i1pIVTEyHGE7qGNQCtAWdHBSaL2OdkJsKmLPqSNJjl6Bd?=
 =?us-ascii?Q?2koAbJWUijyU5R75xpdngevVDP6BSJDk2eAWTZ370S4PV6rIy3ba+0HNM8eJ?=
 =?us-ascii?Q?k1KytfODKXGqCpW91zzkcbTZ2eMMO+/Dyu72OJP84TykzPftL9DCmFrYyiw6?=
 =?us-ascii?Q?0g57ewGRiilfxQC+TLEamWgADj0YQ+Lhp0/PYW5CEISkY4gkEeRp6EdLWPO3?=
 =?us-ascii?Q?AXHpiIsbXCpt2upTGN6fc40wy0hFMl+lsm5dZgK1Y9+r7nTnUR/fxj/9Ae1c?=
 =?us-ascii?Q?/zDlUG5fRnaIRwhG4NIWJMsdHd8+hXnXgR9twTwFQF04LGbeNxe1CgTqXUqH?=
 =?us-ascii?Q?FE3+4TT6eYzk+lmDVUbSiCD/CQjqWPnjwsAQwl91w9cSHz5lvp6xbAEQzzEw?=
 =?us-ascii?Q?/6Z6cgA1ntRq1fO1oyWFpcX1tAFATC92/J4j1zgYkEU0ehmGOf9+xa6hrub3?=
 =?us-ascii?Q?Sf16+i4VpTy605EiMRRTXIbJ7pX/wryr2dNYIdkn47bKvHRPguJnsjxwAhvH?=
 =?us-ascii?Q?2cTusgTujjEBuees+lqJ2ccMpVUSjK9xMAJ8HkS4E4BgKKdplNgBiOtQmbVY?=
 =?us-ascii?Q?3qR2IHLRLmuNAY1de/cL7ei4mXD1+U6TXAJDZ64rS97HKbdcAiI75+TGU+Ng?=
 =?us-ascii?Q?wvfLL/Sf/qXkjWjvv2v5udaOoxGSp1AUAYJZDWWx7NXrRQUVbklw6mRMaTRL?=
 =?us-ascii?Q?buqA8tIS5Tj1VoeoR0X1IADyqjBMi/SAfksIQUuHt+M6ysm6L3zRSMwfBSv0?=
 =?us-ascii?Q?lt8l7L+J1Zmb0EaxxwUhiFsnbJest59BIZGG7SxtHCxf9M7s0f/izzr4n+UG?=
 =?us-ascii?Q?LIvJn+b3Xv1f8dsAFQGCJnZveAV8EBbPKvoZXqmbyMgIV/aHm4kAH5ujfojK?=
 =?us-ascii?Q?SpmwT0jSWbx0EpZtxlrcoAZvmLguIjg1nzG48LBMTGeimilpHOY+Uz8srWEa?=
 =?us-ascii?Q?YWyB03AqwPFkLddvPW2r8u1I5MJnTCDgdlIVUEKS0Clc/7Eozf+OfH5/PSti?=
 =?us-ascii?Q?Jd2nMAml7y20DUNJfWKC+4dPqaMbjFAuUeS48OLUK48DshiUGF/WLLDh+IQI?=
 =?us-ascii?Q?OVS3NVxbSGe4wo/F/50lHQwdu7uwq6JE7eUOccNz?=
Content-Type: text/plain; charset="us-ascii"
Content-Transfer-Encoding: quoted-printable
MIME-Version: 1.0
X-MS-Exchange-CrossTenant-AuthAs: Internal
X-MS-Exchange-CrossTenant-AuthSource: BN9PR11MB5276.namprd11.prod.outlook.com
X-MS-Exchange-CrossTenant-Network-Message-Id: 8e1a1122-f4b8-44c9-1eb7-08da9cebc4e1
X-MS-Exchange-CrossTenant-originalarrivaltime: 22 Sep 2022 22:42:46.0369
 (UTC)
X-MS-Exchange-CrossTenant-fromentityheader: Hosted
X-MS-Exchange-CrossTenant-id: 46c98d88-e344-4ed4-8496-4ed7712e255d
X-MS-Exchange-CrossTenant-mailboxtype: HOSTED
X-MS-Exchange-CrossTenant-userprincipalname: 6lH6d9dJwSORAIhUv54Q2slC2PG2e9JFGTG0G81Pi5FWzAwsfuaUV/fFnV/mgT3zvq3JYatHEwVo6UdcptlSpQ==
X-MS-Exchange-Transport-CrossTenantHeadersStamped: SJ0PR11MB4941
X-OriginatorOrg: intel.com
X-Spam-Status: No, score=-7.1 required=5.0 tests=BAYES_00,DKIMWL_WL_HIGH,
        DKIM_SIGNED,DKIM_VALID,DKIM_VALID_AU,DKIM_VALID_EF,RCVD_IN_DNSWL_HI,
        SPF_HELO_NONE,SPF_NONE autolearn=ham autolearn_force=no version=3.4.6
X-Spam-Checker-Version: SpamAssassin 3.4.6 (2021-04-09) on
        lindbergh.monkeyblade.net
Precedence: bulk
List-ID: <linux-kernel.vger.kernel.org>
X-Mailing-List: linux-kernel@vger.kernel.org

> From: Jason Gunthorpe <jgg@nvidia.com>
> Sent: Thursday, September 22, 2022 8:14 PM
>=20
> On Thu, Sep 22, 2022 at 05:11:00AM +0000, Tian, Kevin wrote:
>=20
> > > Thinking of the interrupt routing as a host resource is the problem -
> > > it is a device resource and just like PASID the ideal design would
> > > have each RID have its own stable numberspace scoped within it. The
> > > entire RID and all its stable numberspace would be copied over.
> > >
> >
> > Unfortunately it is not the case at least on Intel VT-d. The interrupt
> > remapping table is per VT-d instead of per RID.
>=20
> Doesn't that just turn it into a SW problem how it manages it?
>=20

When IRTEs are shared cross RID's I'm not sure how to make the
allocation stable between the src and the dest. They are allocated
dynamically and hierarchically in request_irq().

I'm not sure what SW mechanism can cleanly ensure a stable
allocation scheme between two machines.

Can you elaborate?