Date:   Thu, 16 Feb 2023 20:14:50 -0400
From:   Jason Gunthorpe <jgg@nvidia.com>
To:     Nanyong Sun <sunnanyong@huawei.com>
Cc:     joro@8bytes.org, will@kernel.org, robin.murphy@arm.com,
        mst@redhat.com, jasowang@redhat.com, iommu@lists.linux.dev,
        linux-kernel@vger.kernel.org, kvm@vger.kernel.org,
        virtualization@lists.linux-foundation.org, netdev@vger.kernel.org,
        wangrong68@huawei.com
Subject: Re: [PATCH v2] vhost/vdpa: Add MSI translation tables to iommu for
 software-managed MSI
Message-ID: <Y+7G+tiBCjKYnxcZ@nvidia.com>
References: <20230207120843.1580403-1-sunnanyong@huawei.com>
Content-Type: text/plain; charset=us-ascii
Content-Disposition: inline
In-Reply-To: <20230207120843.1580403-1-sunnanyong@huawei.com>
MIME-Version: 1.0
X-MS-Exchange-AntiSpam-MessageData-ChunkCount: 1
X-MS-Exchange-AntiSpam-MessageData-0: =?us-ascii?Q?rDV5F9gTiZ7GuMjDl4x2RLlaBUXRf42YYKabzH7ydek2NHsQ+b42VN7eHRbF?=
 =?us-ascii?Q?+PsMK60HiRFxksP92mdA1aV6j4kPfAojHfpSAOlHlkqQp/lgf2d8woCc/qFR?=
 =?us-ascii?Q?1TegDu4HgRy1cFMRpabhBSb1qWrIRgnxZGI81jMi/x+r8cKuI+thQVQDlPS2?=
 =?us-ascii?Q?cvsLVvplwpxTW6XkHa2nNu1iCaZ8AMgVlIEp+pATEmUKvEaLZHFsFvrpYWZN?=
 =?us-ascii?Q?2FO8j7x4iGn/xAXwXf4xgyb4ItouTqXQDzSBoRJsR8KdSnzXyzSsKLQsZTy/?=
 =?us-ascii?Q?wH4SYyXcmDd/eU8GIwWQO7muWdMHQGT/0A3NEH9SNMWo1+oXzls7jk8qXuRT?=
 =?us-ascii?Q?LyWo8T9inbBHFCTAqNnRmhov/mO5nNqDpp/XorB/WealiLU1bBot1QypUAds?=
 =?us-ascii?Q?pYFCnWI33Si1fw2A0Sg4VKx16dLIheS+2SpRfe+fVgnrn5VoeNph8BuJ9naH?=
 =?us-ascii?Q?UdglSlxwRBWRZyQVeYfTB8aLiF+dqxKMYrKKaIAxMXtlDa5/YwEztgtqeDkQ?=
 =?us-ascii?Q?SPV/NFKS17nGoyvZTos9KqLhAbRNq8/Zv5y8wk8/usjJppS5cFiBH6Gj/Rxa?=
 =?us-ascii?Q?sc8nVHtTajc+9tOMM/l8vAIZUq4mPFHyZDHmPpb5WYgQP5YpAULIDKlh8I80?=
 =?us-ascii?Q?FZoO+S2+VGbaDdfmAfgkynW1jRManxa3VkmB+y+VUPj7TbCTkuWW4vawQP+4?=
 =?us-ascii?Q?Yg01kaFIvYqsDdJEjaH5eyp1mq+/btftmFX+glXS0/DCbFFAFmF+x9cK462H?=
 =?us-ascii?Q?vOotlwmGrDgxkVOxqUAzGjuZL0LI5Cwi2/m97IDv6PvED2/4WukjW3cy4X6S?=
 =?us-ascii?Q?Q0lba/jQvmLstpJK8xg37M7ARTefd8cLroH6Wt0r5tdGmB/ddLF3lm2cl89e?=
 =?us-ascii?Q?Hs3Cqg9lZj9DxfOsQWR28qzKVlCiDs2hrP6vxbSLb/9kLC9RZAwUq6OG37K0?=
 =?us-ascii?Q?twwYTQ8LGXhzBbus72ucQw0P66jZat+D8wvNbBXxYR6X9A0dcNiQLSOXv3o2?=
 =?us-ascii?Q?/HxCgwq2QMjyHTRAK1m5uO/xtDSjCLS+tiIVblt6VNPtNQPoP3xwZJGPJA2B?=
 =?us-ascii?Q?3Kk2Kc31i0XkZIJ8xQFx6Ae1c6K13cIMlldonNRe3U0iB2BHg6T2CkTIeFVD?=
 =?us-ascii?Q?xFIeLQyHkLpSdvh1Fa7tZ4yxX8a3y3CKNOCnBrbZ4Lac2OZLhJvzGT9CwTqV?=
 =?us-ascii?Q?5ZGSM+KUptqXs+z9uP7VTr1qtl3yrc9smxDQK3kPWEIXe31caNjMEQoWNyAt?=
 =?us-ascii?Q?CVzgHMiv55A2U8iHFAvA2kUUKhOE3E8dh6EaBySqFE9RTEXiR3j6ZzbeRQwm?=
 =?us-ascii?Q?G+04hrNjN4T9GTXdo3Euv2tQ9XfDmSmbYrK8C3dOYnnhfYn4snn+uCkXi8Hf?=
 =?us-ascii?Q?xhtElbTkM98FVGmv/Nn7A0mH126pClrpC96YQ3ZkpNPONHJxkz/LU1rQiCJ1?=
 =?us-ascii?Q?ppy6bhiSc/DyPvUVTZVJIfZcq7bJDcvsmt6DKgAUQBrsQiTBAMMj15E6PPFg?=
 =?us-ascii?Q?bQr6UQoiLp3G+54HlCMXIPgpUm9yiqwpSoakmuaCL7eGb07UN+5Y2RXWIIou?=
 =?us-ascii?Q?IsiB+UG7K8RU1VTE6+zFsDnuCS58o4SB7WUNspL+?=
X-OriginatorOrg: Nvidia.com
X-MS-Exchange-CrossTenant-Network-Message-Id: 8b23dd7f-76fa-4e79-ebd7-08db107bfd54
X-MS-Exchange-CrossTenant-AuthSource: LV2PR12MB5869.namprd12.prod.outlook.com
X-MS-Exchange-CrossTenant-AuthAs: Internal
X-MS-Exchange-CrossTenant-OriginalArrivalTime: 17 Feb 2023 00:14:52.3713
 (UTC)
X-MS-Exchange-CrossTenant-FromEntityHeader: Hosted
X-MS-Exchange-CrossTenant-Id: 43083d15-7273-40c1-b7db-39efd9ccc17a
X-MS-Exchange-CrossTenant-MailboxType: HOSTED
X-MS-Exchange-CrossTenant-UserPrincipalName: M+y28M6ceGFmDTzy7b4M07sU57R1jFv5EXlTgVKzSZTANdd0oRVP/HaP9w3jkWdJ
X-MS-Exchange-Transport-CrossTenantHeadersStamped: IA1PR12MB6355
Precedence: bulk
List-ID: <linux-kernel.vger.kernel.org>
X-Mailing-List: linux-kernel@vger.kernel.org

On Tue, Feb 07, 2023 at 08:08:43PM +0800, Nanyong Sun wrote:
> From: Rong Wang <wangrong68@huawei.com>
> 
> Once enable iommu domain for one device, the MSI
> translation tables have to be there for software-managed MSI.
> Otherwise, platform with software-managed MSI without an
> irq bypass function, can not get a correct memory write event
> from pcie, will not get irqs.
> The solution is to obtain the MSI phy base address from
> iommu reserved region, and set it to iommu MSI cookie,
> then translation tables will be created while request irq.

Probably not what anyone wants to hear, but I would prefer we not add
more uses of this stuff. It looks like we have to get rid of
iommu_get_msi_cookie() :\

I'd like it if vdpa could move to iommufd not keep copying stuff from
it..

Also the iommu_group_has_isolated_msi() check is missing on the vdpa
path, and it is missing the iommu ownership mechanism.

Also which in-tree VDPA driver that uses the iommu runs on ARM? Please
don't propose core changes for unmerged drivers. :(

Jason