Received-SPF: softfail (google.com: domain of transitioning linux-kernel-owner@vger.kernel.org does not designate 23.128.96.19 as permitted sender) client-ip=23.128.96.19;
Date:   Wed, 6 Apr 2022 10:06:14 -0300
From:   Jason Gunthorpe <jgg@nvidia.com>
To:     Robin Murphy <robin.murphy@arm.com>
Cc:     "Tian, Kevin" <kevin.tian@intel.com>,
        Lu Baolu <baolu.lu@linux.intel.com>,
        Joerg Roedel <joro@8bytes.org>,
        Christoph Hellwig <hch@infradead.org>,
        "Raj, Ashok" <ashok.raj@intel.com>, Will Deacon <will@kernel.org>,
        Jean-Philippe Brucker <jean-philippe@linaro.com>,
        Eric Auger <eric.auger@redhat.com>,
        "Liu, Yi L" <yi.l.liu@intel.com>,
        "Pan, Jacob jun" <jacob.jun.pan@intel.com>,
        "iommu@lists.linux-foundation.org" <iommu@lists.linux-foundation.org>,
        "linux-kernel@vger.kernel.org" <linux-kernel@vger.kernel.org>
Subject: Re: [PATCH RFC v2 03/11] iommu/sva: Add iommu_domain type for SVA
Message-ID: <20220406130614.GC2120790@nvidia.com>
References: <20220329053800.3049561-1-baolu.lu@linux.intel.com>
 <20220329053800.3049561-4-baolu.lu@linux.intel.com>
 <20220330190201.GB2120790@nvidia.com>
 <BN9PR11MB5276A39F481E2E31B0FCFEAE8CE39@BN9PR11MB5276.namprd11.prod.outlook.com>
 <20220402233210.GM2120790@nvidia.com>
 <BN9PR11MB527660D99D154F922B0A628B8CE79@BN9PR11MB5276.namprd11.prod.outlook.com>
 <20220406012334.GZ2120790@nvidia.com>
 <BN9PR11MB52764E3DE16E19D1F1515D008CE79@BN9PR11MB5276.namprd11.prod.outlook.com>
 <d0f6caea-0cd9-d839-c9fb-0fe49c2bad16@arm.com>
Content-Type: text/plain; charset=us-ascii
Content-Disposition: inline
In-Reply-To: <d0f6caea-0cd9-d839-c9fb-0fe49c2bad16@arm.com>
MIME-Version: 1.0
X-MS-Exchange-AntiSpam-MessageData-ChunkCount: 1
X-MS-Exchange-AntiSpam-MessageData-0: =?us-ascii?Q?p6mmeYEEUGXEN4rP0QpzFOCBCcxfDqDW7OzCE5NojwUCRcu9fOx5pZQJLhLo?=
 =?us-ascii?Q?yCt5PGTOVLuAxcwKBCTDgsFCui2KB3bfVCoWJ9MbumWVF0xhLrJ+1T07BI0c?=
 =?us-ascii?Q?SQLM1c/sJKu//zo/m6StoCGBGPXROq0kJHUzmN3oEce339pvGarLf6fozhGY?=
 =?us-ascii?Q?dASrBTqFzB7k7S+/dGUtOmeogwQOj5B8uxLojA7BOwnTP0mwo6PuuZ61GGcQ?=
 =?us-ascii?Q?LygP2Kqa/qy41Ugl7D8ay52hDxD7xmyB8ol0kbOLtF+DS6yqcBY7Cxng9/3s?=
 =?us-ascii?Q?ftfShOb410HFxqiSUmTY93T4WWGnfGU7udQupURAlRzKFLBZJ9rUNSovriHg?=
 =?us-ascii?Q?BONBe/0xQvU7UweztQKqyzRi7YkURK1GIiuFD9PpKmit7CGptJVjmJi95SLU?=
 =?us-ascii?Q?QqsuqdLWiRkyBnqW7+ZEwCmAsBEksPNt2z/1X+HKPMMfmx0PDxflzq7I2yvb?=
 =?us-ascii?Q?14rsmwtRuMGkFyp+8DXdoF7k6K0rCsGGrRdQsvCzgjL8pOhlUbLdqzHusLvL?=
 =?us-ascii?Q?YpsGWQYEI5qN/1XpBRjSyQquUvTWcMXS2dUnb1XDh+ixmVQ+eA9+WQNuYCMH?=
 =?us-ascii?Q?LH56C7Ol3SsdIJUNebhuvMITTByfyifDZ0wyw0E7XZJOKQIRwqXnoi1Y7oUB?=
 =?us-ascii?Q?SZYzFiNhA0Hn7Z/iI8q93RBbQY+2m9cEnEb972BnG2lQwlLdvIW6FDPtRQf3?=
 =?us-ascii?Q?IN63eoDVNqv59q3LHxbOG0/L2/SJkLu5GupHq9Rew79qLvPLEvqAWZnwuex2?=
 =?us-ascii?Q?FtQbDhgnxEikkuTOdd6IlbJR2EfOcajaxSxzG532wTkN2+feBC/fKDlTtWjg?=
 =?us-ascii?Q?tYHz+tDOi0VlOLYt1f+NL0ulnvTJOxb5cz1Zq35OVUrvspu3P9Sh7CvRVUDo?=
 =?us-ascii?Q?G4pTZjpiaNpwfVScSmaKcKuEw/g+W5eosO+qf+cX8WE9i2S858QAF8BMQ8ej?=
 =?us-ascii?Q?vjg7rwmlpnO3QEOY+1G756ctJ/L8sfYBmOgowAsTGOc6r90FxTKr5Wn2mT3c?=
 =?us-ascii?Q?99tV9rNLYiDdHP6qL3Ar3fPthozaLmPlOeBGBXDm1eIsc50e8EhUWFEKvnco?=
 =?us-ascii?Q?Il76zAnWAxmYgNW/clJnNm6WdG0hGzzp3OgUaxh1WPXn8atIFEyIId89s3Zo?=
 =?us-ascii?Q?09kqmQDdOaelkOPwpFRcOFy/N761h/fJQu4+IQn4WIxSQM7xgfozUeRzvVt0?=
 =?us-ascii?Q?xF5BZ3uPidDIZctqJLM0Ek6iMIi1lm73ZATrIAN9R1Vz1evbSpVRX3Z1YOBN?=
 =?us-ascii?Q?7ILd0qQPAlh3dhtgVCf+oM+2WZwqIkGMgUyiYmQhBbsFd2Nh2/mN6uwnYM3H?=
 =?us-ascii?Q?q1Yi3eODvGVEwnwkvKkBL6xxqZAkcLsfAvDM+fGXwj/gKe1hdwV1TUvWgaQt?=
 =?us-ascii?Q?4uVaq5o2EERhZ2QzSbyPqU9bQwhZIJNovbeuanDZWlexg09+4c/qyJEPjt2E?=
 =?us-ascii?Q?D3znm0uDWywKT84ExK50FUAMXKPKQD8w3FyHZMR9fnaGA+lTlzbH1w4MqbNd?=
 =?us-ascii?Q?NM9uhvn2XWGMBQWTMAyBAoaT6Ii4x2+DlJjvHC/1BksXjQ0Z+++6Ty7N79AU?=
 =?us-ascii?Q?EsyQcSbfqc4U+dPWlIN0v0i/Amom8CfJJFpd+cpWnNfJKrbzSVXFpSNPT4BV?=
 =?us-ascii?Q?fzaJI9e0UBeQqro9bNHuH6EhtSomEWqihxfOewYncVXBfDgC4xnr0JgBez6A?=
 =?us-ascii?Q?d+esaRaZAn87hn/odkkV2Z6xZBBEjagYby3U8JM+ZW60IiWourL6gTWAdftp?=
 =?us-ascii?Q?Sc6e1Qyc9A=3D=3D?=
X-OriginatorOrg: Nvidia.com
X-MS-Exchange-CrossTenant-Network-Message-Id: c21c890e-1a79-4066-5136-08da17ce3be6
X-MS-Exchange-CrossTenant-AuthSource: MN2PR12MB4192.namprd12.prod.outlook.com
X-MS-Exchange-CrossTenant-AuthAs: Internal
X-MS-Exchange-CrossTenant-OriginalArrivalTime: 06 Apr 2022 13:06:16.3091
 (UTC)
X-MS-Exchange-CrossTenant-FromEntityHeader: Hosted
X-MS-Exchange-CrossTenant-Id: 43083d15-7273-40c1-b7db-39efd9ccc17a
X-MS-Exchange-CrossTenant-MailboxType: HOSTED
X-MS-Exchange-CrossTenant-UserPrincipalName: 8OgJkB9CXCly+dxIjQp81BbFH0/Omhy/Bc0+A52TPxgz0G15otZJo41gACOnTAT3
X-MS-Exchange-Transport-CrossTenantHeadersStamped: DM6PR12MB3115
X-Spam-Status: No, score=-2.0 required=5.0 tests=BAYES_00,DKIMWL_WL_HIGH,
	DKIM_SIGNED,DKIM_VALID,DKIM_VALID_AU,HEADER_FROM_DIFFERENT_DOMAINS,
	MAILING_LIST_MULTI,RDNS_NONE,SPF_HELO_NONE,T_SCC_BODY_TEXT_LINE
	autolearn=no autolearn_force=no version=3.4.6
X-Spam-Checker-Version: SpamAssassin 3.4.6 (2021-04-09) on
	lindbergh.monkeyblade.net
Precedence: bulk
List-ID: <linux-kernel.vger.kernel.org>
X-Mailing-List: linux-kernel@vger.kernel.org

On Wed, Apr 06, 2022 at 01:32:07PM +0100, Robin Murphy wrote:
> a particular IOMMU instance, and potentially allocate separate domains for
> separate devices to represent the same address space, much like
> vfio_iommu_type1_attach_group() does.

I think this VFIO code also needs some more work, it currently assumes
that if the domain ops are the same then the domains are compatible,
and that is not true for ARM SMMU drivers.

I've been thinking of adding a domain callback 'can device attach' and
replacing the ops compare with that instead.

> It's not really worth IOMMU drivers trying to support a domain spanning
> potentially-heterogeneous instances internally, since they can't reasonably
> know what matters in any particular situation. 

In the long run I think it will be worth optimizing. If the SMMU
instances can share IOPTE memory then we get two wins - memory
reduction and reduced work to read dirty bits.

The dirty read in particular is very performance sensitive so if real
work loads have many SMMUs per VM it will become a pain point.

Jason