Date:   Fri, 10 Dec 2021 14:53:40 -0400
From:   Jason Gunthorpe <jgg@nvidia.com>
To:     Jacob Pan <jacob.jun.pan@linux.intel.com>
Cc:     Lu Baolu <baolu.lu@linux.intel.com>,
        iommu@lists.linux-foundation.org,
        LKML <linux-kernel@vger.kernel.org>,
        Joerg Roedel <joro@8bytes.org>,
        Christoph Hellwig <hch@infradead.org>,
        Greg Kroah-Hartman <gregkh@linuxfoundation.org>,
        Jean-Philippe Brucker <jean-philippe@linaro.com>,
        Jacob Pan <jacob.jun.pan@intel.com>,
        Raj Ashok <ashok.raj@intel.com>,
        "Kumar, Sanjay K" <sanjay.k.kumar@intel.com>,
        Dave Jiang <dave.jiang@intel.com>,
        Tony Luck <tony.luck@intel.com>, Yi Liu <yi.l.liu@intel.com>,
        "Tian, Kevin" <kevin.tian@intel.com>,
        Barry Song <21cnbao@gmail.com>,
        "Zanussi, Tom" <tom.zanussi@intel.com>,
        Dan Williams <dan.j.williams@intel.com>
Subject: Re: [PATCH 3/4] iommu/vt-d: Support PASID DMA for in-kernel usage
Message-ID: <20211210185340.GI6385@nvidia.com>
References: <1638884834-83028-1-git-send-email-jacob.jun.pan@linux.intel.com>
 <1638884834-83028-4-git-send-email-jacob.jun.pan@linux.intel.com>
 <20211208132255.GS6385@nvidia.com>
 <20211208111659.6de22e52@jacob-builder>
 <9f724b3a-6028-43d7-b4fc-d8a939e7b2cf@linux.intel.com>
 <20211209152113.64b817b9@jacob-builder>
 <921a766f-d826-2ca4-f739-4d196b32a681@linux.intel.com>
 <20211210095025.38af67ce@jacob-builder>
 <20211210174848.GH6385@nvidia.com>
 <20211210101820.58b24b77@jacob-builder>
Content-Type: text/plain; charset=us-ascii
Content-Disposition: inline
In-Reply-To: <20211210101820.58b24b77@jacob-builder>
MIME-Version: 1.0
X-MS-Exchange-AntiSpam-MessageData-ChunkCount: 1
X-MS-Exchange-AntiSpam-MessageData-0: =?us-ascii?Q?5VqrZKN2K0W59GBBwV9rCgXITglBYbeTnCqTzO0lsn8LkQGyqjrjvk5qSMrE?=
 =?us-ascii?Q?CU9jQadCt+khThr6LiKSxmUiY4Tw0hFCsNQwFRDvihImxK/jkr7IhWgbrHxx?=
 =?us-ascii?Q?xYLC5wBVm3+XR2i9HFV7Y+GWkRmdrLr5ui0m4vWx9XMLQbfHeZ/3pEw/l+by?=
 =?us-ascii?Q?qtHdnPM8rA6xkVwLUr5WAFnEpEVD2RveY5Sa8zid+bFwKodaFyYJEySVGO5x?=
 =?us-ascii?Q?TphqFibjVtZhkqo6Y/LdHMFkHByZXTE2dN/EmGZzhUe15/0kHkWzQ/F1yn4K?=
 =?us-ascii?Q?2xXzFPw+C7p1zDD5cMmcoyLuUfbpdkR7e4BeGaBDInqduAPxB+bhOEQpOcRP?=
 =?us-ascii?Q?0KC7Nb9BJBtYNJ8uax8P4mkWjUjQTYNulz1oelXDbQSoP652iFpIAd4gv9h9?=
 =?us-ascii?Q?IiVOH67dGc97LFW3vkh48AKe5VpWynMLJnFkUBJbTmvNdBas1hjBaut6A2rV?=
 =?us-ascii?Q?0rRbBVwDZwkGuXwC5i6eLpoTxRSGWpZ/GFkOTLng44kDqp5lQSPgteaSRvlU?=
 =?us-ascii?Q?M25WGdMAM4DWnKNkwmhjSpt8dZHt4QGsdG1z8gDvfnqjLvjQMFSI5NNbzuKl?=
 =?us-ascii?Q?P43RgIsnW3Ga3+pG+EBh1soN4+w8eiq6LbM2u6rBBJvOgAcqLRw/YKDAJusl?=
 =?us-ascii?Q?a2fQrghAlFK8s47vQEm18HUk93estIdlwZnielm29nwCo+n8LvBw5lUbkocR?=
 =?us-ascii?Q?b8XeNoNzuG1Of+CVCw9y0c/rq1nH16Z49xE7pzb7kfqMAp5cISL0c32mBFp6?=
 =?us-ascii?Q?UFdHZc1SudaUhjBsPbo4aRTbqLF24+Hry9UVe+Ds5p/jiED/i0o3+LE9DSg3?=
 =?us-ascii?Q?M6BU3zrw/6qJzxiOS78NTmxHzUOqXV1h5wFltACnLbfu+p712bNoL+f4Ye7z?=
 =?us-ascii?Q?9g/QiPu1Tr0lCDA9Vc82bJ3YOmYM63lFeycB/jqDXJnv+q5oLowKAAeGc3lO?=
 =?us-ascii?Q?A59zYqiUZk2/K0Ahtavj/WxVvw9tS82fd66cli5m1wtsYuYRcu306ZKHLe4e?=
 =?us-ascii?Q?IIDnoXwHD3+8nT1NGK8HCPWbd6u8GJUw4BySvFwBVpZDKqy1/XOdUFCLsclj?=
 =?us-ascii?Q?s7EVjm94IWyO/zQFm8CWfM9g6Gvo6+99vjBrpZWhc3pq3QURkz3NfRQarzZU?=
 =?us-ascii?Q?sn0KwGtYbptTqmOdqusC4DVa5CY+QrPqgub+0XnjL7zoLkBGKGAqdz0ixAHJ?=
 =?us-ascii?Q?t0M8kdEn25+FPk9h/Aq8goN1GDtwNpD3FNy83PXTQnlYeeAuotXgxadPoO1d?=
 =?us-ascii?Q?EKQsI1isiQM9ujVLrXg3qoqLo6EYS7uQlxbAHlcxaS1Ctho2iVIIvt+9Yf5R?=
 =?us-ascii?Q?EurX/lzKLk8OaQdm8lZpiCnM3NuXTHcQugK9zjiJ1Gd64Md2689ttJgzdRuu?=
 =?us-ascii?Q?wOCvZpGnRQqrBpv7iPR8bSkgTxAOkMrHc1RvBtKRxulpTsK6gZhQUWJWKtIY?=
 =?us-ascii?Q?xj+WJZnXr7AZoo29MNDVQBgH3ksNLyLfc7FBhmmieBBOiZEdBz2uOhRYvVQz?=
 =?us-ascii?Q?Ck1DQaYXz12egv1JgQ0oX/QmxenC2JAWQi1l+xQ62mtWiseHJt8TspiKp72Q?=
 =?us-ascii?Q?tIXZxfniaL5DMN46MU8=3D?=
X-OriginatorOrg: Nvidia.com
X-MS-Exchange-CrossTenant-Network-Message-Id: 1e6fcc32-309e-4ec7-5a82-08d9bc0e6308
X-MS-Exchange-CrossTenant-AuthSource: BL0PR12MB5506.namprd12.prod.outlook.com
X-MS-Exchange-CrossTenant-AuthAs: Internal
X-MS-Exchange-CrossTenant-OriginalArrivalTime: 10 Dec 2021 18:53:42.7940
 (UTC)
X-MS-Exchange-CrossTenant-FromEntityHeader: Hosted
X-MS-Exchange-CrossTenant-Id: 43083d15-7273-40c1-b7db-39efd9ccc17a
X-MS-Exchange-CrossTenant-MailboxType: HOSTED
X-MS-Exchange-CrossTenant-UserPrincipalName: W2G6Mhqc4Ks+bN8aeuvzNFO+Z9WE6ZtdCQlaay6pND8v7PsnEl5Zlgtl2BkLKn5t
X-MS-Exchange-Transport-CrossTenantHeadersStamped: BL1PR12MB5363
Precedence: bulk
List-ID: <linux-kernel.vger.kernel.org>
X-Mailing-List: linux-kernel@vger.kernel.org

On Fri, Dec 10, 2021 at 10:18:20AM -0800, Jacob Pan wrote:

> > If one device has 10 PASID's pointing to this domain you must flush
> > them all if that is what the HW requires.
> > 
> Yes. My point is that other than PASID 0 is a given, we must track the 10
> PASIDs to avoid wasted flush. It also depend on how TLBs are tagged and
> flush granularity available. But at the API level, should we support all the
> cases?

Yes, iommufd will expose all the cases to userspace and anything is
possible.

A scheme that can only attach a domain to one PASID is functionally
useless except for this IDXD problem, so don't make something so
broken.

Jason