Received-SPF: pass (google.com: domain of linux-kernel+bounces-97108-linux.lists.archive=gmail.com@vger.kernel.org designates 147.75.199.223 as permitted sender) client-ip=147.75.199.223;
Date: Fri, 8 Mar 2024 09:48:56 -0400
From: Jason Gunthorpe <jgg@nvidia.com>
To: Suravee Suthikulpanit <suravee.suthikulpanit@amd.com>
Cc: linux-kernel@vger.kernel.org, iommu@lists.linux.dev, joro@8bytes.org,
	yi.l.liu@intel.com, kevin.tian@intel.com, nicolinc@nvidia.com,
	eric.auger@redhat.com, vasant.hegde@amd.com, jon.grimm@amd.com,
	santosh.shukla@amd.com, Dhaval.Giani@amd.com, pandoh@google.com,
	loganodell@google.com
Subject: Re: [RFCv2 PATCH 1/7] iommu/amd: Introduce struct gcr3_tbl_info.giov
Message-ID: <20240308134856.GI9179@nvidia.com>
References: <20240112000646.98001-1-suravee.suthikulpanit@amd.com>
 <20240112000646.98001-2-suravee.suthikulpanit@amd.com>
Content-Type: text/plain; charset=us-ascii
Content-Disposition: inline
In-Reply-To: <20240112000646.98001-2-suravee.suthikulpanit@amd.com>
Precedence: bulk
MIME-Version: 1.0
X-MS-Exchange-AntiSpam-MessageData-ChunkCount: 1
X-MS-Exchange-AntiSpam-MessageData-0:
	=?us-ascii?Q?2PPhY5S1SdKoAKkJG7RFNXHg1S4Csdn/4LhG37bcT3uWUpkEWgv6BAprEfIJ?=
 =?us-ascii?Q?CRV98MFRuteuhwsgYpZridLwwcx2N6lKu+2hFL53BNYqhTqYBcN6P3EAz5yA?=
 =?us-ascii?Q?XuHAg4DoBWMl6L+kzfVJ5RwilUEg8aUbUvuEnoSQc9klhPQ/1MeQ/k0KcdLn?=
 =?us-ascii?Q?BebPOAMKBW9Ws+5/rIkjFb1OSPJTGAr1kWNHE7AmJRMN7xiV09DPBbEJUw19?=
 =?us-ascii?Q?1nzipKPhtUAq8aZxgKPhiPmhEhOT8bw/dUj4BBvEIhsaZfxIHBoYzVa+kStL?=
 =?us-ascii?Q?2M5VkKdsQKv2Vijc7oJJCLY00Nk71VKpYBxKPT/N/gLEOPAclPI9PoIlNw2U?=
 =?us-ascii?Q?viE+fR3MBUMmrHMH7wg4oNeTeq82+7/HzS8PCge8MSofMerA6/PFviq31veI?=
 =?us-ascii?Q?qXnILRQbjPWbHRHe69S5RUvRXqHXLw1IPk/OPd2iTS2Sf6//3YbCUoKrCx7V?=
 =?us-ascii?Q?tCnR0rLpSFDbq0nIn1WIImB1PSjr+Cf0OpnRhcSFag+zYO9WXFubLQKVhMLR?=
 =?us-ascii?Q?RbDTPCb/8LjvitPtoENqNgOtWzw32PdK3dPydCvcH0U5ORkrDvFqaq1tu2u2?=
 =?us-ascii?Q?YzC4nDB/l0P05gZtOw0WjSc/Jf1AaVw/D36bwGWWCHnqxMoOhwoKm1cZkHsq?=
 =?us-ascii?Q?+pTun1MomL+N67sCinAf/rmqoTu6DOepgYHNcw31SFh2/WUl8IsnsGMVmV2D?=
 =?us-ascii?Q?Wfd8um/rVHE2+WSdRy3qdxikmP1o5XOKolCRsaPkbE90YAkZPi4e/Q081jC1?=
 =?us-ascii?Q?3lOzWy++9VMXqT4ow9JwO50kRLHdO8zh4Uyp89AXW/v9DEg85nQQOT7yURON?=
 =?us-ascii?Q?qG087V9MqbsZsaJGnNr3guYKoQXJeJHkFacviwSAv8DAEha0ADLSURPsq0sf?=
 =?us-ascii?Q?Lz0IUjC+D+OWX9NbsDPdvdoKq2qcPmamboQvH0we8BB0J6WuQt3CekkLVHHV?=
 =?us-ascii?Q?ULMiCJ8MBTPo2aTW5jcW9NJeQRClGs0sVylDiQFgGSJIGw1oOIjj5h/aE/t9?=
 =?us-ascii?Q?vxBgj1Aqj0rdrXm6qLpyOuBIAti0B2SVaftiR0WMe6zaT/l0tH1b2dUune4V?=
 =?us-ascii?Q?2Wm3ftXrH3X93nV4qCr2jA+emM7SULrM3n+O0H9FP0z6INtpgjzXn5oit6oX?=
 =?us-ascii?Q?IeULBlqaYurvwovoFhaCLFw19wHibEt7SOgMtcXbsi3ud4IYaMfd+XH50kNO?=
 =?us-ascii?Q?RhPR3kGruEqon4WJ3C4/Do1CrlftWltSqMrO5pHU8ZC6ht2rhDby54WfYKvI?=
 =?us-ascii?Q?eIg2rQ1cnlo3XFXYJGV3DUp5v7H1UAnTEFdvG1XYXVGAo6ojpO94yKArRnRE?=
 =?us-ascii?Q?spPYWGBubrABk/d6n0uR5YvKB0hyYPKRJJCJAkGOUK6P0tRV8OSGanZb3ao1?=
 =?us-ascii?Q?PyW2cxtbk+haSzZBnhbiq8dJ0YrPn3QBJ2ACNXNPOWb0UNbY5QtoWlyOEzb7?=
 =?us-ascii?Q?9DtaKZiDlf4zgOa2Y2vXoHk7RO5eiks4gTmHDVz4sYenHnwjdM51X3vimVcU?=
 =?us-ascii?Q?ebSRroJF8QaqiJOynIpJz4trM9PYrLRPH5AYNt/MxC6hEyS259rBtY8iATKB?=
 =?us-ascii?Q?mDSjMJY6Fnj4W3KuLPE=3D?=
X-OriginatorOrg: Nvidia.com
X-MS-Exchange-CrossTenant-Network-Message-Id: 67282cab-89f9-49d5-a25c-08dc3f768130
X-MS-Exchange-CrossTenant-AuthSource: DM6PR12MB3849.namprd12.prod.outlook.com
X-MS-Exchange-CrossTenant-AuthAs: Internal
X-MS-Exchange-CrossTenant-OriginalArrivalTime: 08 Mar 2024 13:48:58.8693
 (UTC)
X-MS-Exchange-CrossTenant-FromEntityHeader: Hosted
X-MS-Exchange-CrossTenant-Id: 43083d15-7273-40c1-b7db-39efd9ccc17a
X-MS-Exchange-CrossTenant-MailboxType: HOSTED
X-MS-Exchange-CrossTenant-UserPrincipalName: D0qAlBbJtnWM3yEZezQcZIln5TbR2OhjCw2h0NNeSvFZjmDWr5kle94rm2jpaszI
X-MS-Exchange-Transport-CrossTenantHeadersStamped: DM6PR12MB4282

On Thu, Jan 11, 2024 at 06:06:40PM -0600, Suravee Suthikulpanit wrote:
> To track DTE[GIOV] programming during IOMMU domain attach, also add logic
> to determine if the GIOV is required, and set the variable accordinglly.
> 
> This is also a preparation for adding nested domain support, where the GIOV
> setting is determined by the child domain.
> 
> Signed-off-by: Suravee Suthikulpanit <suravee.suthikulpanit@amd.com>
> ---
>  drivers/iommu/amd/amd_iommu_types.h |  1 +
>  drivers/iommu/amd/iommu.c           | 11 +++++++++--
>  2 files changed, 10 insertions(+), 2 deletions(-)

I really think the DTE handling needs to be cleaned up before nesting
will be easy.

Various bits of the DTE should just flow directly through from the
VM's version of the DTE, it is going to be a mess to do that in this
manner

> @@ -2067,6 +2066,14 @@ static int do_attach(struct iommu_dev_data *dev_data,
>  			free_gcr3_table(dev_data);
>  			return ret;
>  		}
> +
> +		/*
> +		 * GIOV is required for PD_MODE_V2 because we need
> +		 * to support the case where the end-point device
> +		 * does not have PASID in the TLP prefix when setting
> +		 * up to use the v2 table.
> +		 */
> +		dev_data->gcr3_info.giov = true;

Ie who clears this once you set it ? :(

Jason