Received-SPF: pass (google.com: domain of linux-kernel+bounces-97111-linux.lists.archive=gmail.com@vger.kernel.org designates 2604:1380:45d1:ec00::1 as permitted sender) client-ip=2604:1380:45d1:ec00::1;
Date: Fri, 8 Mar 2024 09:51:38 -0400
From: Jason Gunthorpe <jgg@nvidia.com>
To: Suravee Suthikulpanit <suravee.suthikulpanit@amd.com>
Cc: linux-kernel@vger.kernel.org, iommu@lists.linux.dev, joro@8bytes.org,
	yi.l.liu@intel.com, kevin.tian@intel.com, nicolinc@nvidia.com,
	eric.auger@redhat.com, vasant.hegde@amd.com, jon.grimm@amd.com,
	santosh.shukla@amd.com, Dhaval.Giani@amd.com, pandoh@google.com,
	loganodell@google.com
Subject: Re: [RFCv2 PATCH 2/7] iommu/amd: Refactor set_dte_entry
Message-ID: <20240308135138.GJ9179@nvidia.com>
References: <20240112000646.98001-1-suravee.suthikulpanit@amd.com>
 <20240112000646.98001-3-suravee.suthikulpanit@amd.com>
Content-Type: text/plain; charset=us-ascii
Content-Disposition: inline
In-Reply-To: <20240112000646.98001-3-suravee.suthikulpanit@amd.com>
Precedence: bulk
MIME-Version: 1.0
X-MS-Exchange-AntiSpam-MessageData-ChunkCount: 1
X-MS-Exchange-AntiSpam-MessageData-0:
	=?us-ascii?Q?JHzune3yagBphGx5OLj+/iAIf9ZcDCAUzfIosaUVcoOWEqQIAH74NO+xBukR?=
 =?us-ascii?Q?em12LOPfqWK+nylFDOp7poX6UFQI1PUL64j9X4TCvyBnWIGe2AsKEAvFlX3m?=
 =?us-ascii?Q?kg3O3ZLsMGZdwaLXREe+u8wTgRddBX9+XDySznDqhU8eN7FgPHxKl+tgRME6?=
 =?us-ascii?Q?UebJXYyzGWwch6zMlmDhwhTL8G9HAp2nHFVs32vbf1NQMYqN4sFAl2biZrMW?=
 =?us-ascii?Q?NqpqaRERPbvxYxL1NIktbbRLH56m42VMbBQ3f2GHsTNkma7/lHqLE0cq4xbP?=
 =?us-ascii?Q?BlY/kWEgZXMit9FFKSAWvAyeK7/An9kfshuH7Yyx3rJ4DzfwZAipPYIcS26F?=
 =?us-ascii?Q?SezwokX+OBa6PCVxjvmC4dYwZW8lLQylOeaFs7S39FsndQVUmS5iQNoyVdsv?=
 =?us-ascii?Q?vvtD1/tp8iEwMRjZQxULv4ArnWbAK+P1tRkeZKXbyAZxjt+bjuLA/1bKh887?=
 =?us-ascii?Q?sMgUrSnAcmcLoZT0suNsBS9u+f0BYS/oxv1k6WGakTW3LqqVkFFPXKAZUR6+?=
 =?us-ascii?Q?9TjYjtGHv0N7DPb2jqHSptuRpNMIWb+VQXaprtXkz8So6a7QLk7+fN8ymCgI?=
 =?us-ascii?Q?5CSE77ntgqJad9baJWifw0qgq4iuKCxfAW/3GT5wfj7U5y6Ofxz2jlTBVqiA?=
 =?us-ascii?Q?9iDGDmZXBrEa88tCJu9tOE4wSydlmBDiEyQk7RpU+4VPO5cyC6QR+zMaMMQ0?=
 =?us-ascii?Q?8W6NCPz38zRAFFkdaPJPZ2yvN60YpaeLlU7XbtRulClGxEE0wZ1T9dzYPLha?=
 =?us-ascii?Q?GPe8LJwro2CRFYjGfkObBtBz0mMvG+JkVY3tYghxWePLFQLVR34Jr3kCsHea?=
 =?us-ascii?Q?vH7mEAkCsgyMqh6tftfJIeh25hNDQhUQgboQtU/caBnVq7aYc5V4pasPREAL?=
 =?us-ascii?Q?liBLStkTRWBk4/rjqG2Rej76G7/r577FuqKk9+urfTQ4ydpEiJKhgQwrZeN9?=
 =?us-ascii?Q?djZMA5zSgKMQwuJf5hkDMMTONBYHIW7ieizE8MjzLYavQCe5Gc5jBv/RgKpn?=
 =?us-ascii?Q?Y1QbfDXNHnsp/m5CWPwjr0SRwTXLZ3AbHaD9MGZUwcmxHOmMbEH7QeHa6+TJ?=
 =?us-ascii?Q?jNzZnL2G8WrDtc0jaoxDeUVsUS4GJhtTOfpeUAfOYVVaif7hbYUflf8VuBrk?=
 =?us-ascii?Q?Dpg6FR5IBoT+BiWIbSAiOJzoeNFE0jIvZ4w0Gp08gslWM/vmTuCvyMas55Qm?=
 =?us-ascii?Q?RpAYEKG6ZSUQOOljfWCCsB8aQ+wAosScjk60Wwk+/AF38RI7zEyh2jgIAAe5?=
 =?us-ascii?Q?f4lS39DehZ1OSv35RTs7spFz+A/LPeYqZXgGfMXXNxdhZpW/K5IRqMLgo4Pp?=
 =?us-ascii?Q?GvMGQgePcl10fA0jcVDY0wMWn+06f2YznC7KpoiQi9gzxuvnVfrhnRC3o4m1?=
 =?us-ascii?Q?XygMguyp6MwcvrNOiLeZ4ulMo8fHk6VwTYM/RwvuqriZrNkomPRJIVJbe8gK?=
 =?us-ascii?Q?gvR2PcQfLXIQ0Oofh9GxS0b1iDKlBu+C2Uicg9p+ay/J7iwwqYyHgMYiLkcT?=
 =?us-ascii?Q?3uWiAV57no5BWzw1lZHWQ+EZiXunzOfYk0fZe4n+tcP6FlVjQTtk4xOKx+Sc?=
 =?us-ascii?Q?YRnLDRZ5x6jFBjNzFT4=3D?=
X-OriginatorOrg: Nvidia.com
X-MS-Exchange-CrossTenant-Network-Message-Id: b7f11d74-d32d-442c-c745-08dc3f76e18f
X-MS-Exchange-CrossTenant-AuthSource: DM6PR12MB3849.namprd12.prod.outlook.com
X-MS-Exchange-CrossTenant-AuthAs: Internal
X-MS-Exchange-CrossTenant-OriginalArrivalTime: 08 Mar 2024 13:51:40.3961
 (UTC)
X-MS-Exchange-CrossTenant-FromEntityHeader: Hosted
X-MS-Exchange-CrossTenant-Id: 43083d15-7273-40c1-b7db-39efd9ccc17a
X-MS-Exchange-CrossTenant-MailboxType: HOSTED
X-MS-Exchange-CrossTenant-UserPrincipalName: uyUgbWE6fqsNqYxtJVrzLvXPHtOEC3jVVKnDmxexpw0puJJGbL+fkVv1CQ80w3Ab
X-MS-Exchange-Transport-CrossTenantHeadersStamped: DM6PR12MB4282

On Thu, Jan 11, 2024 at 06:06:41PM -0600, Suravee Suthikulpanit wrote:

> -	old_domid = dev_table[devid].data[1] & DEV_DOMID_MASK;
> -	dev_table[devid].data[1]  = flags;
> -	dev_table[devid].data[0]  = pte_root;
> +	dev_table[devid].data[0] = target.data[0];
> +	dev_table[devid].data[1] = target.data[1];
> +	dev_table[devid].data[2] = target.data[2];

Maybe you could have convinced me this doesn't have bugs today with 2
qword updates, but no way this is OK.

The multi qword update needs to be sequenced correctly so the HW
doesn't have tearing. The manual talks about this and it talks about
128 bit updates too.

This needs to follow the design ARM has where it uses the special
update logic to reliably sequence the change. Since nesting is making
the update more complex it becomes important.

Jason