Received-SPF: pass (google.com: domain of linux-kernel+bounces-182382-linux.lists.archive=gmail.com@vger.kernel.org designates 139.178.88.99 as permitted sender) client-ip=139.178.88.99;
Date: Fri, 17 May 2024 14:11:17 -0300
From: Jason Gunthorpe <jgg@nvidia.com>
To: Alex Williamson <alex.williamson@redhat.com>
Cc: "Tian, Kevin" <kevin.tian@intel.com>,
	"Zhao, Yan Y" <yan.y.zhao@intel.com>,
	"kvm@vger.kernel.org" <kvm@vger.kernel.org>,
	"linux-kernel@vger.kernel.org" <linux-kernel@vger.kernel.org>,
	"x86@kernel.org" <x86@kernel.org>,
	"iommu@lists.linux.dev" <iommu@lists.linux.dev>,
	"pbonzini@redhat.com" <pbonzini@redhat.com>,
	"seanjc@google.com" <seanjc@google.com>,
	"dave.hansen@linux.intel.com" <dave.hansen@linux.intel.com>,
	"luto@kernel.org" <luto@kernel.org>,
	"peterz@infradead.org" <peterz@infradead.org>,
	"tglx@linutronix.de" <tglx@linutronix.de>,
	"mingo@redhat.com" <mingo@redhat.com>,
	"bp@alien8.de" <bp@alien8.de>, "hpa@zytor.com" <hpa@zytor.com>,
	"corbet@lwn.net" <corbet@lwn.net>,
	"joro@8bytes.org" <joro@8bytes.org>,
	"will@kernel.org" <will@kernel.org>,
	"robin.murphy@arm.com" <robin.murphy@arm.com>,
	"baolu.lu@linux.intel.com" <baolu.lu@linux.intel.com>,
	"Liu, Yi L" <yi.l.liu@intel.com>
Subject: Re: [PATCH 4/5] vfio/type1: Flush CPU caches on DMA pages in
 non-coherent domains
Message-ID: <20240517171117.GB20229@nvidia.com>
References: <20240507061802.20184-1-yan.y.zhao@intel.com>
 <20240507062138.20465-1-yan.y.zhao@intel.com>
 <20240509121049.58238a6f.alex.williamson@redhat.com>
 <Zj33cUe7HYOIfj5N@yzhao56-desk.sh.intel.com>
 <20240510105728.76d97bbb.alex.williamson@redhat.com>
 <ZkG9IEQwi7HG3YBk@yzhao56-desk.sh.intel.com>
 <BN9PR11MB52766D78684F6206121590B98CED2@BN9PR11MB5276.namprd11.prod.outlook.com>
 <20240516143159.0416d6c7.alex.williamson@redhat.com>
Content-Type: text/plain; charset=us-ascii
Content-Disposition: inline
In-Reply-To: <20240516143159.0416d6c7.alex.williamson@redhat.com>
Precedence: bulk
MIME-Version: 1.0
X-MS-Exchange-AntiSpam-MessageData-ChunkCount: 1
X-MS-Exchange-AntiSpam-MessageData-0:
	=?us-ascii?Q?FZPxmdGBnD+wr/rxlKDhT+/qA3jBhgTQr0q+msHYzD2tjcymw9MG/oN0O9NM?=
 =?us-ascii?Q?zVJL4C/kKjQeOiMdt/tjKGp8mP1cVa4hm0kgKR32YA1sSkBylHwi0trxNZf4?=
 =?us-ascii?Q?qu9SOTJjvzpZsq6/XbpIkwxPGucTYYenukC+69XEFOm3u+Lwwd0nXOR/k+J0?=
 =?us-ascii?Q?twt5VvA3pPlrJpqXwQtA1ELUEXRu8TfYfN+Rvs2jUbhwsxvlFEyueeI1fo1F?=
 =?us-ascii?Q?Yc9Oeso/IrvO8oXcICTrYB7bMLHV8Cnm9Tb6j691WSr7TbZBbSe6J7pIR0Wc?=
 =?us-ascii?Q?wOkK4eGAwPZygzC9ajZMrvcITKZeyuRkZ//qpJuofnhpNfuVpQ37Y1j4uP6z?=
 =?us-ascii?Q?E3DEgaQ8kCTwu0VVVSHziQSQWDAAJKZhjnxqrcLYbZhu2uYhxoRa47a6tscQ?=
 =?us-ascii?Q?fPV9xRifDgj16i0LK/zB91i+ixcoRdCHOgudeVlTRa2w2swunb/DrUDTSSU+?=
 =?us-ascii?Q?P7CBJS0AOtI3lWWPbwmv39V+OS1Ij86PTq6fMB694VTFSg6VupVFujrJ/pkG?=
 =?us-ascii?Q?Hm+bsc4d9KWXHjeV9fZ2YiABUbAT+PY9MmZk0yKycvmO2lbdNave1sJMP6tO?=
 =?us-ascii?Q?HDBldSZ5P+o+5iHtGNxqwC9WlErAzuS/eehqyaRAPL37bmcny7ZiBSZSEJT6?=
 =?us-ascii?Q?cD9Cifmh+dkpOxXJhAPzbfKkUM1eHmQ1GfwkgHw/e3+jiOwk0i7CyKUJl//C?=
 =?us-ascii?Q?y6eZOoKTR30i9KcNolUAY01PhNO2HYjHtG0wpFCh/0ZG4gj5jAbpjVjTk/jw?=
 =?us-ascii?Q?QgnflLrTXy3y9cfuzuvGBHItodmY8gfUtQlWfsvQF5A1N0r1zrWVZ7z1vinV?=
 =?us-ascii?Q?g8KguhxFMtJgCeYo8yZLuvRf8Pf7lofHey8rL8A+XyDC7z/wCZrXnB9ZOfsz?=
 =?us-ascii?Q?mxZJE4PlSnosxuw8PuQwqPYwjMa2Woelc4ZJwDDgeM6lkYZktdM9GhLnLJFK?=
 =?us-ascii?Q?u14KvSUuXArYUx5V9H0pL3kX8lIkR4AAPRnGFduhXZgtx2G76hZyPpFUOQPa?=
 =?us-ascii?Q?PABPYETRTI7trzcpnC9Yc3Wl/k6+YeAxK1mbo7k8DxS8szoNyOZhMNccA3oU?=
 =?us-ascii?Q?4j8ZdvjVtx8TDlz6fIwk6FcYYRR4xFqQhP7X6vTaLZLn7cMDu9DqnUP6t58A?=
 =?us-ascii?Q?mWsmD090NZPj7E7ZIsCPVASP6adv2iPIueQQntMOAFxWsDDY/sfkSwFLI0YV?=
 =?us-ascii?Q?qJNw4oFf+d86kOXCIWlY89YgcBB3IUXhj30iPN9BT+NMWguEqefM/xE3LwwA?=
 =?us-ascii?Q?V5qHDQIozNtHLHs0hAkiP5UZj5nr3lt4PP5l3KBgh7kzHovykV4MM1eHZU31?=
 =?us-ascii?Q?HNY8HEwUg7gwfJL05igD/leNLmPqYaUWpWFXm87+YqK6pzWQ9Z7Xm1f+O2DG?=
 =?us-ascii?Q?judHUnK0e+3e2gT8WFzetCBqSRzEvGUQYUI9FdpCbc70wtqJrGWlWxTm9d1F?=
 =?us-ascii?Q?K2e1drNrgdZ0Vr3KFMbHo3J8mxtpkDjV5on1dxuj3HO6rb7cKuHIT001f2/3?=
 =?us-ascii?Q?6TggP3t3rQS6T6cp1iD1GcUYCFKHHPWOskvSzzdyddDzxzYf4eCf4a2eqZtA?=
 =?us-ascii?Q?S9zTWlOCVoWrXdqO+WdKT5r6YrJz1UYggJH2qip0?=
X-OriginatorOrg: Nvidia.com
X-MS-Exchange-CrossTenant-Network-Message-Id: 863b1499-7d6f-46f8-838a-08dc76945e20
X-MS-Exchange-CrossTenant-AuthSource: DM6PR12MB3849.namprd12.prod.outlook.com
X-MS-Exchange-CrossTenant-AuthAs: Internal
X-MS-Exchange-CrossTenant-OriginalArrivalTime: 17 May 2024 17:11:18.7359
 (UTC)
X-MS-Exchange-CrossTenant-FromEntityHeader: Hosted
X-MS-Exchange-CrossTenant-Id: 43083d15-7273-40c1-b7db-39efd9ccc17a
X-MS-Exchange-CrossTenant-MailboxType: HOSTED
X-MS-Exchange-CrossTenant-UserPrincipalName: dFIQsdTditoQIL8aaC1gSOsefQFSg9ykI9rWG65mTgGzle58FFiWkp+Xub6J3zrh
X-MS-Exchange-Transport-CrossTenantHeadersStamped: IA0PR12MB8424

On Thu, May 16, 2024 at 02:31:59PM -0600, Alex Williamson wrote:

> Yes, exactly.  Zero'ing the page would obviously reestablish the
> coherency, but the page could be reallocated without being zero'd and as
> you describe the owner of that page could then get inconsistent
> results.  

I think if we care about the performance of this stuff enough to try
and remove flushes we'd be better off figuring out how to disable no
snoop in PCI config space and trust the device not to use it and avoid
these flushes.

iommu enforcement is nice, but at least ARM has been assuming that the
PCI config space bit is sufficient.

Intel/AMD are probably fine here as they will only flush for weird GPU
cases, but I expect ARM is going to be unhappy.

Jason