Received-SPF: pass (google.com: domain of linux-kernel+bounces-87092-linux.lists.archive=gmail.com@vger.kernel.org designates 2604:1380:45d1:ec00::1 as permitted sender) client-ip=2604:1380:45d1:ec00::1;
From: Ankit Agrawal <ankita@nvidia.com>
To: Alex Williamson <alex.williamson@redhat.com>
CC: Jason Gunthorpe <jgg@nvidia.com>, Yishai Hadas <yishaih@nvidia.com>,
	"shameerali.kolothum.thodi@huawei.com"
	<shameerali.kolothum.thodi@huawei.com>, "kevin.tian@intel.com"
	<kevin.tian@intel.com>, Aniket Agashe <aniketa@nvidia.com>, Neo Jia
	<cjia@nvidia.com>, Kirti Wankhede <kwankhede@nvidia.com>, "Tarun Gupta
 (SW-GPU)" <targupta@nvidia.com>, Vikram Sethi <vsethi@nvidia.com>, Andy
 Currid <acurrid@nvidia.com>, Alistair Popple <apopple@nvidia.com>, John
 Hubbard <jhubbard@nvidia.com>, Dan Williams <danw@nvidia.com>, Rahul
 Rameshbabu <rrameshbabu@nvidia.com>, Zhi Wang <zhiw@nvidia.com>, "Anuj
 Aggarwal (SW-GPU)" <anuaggarwal@nvidia.com>, Matt Ochs <mochs@nvidia.com>,
	"kvm@vger.kernel.org" <kvm@vger.kernel.org>, "linux-kernel@vger.kernel.org"
	<linux-kernel@vger.kernel.org>
Subject: Re: [PATCH v1 1/1] vfio/nvgrace-gpu: Convey kvm that the device is wc
 safe
Thread-Topic: [PATCH v1 1/1] vfio/nvgrace-gpu: Convey kvm that the device is
 wc safe
Thread-Index: AQHaan8STeXbyerayU2Kw+zSatmIBbEhesSAgAAC+YCAAAYi6Q==
Date: Thu, 29 Feb 2024 16:31:09 +0000
Message-ID:
 <SA1PR12MB719972594616B5BF71356243B05F2@SA1PR12MB7199.namprd12.prod.outlook.com>
References: <20240228194801.2299-1-ankita@nvidia.com>
	<20240229085639.484b920c.alex.williamson@redhat.com>
 <20240229090717.67b9c2ca.alex.williamson@redhat.com>
In-Reply-To: <20240229090717.67b9c2ca.alex.williamson@redhat.com>
Accept-Language: en-US
Content-Language: en-US
msip_labels:
authentication-results: dkim=none (message not signed)
 header.d=none;dmarc=none action=none header.from=nvidia.com;
x-ms-publictraffictype: Email
x-ms-traffictypediagnostic: SA1PR12MB7199:EE_|PH7PR12MB6465:EE_
x-ms-office365-filtering-correlation-id: e1388342-dd08-4dc6-5c40-08dc3943d5f9
x-ms-exchange-senderadcheck: 1
x-ms-exchange-antispam-relay: 0
x-microsoft-antispam: BCL:0;
x-microsoft-antispam-message-info:
 Cx0FFJ3LgH4IwPLC+ozsSIfjcv+rI4hKZnMC3UUic5BHNK5oTJdP4/RmnA62udFBgB5QMDShGuKoHjqp1nO4DbNx6ubCSAJ6NK5lnIGHEBDkRCaBu88s/3g24BlsJjI05Xgnqw5JZxEDFuLxi9xVJrDw0n7p82uQ9a5YcDHxP/wRBGgWjNlauInTbz2OXmxPxc44p4Enfx17O48q5sRPs0w8rwh/UMsijFKCnD0afgW7hyiSEujXMRl8jjGBDZpCAuPA6+fPAsu0O9ypmD/vqlIkE7jMUVdKVVt+ju+mXepECXDIm8Mvmyb3fwzS3N15SuMECYHI8Wy+oyxqjLQlifNr2Sj+GMOju5QHKun3ouPrDVKnGVEgAwbKSibzZdgH9FuHEuWtVomcbIftUZqO/S2oO/4aCRBsm43C7DGBUGF800U6CGg9DcKdwoK57ND1omBhuLOOZJXggIdJwjOLoDPZHjNQwr28Q44IqF4KsLoXUyDitw5F0kD9gLmsxgA29QzCk4lJXHwpXKErhlyIJ2RH07Yc+fbgyVNNtlP/r/FBuojsybbdHSD6EZeA1dEvWC1C+kmG17hLxpi//KYZM7E6AQq4j4wxOOCAplrj4rJZ4HmmDntb8NP2n2Mj/7BMm5b3tyKgElnmOwRhqgBVsi5m9hCj4Qhm22BG5VNvOaKReueIgYpylqOljkGcsRaI3i2WSDKziMJXiH7rUbqyDQSyFEbbyrqyN5JbQD9fn74=
x-forefront-antispam-report:
 CIP:255.255.255.255;CTRY:;LANG:en;SCL:1;SRV:;IPV:NLI;SFV:NSPM;H:SA1PR12MB7199.namprd12.prod.outlook.com;PTR:;CAT:NONE;SFS:(13230031)(38070700009);DIR:OUT;SFP:1101;
x-ms-exchange-antispam-messagedata-chunkcount: 1
x-ms-exchange-antispam-messagedata-0:
 =?iso-8859-1?Q?ukYf7fQIn5fbQ9WDJds07NrAByw6q9sSIGvhchw5EP0WqOTIzkrR3qkGGA?=
 =?iso-8859-1?Q?Ve7GWYhlNmXt3AYy6dxVBgsIruQ1OmH/+E9aMx9dkCV1CP6rgMQEJLq8Lp?=
 =?iso-8859-1?Q?coyZDMX/sDKi9pudxnjuHp4zTMqFHtErVPxW7oa+sCYAqYWTvKQt2BIl/I?=
 =?iso-8859-1?Q?bN+LzNbRo4THfgei7lZB68yNMAye5G3XV3bw2zTavyn7krLVetShT5Bb7f?=
 =?iso-8859-1?Q?lrsbUZ1wFvDFcPFWuqoM+hfcSUVDfiNn6aJTKdpEWmOaafIo7ULj2uDmZ9?=
 =?iso-8859-1?Q?gyv/3J0W/D2l9eiKVqdc792xESyaDbsgpuAF1xm83fX4rGwSegYcy4gVVM?=
 =?iso-8859-1?Q?h0QOUyX4FNazxs/qLsaLXIXMy99Mkyu+egOxockwWgj0djIg9GtXFzfct8?=
 =?iso-8859-1?Q?dvbUq0VAxcqEf+YCmSaF4F9M9oVRzyQgKh3CJlwj3AoV/wAdLhqYhN5Cvz?=
 =?iso-8859-1?Q?l4KTtyt/I0XrFlNXMSmWS1et6XTBh/V6bT8UUjuJNlVtuW8ZnhezeNT2Ht?=
 =?iso-8859-1?Q?TwkrBcrpyqb8LWPVRDgPu1hdfVOjbR5efeWPmdJFp5qH2Wv4JXWDqjXS7f?=
 =?iso-8859-1?Q?PtsOJm9Hd3JVg5dc7jy0wrDw5pmNYYtnswUtmvzN+BmEAp5bN4ToOzXOqA?=
 =?iso-8859-1?Q?pHSzdhiZ9ZHSZrSvTVe9b2o0xy1Nc1+6aysUe3R8d4nKZgygz3luk8jlyf?=
 =?iso-8859-1?Q?TzDzRTPZumEe+BNA6RPFFu1V5spsESayMSl0N11Ytiqv9i/hUpeDKs0tAy?=
 =?iso-8859-1?Q?pJ1uKbvNbDItgyC6i7cOJPY5yc13pEcaQA/418TTADvUh2A4aoRJDHITec?=
 =?iso-8859-1?Q?TR9yHysoeMaenzHf9pq7b9Q37R6o5nZxSIG3sRKlLFa+V3p8UcswxduKlW?=
 =?iso-8859-1?Q?tCbmYNMpDRG9K72bVyIxf+6HBc5S2TaIME1JRr66Fa0Z6O4O5e0beQTVAu?=
 =?iso-8859-1?Q?VZyHQB4o3oHjKC86hWPl5LTgIlisEcN3wrd2fk9CIn7X1hratZcR3adgO0?=
 =?iso-8859-1?Q?85j4tY3IO6eDp8hb+5WXV72jBhnczaniL79voMVRyS+NB8foX+cCPZ7qrt?=
 =?iso-8859-1?Q?n/Spryq1m2HT5f3V3y2MWfgwj5pw+JtR88OpoKOJv7hOOfRqIjm4k/XBXX?=
 =?iso-8859-1?Q?G5jhPW5SxJRm9nR53FWy5tsqLD3J55Pl9nlYLDAG1Lpkm7oMM6HUUIdpPk?=
 =?iso-8859-1?Q?Grln9t6ACW+d6hP1io0yhWXSvX08We3UbI7JfhW5seUC5ZJj/5BV49RmvX?=
 =?iso-8859-1?Q?Iw5dn4sNEwhImEfax++wAkHOmkRVlBAk1dSzRiA2bcLFhNzqHwcwInL9Mw?=
 =?iso-8859-1?Q?QWpcTy0IfSfSO5O/jTHmN3loMpIUVe6zakwzs0p8bbAXJekS0Vm7wiF18Y?=
 =?iso-8859-1?Q?rQIhaMWUj3NLq2zn4xkSHb5VPmE9wAQqnCatNMh0QiHDPvQW8S7mxUhgu/?=
 =?iso-8859-1?Q?cnpjKhzlsgsiUBLRpE6ydjEWd4YW/hXc7rXGmmy3qpvy8b5L2/mL7Wi5Ui?=
 =?iso-8859-1?Q?EcPx9z4QYZYKFMGNSx+dcKY1KcBJNg0M3Ef1eEvsyfSsbM4qm3RIPn5g0f?=
 =?iso-8859-1?Q?7QygrBVLy9rB2tcpHRpCBXZ5961l0f53S37yYyOc2fkpoUuisf3304VD+l?=
 =?iso-8859-1?Q?RPnBSq9IaMlTg=3D?=
Content-Type: text/plain; charset="iso-8859-1"
Content-Transfer-Encoding: quoted-printable
Precedence: bulk
X-Mailing-List: linux-kernel@vger.kernel.org
List-Id: <linux-kernel.vger.kernel.org>
List-Subscribe: <mailto:linux-kernel+subscribe@vger.kernel.org>
List-Unsubscribe: <mailto:linux-kernel+unsubscribe@vger.kernel.org>
MIME-Version: 1.0
X-OriginatorOrg: Nvidia.com
X-MS-Exchange-CrossTenant-AuthAs: Internal
X-MS-Exchange-CrossTenant-AuthSource: SA1PR12MB7199.namprd12.prod.outlook.com
X-MS-Exchange-CrossTenant-Network-Message-Id: e1388342-dd08-4dc6-5c40-08dc3943d5f9
X-MS-Exchange-CrossTenant-originalarrivaltime: 29 Feb 2024 16:31:09.5168
 (UTC)
X-MS-Exchange-CrossTenant-fromentityheader: Hosted
X-MS-Exchange-CrossTenant-id: 43083d15-7273-40c1-b7db-39efd9ccc17a
X-MS-Exchange-CrossTenant-mailboxtype: HOSTED
X-MS-Exchange-CrossTenant-userprincipalname: wxI60WQ5djiSttmEXA9uNQKapU1UFrlb+Jhfe5vLmHKqLIgDfC8qjN+HFRpv+zXr8q8hXnPSb2S1DKgMdwNOEA==
X-MS-Exchange-Transport-CrossTenantHeadersStamped: PH7PR12MB6465

>> > +=A0=A0 /*=0A=
>> > +=A0=A0=A0 * The VM_ALLOW_ANY_UNCACHED VMA flag is implemented for ARM=
64,=0A=
>> > +=A0=A0=A0 * allowing KVM stage 2 device mapping attributes to use Nor=
mal-NC=0A=
>> > +=A0=A0=A0 * rather than DEVICE_nGnRE, which allows guest mappings=0A=
>> > +=A0=A0=A0 * supporting write-combining attributes (WC). This also=0A=
>> > +=A0=A0=A0 * unlocks memory-like operations such as unaligned accesses=
=0A=
>> > +=A0=A0=A0 * This setting suits the fake BARs as they are expected to=
=0A=
>> > +=A0=A0=A0 * demonstrate such properties within the guest.=0A=
>> > +=A0=A0=A0 *=0A=
>> > +=A0=A0=A0 * ARM does not architecturally guarantee this is safe, and =
indeed=0A=
>> > +=A0=A0=A0 * some MMIO regions like the GICv2 VCPU interface can trigg=
er=0A=
>> > +=A0=A0=A0 * uncontained faults if Normal-NC is used. The nvgrace-gpu=
=0A=
>> > +=A0=A0=A0 * however is safe in that the platform guarantees that no=
=0A=
>> > +=A0=A0=A0 * action taken on the MMIO mapping can trigger an uncontain=
ed=0A=
>> > +=A0=A0=A0 * failure. Hence VM_ALLOW_ANY_UNCACHED is set in the VMA fl=
ags.=0A=
>> > +=A0=A0=A0 */=0A=
>> > +=A0=A0 vm_flags_set(vma, VM_ALLOW_ANY_UNCACHED);=0A=
>> > +=0A=
>> >=A0=A0=A0=A0 return 0;=0A=
>> >=A0 }=0A=
>> >=0A=
>>=0A=
>> The commit log sort of covers it, but this comment doesn't seem to=0A=
>> cover why we're setting an uncached attribute to the usemem region=0A=
>> which we're specifically mapping as coherent... did we end up giving=0A=
>> this flag a really poor name if it's being used here to allow unaligned=
=0A=
>> access?=A0 Thanks,=0A=
>=0A=
> Also, this is setting the vma flag *after* the call to=0A=
> remap_pfn_range(), which seems quite sketchy.=A0 Thanks,=0A=
=0A=
I will move it to the block that Jason pointed.=0A=
=0A=
> Alex=0A=