Received-SPF: pass (google.com: domain of linux-kernel-owner@vger.kernel.org designates 2620:137:e000::3:6 as permitted sender) client-ip=2620:137:e000::3:6;
From:   "Tian, Kevin" <kevin.tian@intel.com>
To:     Jason Gunthorpe <jgg@nvidia.com>
CC:     "ankita@nvidia.com" <ankita@nvidia.com>,
        "alex.williamson@redhat.com" <alex.williamson@redhat.com>,
        "yishaih@nvidia.com" <yishaih@nvidia.com>,
        "shameerali.kolothum.thodi@huawei.com" 
        <shameerali.kolothum.thodi@huawei.com>,
        "aniketa@nvidia.com" <aniketa@nvidia.com>,
        "cjia@nvidia.com" <cjia@nvidia.com>,
        "kwankhede@nvidia.com" <kwankhede@nvidia.com>,
        "targupta@nvidia.com" <targupta@nvidia.com>,
        "vsethi@nvidia.com" <vsethi@nvidia.com>,
        "Currid, Andy" <acurrid@nvidia.com>,
        "apopple@nvidia.com" <apopple@nvidia.com>,
        "jhubbard@nvidia.com" <jhubbard@nvidia.com>,
        "danw@nvidia.com" <danw@nvidia.com>,
        "anuaggarwal@nvidia.com" <anuaggarwal@nvidia.com>,
        "dnigam@nvidia.com" <dnigam@nvidia.com>,
        "udhoke@nvidia.com" <udhoke@nvidia.com>,
        "kvm@vger.kernel.org" <kvm@vger.kernel.org>,
        "linux-kernel@vger.kernel.org" <linux-kernel@vger.kernel.org>
Subject: RE: [PATCH v11 1/1] vfio/nvgpu: Add vfio pci variant module for grace
 hopper
Thread-Topic: [PATCH v11 1/1] vfio/nvgpu: Add vfio pci variant module for
 grace hopper
Thread-Index: AQHZ+VwgDWGk0VCFcUeltplt5s48drBCtrFAgAAx5oCAAPC30IAAqQQAgAKZLmA=
Date:   Fri, 13 Oct 2023 03:43:54 +0000
Message-ID: <BN9PR11MB52760535A35EB5EE710625B78CD2A@BN9PR11MB5276.namprd11.prod.outlook.com>
References: <20231007202254.30385-1-ankita@nvidia.com>
 <BN9PR11MB52762EE10CBBDF8AB98A53788CCDA@BN9PR11MB5276.namprd11.prod.outlook.com>
 <20231010113357.GG3952@nvidia.com>
 <BN9PR11MB5276ECF96BAC7F59C93B5E4A8CCCA@BN9PR11MB5276.namprd11.prod.outlook.com>
 <20231011120026.GU3952@nvidia.com>
In-Reply-To: <20231011120026.GU3952@nvidia.com>
Accept-Language: en-US
Content-Language: en-US
x-ms-exchange-antispam-messagedata-chunkcount: 1
x-ms-exchange-antispam-messagedata-0: =?us-ascii?Q?0mC1kJS6bJmJI1J5iCsBF1tHTogIxWr/Z/8tI9SlNledeZ5BjB7mxd4EvPFK?=
 =?us-ascii?Q?KJ5MU0Q9286KblPeC1mOlAXY9yrNcQjLdFO8jMfrQ5yC+Hr+pWBPwG2ABaXH?=
 =?us-ascii?Q?qetoib+Z86esy12a/vKIYEb/Sk98XddUX8TJjV6WWwgja1K3mQaa226RiObO?=
 =?us-ascii?Q?EsGbqqziwnzLNUrPHSFgA8iqBPjZ4zFl1Td7ioCTD5TUknaQ8k0cHOr7r39g?=
 =?us-ascii?Q?RlluJm5sMp4P2kmHnU9aVvx44CDkBV9hAEVEXnwKWk8zrp2f1N4Ph9yMfMRE?=
 =?us-ascii?Q?WPJZnkJ6UB+he5vTPh9QfGyl2f858Wk31PQ/UJjnDN//dsRkl9svpw7d2RGY?=
 =?us-ascii?Q?S/S1Sqw6a6gZlp8Vm7BayimX036yBWnC0GXtLvGfr32M73Sbr1SFLb6Lzt5F?=
 =?us-ascii?Q?Dsty+FU+wPQjv+IybOtmLK1osgWodUcr6Q9cOFnuRCxS9MeRa/dKfWaVyJdm?=
 =?us-ascii?Q?796dt6waf7epFBkljTtacwC/N3T/jVViLdJDwS5h1KS7q6cEHtja4Q4olr5H?=
 =?us-ascii?Q?w0oBIkPuoAoCaCStrAvyLfuHN6YtLvlV7fRmyJ2//sB5c+yWbBgYW3qm9B+H?=
 =?us-ascii?Q?H53KKO27LlgP/JrqOhN3sgIRU68+PiSvrA1BT7jIb5ekLP4E2chSqJ3VQmxe?=
 =?us-ascii?Q?ueaHwQWSnoVfJHjnwiBU4w2qOH7wPxIMhCtrTwX7f99jihEk9b5kGqQWmVp+?=
 =?us-ascii?Q?PuGCUfZx4yMnIB+zhtulMFnfEiHAcv/+fRGSIg270Bl43j4whAaNPclQDrWE?=
 =?us-ascii?Q?7yTR8tkNUCmvxsA+CvLjKxJfoP2ju2inSkSJ96HUGgbrXpIIzo8xpF/OzrPo?=
 =?us-ascii?Q?0oHxU5X/MI1L7cILslMQV4AKw2YizcpK9lrZB3lAR/jHpToPFU4GAXEdrP2z?=
 =?us-ascii?Q?9LBqG2SyliN3GPiZz/HAKAzUxeePhWGkRoOt19SOO4zJMLTWE+iLwtnelaKr?=
 =?us-ascii?Q?Wrvv7alzcUHjKjLI2z2wrt52CnSKAhIbHJXYyKivWyudbOgrW9ngw7gttXvR?=
 =?us-ascii?Q?axPhvDcR16BhOuC4Jz02TKWb6VGPLxtpoe9zUzuHudsB5bzZsQhq5S3BBAha?=
 =?us-ascii?Q?jJqPHNG1Y5DhdHbiTmPjZ9bz3ZIpmQYKHZYIyln9bJmfDwYylMAZ/iBg2gEe?=
 =?us-ascii?Q?ck/F2S/8rrlW8shtyTqSrNJ3USZfkVSR00b8ZHO3UOJOCCC2XZSlE8U0fFJo?=
 =?us-ascii?Q?VmwZ/HVRs2HMwHETRfhCVlxrM/0yvIe3aZtQNphRvd4yczZ8xZBtwmIgVECr?=
 =?us-ascii?Q?iLIK3BQzxj7GQTzvxCP5jMes/dBMCsJOHYIEzB7R4Fp3F3m/sIpX1BMeJO0c?=
 =?us-ascii?Q?spmiYeebS+dA/s2+nmfzwtqx/DQdN9fEXS9Ol4Y3nMLnSqQLbX7gVhPgNdXA?=
 =?us-ascii?Q?3+8e62+iyimsp1iIP8ol9vwd7QhGL68SLCZ1Og+odcZygmIi31395AYLXr1V?=
 =?us-ascii?Q?q/z8mV7JGcYwY3gqUC3NITw9iuSaDa/bxfUYKL2UkKkytDXpFvfolAb5RKdF?=
 =?us-ascii?Q?cgecUCA36q9pZWrnoZKKRZek386Yn6dVW2yt/XzSy3uUkeUBidM78j+vA+2D?=
 =?us-ascii?Q?f+iG7rTZDvVjtB2qMk5Yg9+vHKKlO/E6KoqyBaFZ?=
Content-Type: text/plain; charset="us-ascii"
Content-Transfer-Encoding: quoted-printable
MIME-Version: 1.0
X-MS-Exchange-CrossTenant-AuthAs: Internal
X-MS-Exchange-CrossTenant-AuthSource: BN9PR11MB5276.namprd11.prod.outlook.com
X-MS-Exchange-CrossTenant-Network-Message-Id: 130c0867-b33b-4be2-daad-08dbcb9e9fc8
X-MS-Exchange-CrossTenant-originalarrivaltime: 13 Oct 2023 03:43:54.8592
 (UTC)
X-MS-Exchange-CrossTenant-fromentityheader: Hosted
X-MS-Exchange-CrossTenant-id: 46c98d88-e344-4ed4-8496-4ed7712e255d
X-MS-Exchange-CrossTenant-mailboxtype: HOSTED
X-MS-Exchange-CrossTenant-userprincipalname: s/K9wXx4+g3TUvf91vRh/ydxL/OWUsBrpybiumnzba3RZ/oNjPdnIjR7o1P6SI7LGh9RCYAyNB+Dyi0luVFHlQ==
X-MS-Exchange-Transport-CrossTenantHeadersStamped: CH3PR11MB7842
X-OriginatorOrg: intel.com
X-Spam-Status: No, score=-0.9 required=5.0 tests=DKIMWL_WL_HIGH,DKIM_SIGNED,
	DKIM_VALID,DKIM_VALID_AU,HEADER_FROM_DIFFERENT_DOMAINS,
	MAILING_LIST_MULTI,SPF_HELO_NONE,SPF_PASS autolearn=unavailable
	autolearn_force=no version=3.4.6
X-Spam-Checker-Version: SpamAssassin 3.4.6 (2021-04-09) on pete.vger.email
Precedence: bulk
List-ID: <linux-kernel.vger.kernel.org>
X-Mailing-List: linux-kernel@vger.kernel.org
X-Greylist: Sender passed SPF test, not delayed by milter-greylist-4.6.4 (pete.vger.email [0.0.0.0]); Thu, 12 Oct 2023 20:44:16 -0700 (PDT)

> From: Jason Gunthorpe <jgg@nvidia.com>
> Sent: Wednesday, October 11, 2023 8:00 PM
>=20
> On Wed, Oct 11, 2023 at 02:00:30AM +0000, Tian, Kevin wrote:
> > > From: Jason Gunthorpe <jgg@nvidia.com>
> > > Sent: Tuesday, October 10, 2023 7:34 PM
> > >
> > > On Tue, Oct 10, 2023 at 08:42:13AM +0000, Tian, Kevin wrote:
> > > > > From: ankita@nvidia.com <ankita@nvidia.com>
> > > > > Sent: Sunday, October 8, 2023 4:23 AM
> > > > >
> > > > > PCI BAR are aligned to the power-of-2, but the actual memory on t=
he
> > > > > device may not. A read or write access to the physical address fr=
om
> the
> > > > > last device PFN up to the next power-of-2 aligned physical addres=
s
> > > > > results in reading ~0 and dropped writes.
> > > > >
> > > >
> > > > my question to v10 was not answered. posted again:
> > >
> > > The device FW knows and tells the VM.
> > >
> >
> > This driver reads the size info from ACPI and records it as
> > nvdev->memlength.
>=20
> Yes, the ACPI tables have a copy of the size.

So the ACPI copy is mainly introduced for the use of VFIO given the
real device driver can already retrieve it from device FW?

>=20
> > But nvdev->memlength is not exposed to the userspace. How does the
> virtual
> > FW acquires this knowledge and then report it to the VM?
>=20
> It isn't virtual FW, I said device FW. The device itself knows how it
> is configured and it can report details about the the memory
> space. The VM just DMA's a RPC and asks it.
>=20

Interesting. Perhaps this message should be included in the commit
message to help compose a clear picture.

With that,

Reviewed-by: Kevin Tian <kevin.tian@intel.com>