Received-SPF: pass (google.com: domain of linux-kernel-owner@vger.kernel.org designates 2620:137:e000::3:3 as permitted sender) client-ip=2620:137:e000::3:3;
Date:   Tue, 10 Oct 2023 08:33:57 -0300
From:   Jason Gunthorpe <jgg@nvidia.com>
To:     "Tian, Kevin" <kevin.tian@intel.com>
Cc:     "ankita@nvidia.com" <ankita@nvidia.com>,
        "alex.williamson@redhat.com" <alex.williamson@redhat.com>,
        "yishaih@nvidia.com" <yishaih@nvidia.com>,
        "shameerali.kolothum.thodi@huawei.com" 
        <shameerali.kolothum.thodi@huawei.com>,
        "aniketa@nvidia.com" <aniketa@nvidia.com>,
        "cjia@nvidia.com" <cjia@nvidia.com>,
        "kwankhede@nvidia.com" <kwankhede@nvidia.com>,
        "targupta@nvidia.com" <targupta@nvidia.com>,
        "vsethi@nvidia.com" <vsethi@nvidia.com>,
        "Currid, Andy" <acurrid@nvidia.com>,
        "apopple@nvidia.com" <apopple@nvidia.com>,
        "jhubbard@nvidia.com" <jhubbard@nvidia.com>,
        "danw@nvidia.com" <danw@nvidia.com>,
        "anuaggarwal@nvidia.com" <anuaggarwal@nvidia.com>,
        "dnigam@nvidia.com" <dnigam@nvidia.com>,
        "udhoke@nvidia.com" <udhoke@nvidia.com>,
        "kvm@vger.kernel.org" <kvm@vger.kernel.org>,
        "linux-kernel@vger.kernel.org" <linux-kernel@vger.kernel.org>
Subject: Re: [PATCH v11 1/1] vfio/nvgpu: Add vfio pci variant module for
 grace hopper
Message-ID: <20231010113357.GG3952@nvidia.com>
References: <20231007202254.30385-1-ankita@nvidia.com>
 <BN9PR11MB52762EE10CBBDF8AB98A53788CCDA@BN9PR11MB5276.namprd11.prod.outlook.com>
Content-Type: text/plain; charset=us-ascii
Content-Disposition: inline
In-Reply-To: <BN9PR11MB52762EE10CBBDF8AB98A53788CCDA@BN9PR11MB5276.namprd11.prod.outlook.com>
MIME-Version: 1.0
X-MS-Exchange-AntiSpam-MessageData-ChunkCount: 1
X-MS-Exchange-AntiSpam-MessageData-0: =?us-ascii?Q?ionGmYR1dHsDim95lOP9n7YMuLwX9ejAPra8aV2BnYI00mNr6fx9yIhycFBO?=
 =?us-ascii?Q?ZYOiKhFSYaaAS1UZG+dXGwYCqWULkiegNEsoJQEf+tiqbpLvyCIaG1OVDYrl?=
 =?us-ascii?Q?imd4+ZM3+P6qscqmrBRn6kgGPVZ/WqctbZE2j8jT3qaj38V5+Md1hjvtPcQG?=
 =?us-ascii?Q?zsJNNTDFBZV6ml7Y1vnA8KxsFPAPTBRcI+xbgFiyQSAKxS2i+TtlHb9cCa/2?=
 =?us-ascii?Q?th9mi2fRryBMB4rEJTlseIverliNn7IkbRUNoM7gnSD4slHCf5yZSIV4G3Qu?=
 =?us-ascii?Q?EAWp+q7wxB1z3LEFQHXr/iSzrCqc3QkX3JaJhx8jQGU7iTd4BIDzUHDpVQzh?=
 =?us-ascii?Q?sQkNdb++FMxDhw++gdRFKRJjzrlx/AjqFt3l1ridcDQupCHTUsKLuSxjxZHJ?=
 =?us-ascii?Q?MSWR8O3llJ/nxIOxUgSaOtQHARaEAAW074SV1vJDu40bxaEVJQr5ACFQeFJF?=
 =?us-ascii?Q?nzieJmzF8xJAsaPPaogrxzNzVq60erqopEGD3j6mzCWCUAT9TioG9Fn4eURM?=
 =?us-ascii?Q?9+iJ1L7N3oW6H3TZo+IjF2l51sFaEVEWgPU8UqVfU28bnXhkhVzRDGwbOqCs?=
 =?us-ascii?Q?MNwGYbQugniIw6uMmK80+UlAdbelaQC8abURnyGCE+zhARIhVWksM+WerQyB?=
 =?us-ascii?Q?v5NOqwp1+FqlLH9tKJJAalEpJILrTAU4AKETGnUNEv4s3sKSjSuZMdj2Vk/Q?=
 =?us-ascii?Q?QkFcuvBXUv/LDfdPJOnXEFnVvMYg67ZJqooy7Orh+mUcMTtXAyvKFjMUBw0v?=
 =?us-ascii?Q?+RegHBTS0w0isTK2EaouYl00bw7Pmxe7LRzcrCtqNp5JeyEPGwAo5PhBIPen?=
 =?us-ascii?Q?3I+VfUDavd9CsDpALmNSZByvWHH+xhzGnUgs8SgTbsZK9aTqRoYvJRUtU/za?=
 =?us-ascii?Q?hvwGVWTRPZKAAQ7doYFNeXnmhGN5g+5whyrWvkSLrQZ/8OiNxxEaPUv4SMsH?=
 =?us-ascii?Q?tgJSGhuvsTQjcfOPbO87g+ke8dKCTpw7VXBLdwRFw0qTzwGNPiR7oc8tUhSQ?=
 =?us-ascii?Q?1SFzL4tXhkA9nz5AmWXwKyKXm34Dl2iMga+JkR7S8llt7fFJS5rxC/DA6wPW?=
 =?us-ascii?Q?vqEI9xf2CdLnI496ZJ6sIOau1RMCfWeVelYSNweyoG+bs9na1TlNRRckQ5r9?=
 =?us-ascii?Q?pxeQHypS2shnKq3GO0pRoQnbuh02qTzBof8tI+2b99fBTAZOCv2BCu5Zdv5+?=
 =?us-ascii?Q?gk5wrYOk/AsjrczE/6ADVCco4Ud+nnaIQmwJVRMgoHBCq57hsEQqVTE4htjk?=
 =?us-ascii?Q?+eNQQYZeMlehBKJ2b9f7QKp119zMxxx4W2kk/MoSy5i4jWdMuuKpLB5f82ue?=
 =?us-ascii?Q?fPFV3OfOObFn4IkkL9NY5Z3gIbyaBlaj9M8RhmXuFeM1gZM6sq8IrCbjQzSQ?=
 =?us-ascii?Q?V5XreD45s+NRt1wd0gd4b8PvTnFqzK5Qy/oc4K4vlbVr5N01GW9drswFRjuv?=
 =?us-ascii?Q?3WU8fOynl7VPKE/ZZi3WOuGAE3UiBe3RH9aY6Bkk8mlImXlJ8jF3TFv6K0B4?=
 =?us-ascii?Q?E8e4pYiEk+s9zbq+t/UcBofHxBat0fbbK8dFnNlUGHXPc25vtn7tmtrXJ20L?=
 =?us-ascii?Q?TOt+i60SbTibdHGCa2w=3D?=
X-OriginatorOrg: Nvidia.com
X-MS-Exchange-CrossTenant-Network-Message-Id: 10d31edf-080a-40ab-0075-08dbc984cb04
X-MS-Exchange-CrossTenant-AuthSource: LV2PR12MB5869.namprd12.prod.outlook.com
X-MS-Exchange-CrossTenant-AuthAs: Internal
X-MS-Exchange-CrossTenant-OriginalArrivalTime: 10 Oct 2023 11:33:58.3742
 (UTC)
X-MS-Exchange-CrossTenant-FromEntityHeader: Hosted
X-MS-Exchange-CrossTenant-Id: 43083d15-7273-40c1-b7db-39efd9ccc17a
X-MS-Exchange-CrossTenant-MailboxType: HOSTED
X-MS-Exchange-CrossTenant-UserPrincipalName: dmfpZbT8hskH/FYT2nV6eUn3qEj1lZf9X4gMr7PgAbDPkdpIlCBxxNfSqIretaT5
X-MS-Exchange-Transport-CrossTenantHeadersStamped: SA3PR12MB8804
X-Spam-Status: No, score=2.7 required=5.0 tests=DKIMWL_WL_HIGH,DKIM_SIGNED,
	DKIM_VALID,DKIM_VALID_AU,HEADER_FROM_DIFFERENT_DOMAINS,
	MAILING_LIST_MULTI,RCVD_IN_SBL_CSS,SPF_HELO_NONE,SPF_PASS autolearn=no
	autolearn_force=no version=3.4.6
X-Spam-Checker-Version: SpamAssassin 3.4.6 (2021-04-09) on lipwig.vger.email
Precedence: bulk
List-ID: <linux-kernel.vger.kernel.org>
X-Mailing-List: linux-kernel@vger.kernel.org
X-Greylist: Sender passed SPF test, not delayed by milter-greylist-4.6.4 (lipwig.vger.email [0.0.0.0]); Tue, 10 Oct 2023 04:34:19 -0700 (PDT)
X-Spam-Level: **

On Tue, Oct 10, 2023 at 08:42:13AM +0000, Tian, Kevin wrote:
> > From: ankita@nvidia.com <ankita@nvidia.com>
> > Sent: Sunday, October 8, 2023 4:23 AM
> > 
> > PCI BAR are aligned to the power-of-2, but the actual memory on the
> > device may not. A read or write access to the physical address from the
> > last device PFN up to the next power-of-2 aligned physical address
> > results in reading ~0 and dropped writes.
> > 
> 
> my question to v10 was not answered. posted again:
> --
> Though the variant driver emulates the access to the offset beyond
> the available memory size, how does the userspace driver or the guest
> learn to know the actual size and avoid using the invalid hole to hold
> valid data?

The device FW knows and tells the VM.

Jason