From: Ankit Agrawal <[email protected]>
The NVIDIA Grace Hopper GPUs have device memory that is supposed to be
used as a regular RAM. It is accessible through CPU-GPU chip-to-chip
cache coherent interconnect and is present in the system physical
address space. The device memory is split into two regions - termed
as usemem and resmem - in the system physical address space,
with each region mapped and exposed to the VM as a separate fake
device BAR [1].
Owing to a hardware defect for Multi-Instance GPU (MIG) feature [2],
there is a requirement - as a workaround - for the resmem BAR to
display uncached memory characteristics. Based on [3], on system with
FWB enabled such as Grace Hopper, the requisite properties
(uncached, unaligned access) can be achieved through a VM mapping (S1)
of NORMAL_NC and host mapping (S2) of MT_S2_FWB_NORMAL_NC.
KVM currently maps the MMIO region in S2 as MT_S2_FWB_DEVICE_nGnRE by
default. The fake device BARs thus displays DEVICE_nGnRE behavior in the
VM.
The following table summarizes the behavior for the various S1 and S2
mapping combinations for systems with FWB enabled [3].
S1 | S2 | Result
NORMAL_NC | NORMAL_NC | NORMAL_NC
NORMAL_NC | DEVICE_nGnRE | DEVICE_nGnRE
Recently a change was added that modifies this default behavior and
make KVM map MMIO as MT_S2_FWB_NORMAL_NC when a VMA flag
VM_ALLOW_ANY_UNCACHED is set [4]. Setting S2 as MT_S2_FWB_NORMAL_NC
provides the desired behavior (uncached, unaligned access) for resmem.
To use VM_ALLOW_ANY_UNCACHED flag, the platform must guarantee that
no action taken on the MMIO mapping can trigger an uncontained
failure. The Grace Hopper satisfies this requirement. So set
the VM_ALLOW_ANY_UNCACHED flag in the VMA.
Applied over next-20240227.
base-commit: 22ba90670a51
Link: https://lore.kernel.org/all/[email protected]/ [1]
Link: https://www.nvidia.com/en-in/technologies/multi-instance-gpu/ [2]
Link: https://developer.arm.com/documentation/ddi0487/latest/ section D8.5.5 [3]
Link: https://lore.kernel.org/all/[email protected]/ [4]
Cc: Alex Williamson <[email protected]>
Cc: Kevin Tian <[email protected]>
Cc: Jason Gunthorpe <[email protected]>
Cc: Vikram Sethi <[email protected]>
Cc: Zhi Wang <[email protected]>
Signed-off-by: Ankit Agrawal <[email protected]>
---
drivers/vfio/pci/nvgrace-gpu/main.c | 11 ++++++++++-
1 file changed, 10 insertions(+), 1 deletion(-)
diff --git a/drivers/vfio/pci/nvgrace-gpu/main.c b/drivers/vfio/pci/nvgrace-gpu/main.c
index 25814006352d..a7fd018aa548 100644
--- a/drivers/vfio/pci/nvgrace-gpu/main.c
+++ b/drivers/vfio/pci/nvgrace-gpu/main.c
@@ -160,8 +160,17 @@ static int nvgrace_gpu_mmap(struct vfio_device *core_vdev,
* The carved out region of the device memory needs the NORMAL_NC
* property. Communicate as such to the hypervisor.
*/
- if (index == RESMEM_REGION_INDEX)
+ if (index == RESMEM_REGION_INDEX) {
+ /*
+ * The nvgrace-gpu module has no issues with uncontained
+ * failures on NORMAL_NC accesses. VM_ALLOW_ANY_UNCACHED is
+ * set to communicate to the KVM to S2 map as NORMAL_NC.
+ * This opens up guest usage of NORMAL_NC for this mapping.
+ */
+ vm_flags_set(vma, VM_ALLOW_ANY_UNCACHED);
+
vma->vm_page_prot = pgprot_writecombine(vma->vm_page_prot);
+ }
/*
* Perform a PFN map to the memory and back the device BAR by the
--
2.34.1
On Thu, Feb 29, 2024 at 07:39:34PM +0000, [email protected] wrote:
> From: Ankit Agrawal <[email protected]>
>
> The NVIDIA Grace Hopper GPUs have device memory that is supposed to be
> used as a regular RAM. It is accessible through CPU-GPU chip-to-chip
> cache coherent interconnect and is present in the system physical
> address space. The device memory is split into two regions - termed
> as usemem and resmem - in the system physical address space,
> with each region mapped and exposed to the VM as a separate fake
> device BAR [1].
>
> Owing to a hardware defect for Multi-Instance GPU (MIG) feature [2],
> there is a requirement - as a workaround - for the resmem BAR to
> display uncached memory characteristics. Based on [3], on system with
> FWB enabled such as Grace Hopper, the requisite properties
> (uncached, unaligned access) can be achieved through a VM mapping (S1)
> of NORMAL_NC and host mapping (S2) of MT_S2_FWB_NORMAL_NC.
>
> KVM currently maps the MMIO region in S2 as MT_S2_FWB_DEVICE_nGnRE by
> default. The fake device BARs thus displays DEVICE_nGnRE behavior in the
> VM.
>
> The following table summarizes the behavior for the various S1 and S2
> mapping combinations for systems with FWB enabled [3].
> S1 | S2 | Result
> NORMAL_NC | NORMAL_NC | NORMAL_NC
> NORMAL_NC | DEVICE_nGnRE | DEVICE_nGnRE
>
> Recently a change was added that modifies this default behavior and
> make KVM map MMIO as MT_S2_FWB_NORMAL_NC when a VMA flag
> VM_ALLOW_ANY_UNCACHED is set [4]. Setting S2 as MT_S2_FWB_NORMAL_NC
> provides the desired behavior (uncached, unaligned access) for resmem.
>
> To use VM_ALLOW_ANY_UNCACHED flag, the platform must guarantee that
> no action taken on the MMIO mapping can trigger an uncontained
> failure. The Grace Hopper satisfies this requirement. So set
> the VM_ALLOW_ANY_UNCACHED flag in the VMA.
>
> Applied over next-20240227.
> base-commit: 22ba90670a51
>
> Link: https://lore.kernel.org/all/[email protected]/ [1]
> Link: https://www.nvidia.com/en-in/technologies/multi-instance-gpu/ [2]
> Link: https://developer.arm.com/documentation/ddi0487/latest/ section D8.5.5 [3]
> Link: https://lore.kernel.org/all/[email protected]/ [4]
>
> Cc: Alex Williamson <[email protected]>
> Cc: Kevin Tian <[email protected]>
> Cc: Jason Gunthorpe <[email protected]>
> Cc: Vikram Sethi <[email protected]>
> Cc: Zhi Wang <[email protected]>
> Signed-off-by: Ankit Agrawal <[email protected]>
> ---
> drivers/vfio/pci/nvgrace-gpu/main.c | 11 ++++++++++-
> 1 file changed, 10 insertions(+), 1 deletion(-)
Reviewed-by: Jason Gunthorpe <[email protected]>
Jason
On Thu, 29 Feb 2024 19:39:34 +0000
<[email protected]> wrote:
> From: Ankit Agrawal <[email protected]>
>
> The NVIDIA Grace Hopper GPUs have device memory that is supposed to be
> used as a regular RAM. It is accessible through CPU-GPU chip-to-chip
> cache coherent interconnect and is present in the system physical
> address space. The device memory is split into two regions - termed
> as usemem and resmem - in the system physical address space,
> with each region mapped and exposed to the VM as a separate fake
> device BAR [1].
>
> Owing to a hardware defect for Multi-Instance GPU (MIG) feature [2],
> there is a requirement - as a workaround - for the resmem BAR to
> display uncached memory characteristics. Based on [3], on system with
> FWB enabled such as Grace Hopper, the requisite properties
> (uncached, unaligned access) can be achieved through a VM mapping (S1)
> of NORMAL_NC and host mapping (S2) of MT_S2_FWB_NORMAL_NC.
>
> KVM currently maps the MMIO region in S2 as MT_S2_FWB_DEVICE_nGnRE by
> default. The fake device BARs thus displays DEVICE_nGnRE behavior in the
> VM.
>
> The following table summarizes the behavior for the various S1 and S2
> mapping combinations for systems with FWB enabled [3].
> S1 | S2 | Result
> NORMAL_NC | NORMAL_NC | NORMAL_NC
> NORMAL_NC | DEVICE_nGnRE | DEVICE_nGnRE
>
> Recently a change was added that modifies this default behavior and
> make KVM map MMIO as MT_S2_FWB_NORMAL_NC when a VMA flag
> VM_ALLOW_ANY_UNCACHED is set [4]. Setting S2 as MT_S2_FWB_NORMAL_NC
> provides the desired behavior (uncached, unaligned access) for resmem.
>
> To use VM_ALLOW_ANY_UNCACHED flag, the platform must guarantee that
> no action taken on the MMIO mapping can trigger an uncontained
> failure. The Grace Hopper satisfies this requirement. So set
> the VM_ALLOW_ANY_UNCACHED flag in the VMA.
>
> Applied over next-20240227.
> base-commit: 22ba90670a51
>
> Link: https://lore.kernel.org/all/[email protected]/ [1]
> Link: https://www.nvidia.com/en-in/technologies/multi-instance-gpu/ [2]
> Link: https://developer.arm.com/documentation/ddi0487/latest/ section D8.5.5 [3]
> Link: https://lore.kernel.org/all/[email protected]/ [4]
>
> Cc: Alex Williamson <[email protected]>
> Cc: Kevin Tian <[email protected]>
> Cc: Jason Gunthorpe <[email protected]>
> Cc: Vikram Sethi <[email protected]>
> Cc: Zhi Wang <[email protected]>
> Signed-off-by: Ankit Agrawal <[email protected]>
> ---
> drivers/vfio/pci/nvgrace-gpu/main.c | 11 ++++++++++-
> 1 file changed, 10 insertions(+), 1 deletion(-)
Applied to vfio next branch for v6.9.
Oliver, FYI I did merge the branch you provided in [1] for this, thanks
for the foresight in providing that.
Alex
[1]https://lore.kernel.org/all/[email protected]/
>
> diff --git a/drivers/vfio/pci/nvgrace-gpu/main.c b/drivers/vfio/pci/nvgrace-gpu/main.c
> index 25814006352d..a7fd018aa548 100644
> --- a/drivers/vfio/pci/nvgrace-gpu/main.c
> +++ b/drivers/vfio/pci/nvgrace-gpu/main.c
> @@ -160,8 +160,17 @@ static int nvgrace_gpu_mmap(struct vfio_device *core_vdev,
> * The carved out region of the device memory needs the NORMAL_NC
> * property. Communicate as such to the hypervisor.
> */
> - if (index == RESMEM_REGION_INDEX)
> + if (index == RESMEM_REGION_INDEX) {
> + /*
> + * The nvgrace-gpu module has no issues with uncontained
> + * failures on NORMAL_NC accesses. VM_ALLOW_ANY_UNCACHED is
> + * set to communicate to the KVM to S2 map as NORMAL_NC.
> + * This opens up guest usage of NORMAL_NC for this mapping.
> + */
> + vm_flags_set(vma, VM_ALLOW_ANY_UNCACHED);
> +
> vma->vm_page_prot = pgprot_writecombine(vma->vm_page_prot);
> + }
>
> /*
> * Perform a PFN map to the memory and back the device BAR by the
>> To use VM_ALLOW_ANY_UNCACHED flag, the platform must guarantee that
>> no action taken on the MMIO mapping can trigger an uncontained
>> failure. The Grace Hopper satisfies this requirement. So set
>> the VM_ALLOW_ANY_UNCACHED flag in the VMA.
>>
>> Applied over next-20240227.
>> base-commit: 22ba90670a51
>>
>> Link: https://lore.kernel.org/all/[email protected]/?[1]
>> Link: https://www.nvidia.com/en-in/technologies/multi-instance-gpu/?[2]
>> Link: https://developer.arm.com/documentation/ddi0487/latest/?section D8.5.5 [3]
>> Link: https://lore.kernel.org/all/[email protected]/?[4]
>>
>> Cc: Alex Williamson <[email protected]>
>> Cc: Kevin Tian <[email protected]>
>> Cc: Jason Gunthorpe <[email protected]>
>> Cc: Vikram Sethi <[email protected]>
>> Cc: Zhi Wang <[email protected]>
>> Signed-off-by: Ankit Agrawal <[email protected]>
>> ---
>>? drivers/vfio/pci/nvgrace-gpu/main.c | 11 ++++++++++-
>>? 1 file changed, 10 insertions(+), 1 deletion(-)
>
> Applied to vfio next branch for v6.9.
>
> Oliver, FYI I did merge the branch you provided in [1] for this, thanks
> for the foresight in providing that.
>
> Alex
>
> [1]https://lore.kernel.org/all/[email protected]/
Many thanks Alex, appreciate your help with this!