Received-SPF: pass (google.com: domain of linux-kernel-owner@vger.kernel.org designates 2620:137:e000::1:20 as permitted sender) client-ip=2620:137:e000::1:20;
Date:   Wed, 28 Dec 2022 13:42:37 +0800
From:   Yan Zhao <yan.y.zhao@intel.com>
To:     Sean Christopherson <seanjc@google.com>
CC:     Paolo Bonzini <pbonzini@redhat.com>,
        Zhenyu Wang <zhenyuw@linux.intel.com>,
        Zhi Wang <zhi.a.wang@intel.com>, <kvm@vger.kernel.org>,
        <intel-gvt-dev@lists.freedesktop.org>,
        <intel-gfx@lists.freedesktop.org>, <linux-kernel@vger.kernel.org>,
        Ben Gardon <bgardon@google.com>
Subject: Re: [PATCH 03/27] drm/i915/gvt: Incorporate KVM memslot info into
 check for 2MiB GTT entry
Message-ID: <Y6vXTcxDNovrmeVB@yzhao56-desk.sh.intel.com>
Reply-To: Yan Zhao <yan.y.zhao@intel.com>
References: <20221223005739.1295925-1-seanjc@google.com>
 <20221223005739.1295925-4-seanjc@google.com>
Content-Type: text/plain; charset="us-ascii"
Content-Disposition: inline
In-Reply-To: <20221223005739.1295925-4-seanjc@google.com>
MIME-Version: 1.0
X-MS-Exchange-AntiSpam-MessageData-ChunkCount: 1
X-MS-Exchange-AntiSpam-MessageData-0: =?us-ascii?Q?zGUzR0n/127hK3CfNyyWtrTXf67bYQYzyXqr/rRC66tnFxnZfVpeykMZX/My?=
 =?us-ascii?Q?lh16dwiQQgDqUT578YDs+f0aO7HvNslOT4EdMMlb00O5h9Up53Kk8sQi04nx?=
 =?us-ascii?Q?WFAJR0vSK2W9Efy5Ui7jZettWOeNltWnYqXigFKaMqFyxQpgIa406YkDGWYG?=
 =?us-ascii?Q?+hoF5PfbzKt5d5izOWms2KKtmYj/eNS0OflcG23LSUOC0Q/aFxDdhCGZ8ItK?=
 =?us-ascii?Q?KWcTWBAmXvvZSih3JFxu48O94Yp3RGGARdf/bscfaSuJwJ9i203zFNWucl2Y?=
 =?us-ascii?Q?L279hQJvJCP6m8U+Gu7SvbUoBuXOpbzV6IrnJ4LhZNOkEJMBAcc+F6nYph8N?=
 =?us-ascii?Q?uf8xw5pSfDAZ28bfVOGsJC6MGmKAnoKVOtZCp4midtU2T4wCqaOgTfhTkE6a?=
 =?us-ascii?Q?Vtv9tNd9D0wrQ/lrppCVGOMr0NNKIYSJaun2jBMgillo07ziab2FzcRlmvE6?=
 =?us-ascii?Q?iujHcu8epYWyTT5HaJJQCHYWMejGo8k/4UEc//OeBg6OboNwAoHrWAI35c19?=
 =?us-ascii?Q?wuBET3EtHRROIzCAcOV3Ll8P0B+4YI1axqtoa1azoc1MVmq9h/Hl+XTNbIxs?=
 =?us-ascii?Q?E9KHyOZOEyTn8ZL+hX9dMYx7RWGzMNZR4JgMNpcr6j2QGyE4C5gjLYy5zAtD?=
 =?us-ascii?Q?nEasS++RSheo/XDAxe9vb0B4YeJDj6RNR/3S8P4gg/l40ZA05l3rdeOEhjLb?=
 =?us-ascii?Q?uPRQMPbDcibw8JxTJyilVg058WSJpyk3msRXkBxahJHOMjyOyRiu/awaC4n5?=
 =?us-ascii?Q?9NJuQ8dNbhDr28B/ouk489F1iNhpVrxAYp5oOw7dlY36eyauaE2nqQWDC4O9?=
 =?us-ascii?Q?0VH+u39ZjZxZjdpWcn6fCxI6L1/0BWcwMljXbL/PfiZ0vmFybZMi9csx629S?=
 =?us-ascii?Q?rz0OztLY7pj2O+TR1I5TCKiaMqicu35FFFSnkfVjxSRXPYcfQdtGVdaRjZmm?=
 =?us-ascii?Q?D3gb4y/MKpVtcdFkTod6j0bLx03xnycJKYuXNb6jh6I3KRNrv7xwZ3b4bcDX?=
 =?us-ascii?Q?H9MI4pKvukrZhzYZYrq+FhSW5mECi7ZvjLvAANOwI1WDga69ov9nwJMCdclX?=
 =?us-ascii?Q?Ikgcqfybk/Cw2gMrIIhjfw+TJzJxWWYzlZSA5Rcp2rpThvuBk44eyEnSdE57?=
 =?us-ascii?Q?rWqcFJ7KM8LGsd0vHRSUGk7QON0qFSglCSqv10uU5Lv8RtGm/uqmL6vFB347?=
 =?us-ascii?Q?Skqi3GtxgRN4LgVmM4KMgqYff3WQ+1qU5ySVPKBFCHoaVWlJYqfsN9TVfRfk?=
 =?us-ascii?Q?ugrP2plWP/5vUDE+BGjlOMr6BQ79DRL8HYptSE51xfQ9nSX+BTdHmAks0Gcw?=
 =?us-ascii?Q?spkt9YgfrU1EXV22ZgmHqkDqF9tqnOLrwCj3C93J/y2UO76wYhNwvTvuwyjI?=
 =?us-ascii?Q?sD23yz4QGd4UtQy06uPUtSY/seMUhCN+HARC+6CoyZdHUCsqq7wn49rYwU9B?=
 =?us-ascii?Q?D8Oq9hLG81NvRGxIA25waKYwf+clhxw2bt8BsnqQO4x/IULVR6gNua8+y+K0?=
 =?us-ascii?Q?+OSbCq6ZIqyCQgz4Nn086FuDPn5cONjIS6/BWDAhollWC6Lel5oRIUle3/Io?=
 =?us-ascii?Q?X0nrnGSk6axif2nE08Lcy7u/haHzoXrBB8RsDrk5?=
X-MS-Exchange-CrossTenant-Network-Message-Id: 864a10fc-fc51-4d7c-6f8c-08dae89993a8
X-MS-Exchange-CrossTenant-AuthSource: DS7PR11MB5966.namprd11.prod.outlook.com
X-MS-Exchange-CrossTenant-AuthAs: Internal
X-MS-Exchange-CrossTenant-OriginalArrivalTime: 28 Dec 2022 06:05:53.5101
 (UTC)
X-MS-Exchange-CrossTenant-FromEntityHeader: Hosted
X-MS-Exchange-CrossTenant-Id: 46c98d88-e344-4ed4-8496-4ed7712e255d
X-MS-Exchange-CrossTenant-MailboxType: HOSTED
X-MS-Exchange-CrossTenant-UserPrincipalName: wpqUOU1F6T5PrNPZX7/Xew4EPl8rwRE/+whrYrZHHtEPNFo6st0VjRUB21Lof8JWwZmIEU8KnkrvNV+DJdBX/w==
X-MS-Exchange-Transport-CrossTenantHeadersStamped: DM6PR11MB4690
X-OriginatorOrg: intel.com
X-Spam-Status: No, score=-7.1 required=5.0 tests=BAYES_00,DKIMWL_WL_HIGH,
        DKIM_SIGNED,DKIM_VALID,DKIM_VALID_AU,DKIM_VALID_EF,RCVD_IN_DNSWL_HI,
        SPF_HELO_NONE,SPF_NONE autolearn=ham autolearn_force=no version=3.4.6
X-Spam-Checker-Version: SpamAssassin 3.4.6 (2021-04-09) on
        lindbergh.monkeyblade.net
Precedence: bulk
List-ID: <linux-kernel.vger.kernel.org>
X-Mailing-List: linux-kernel@vger.kernel.org

On Fri, Dec 23, 2022 at 12:57:15AM +0000, Sean Christopherson wrote:
> Honor KVM's max allowed page size when determining whether or not a 2MiB
> GTT shadow page can be created for the guest.  Querying KVM's max allowed
> size is somewhat odd as there's no strict requirement that KVM's memslots
> and VFIO's mappings are configured with the same gfn=>hva mapping, but
Without vIOMMU, VFIO's mapping is configured with the same as KVM's
memslots, i.e. with the same gfn==>HVA mapping


> the check will be accurate if userspace wants to have a functional guest,
> and at the very least checking KVM's memslots guarantees that the entire
> 2MiB range has been exposed to the guest.

I think just check the entrie 2MiB GFN range are all within KVM memslot is
enough.
If for some reason, KVM maps a 2MiB range in 4K sizes, KVMGT can still map
it in IOMMU size in 2MiB size as long as the PFNs are continous and the
whole range is all exposed to guest.
Actually normal device passthrough with VFIO-PCI also maps GFNs in a
similar way, i.e. maps a guest visible range in as large size as
possible as long as the PFN is continous. 
> 
> Note, KVM may also restrict the mapping size for reasons that aren't
> relevant to KVMGT, e.g. for KVM's iTLB multi-hit workaround or if the gfn
Will iTLB multi-hit affect DMA?
AFAIK, IOMMU mappings currently never sets exec bit (and I'm told this bit is
under discussion to be removed).


> is write-tracked (KVM's write-tracking only handles writes from vCPUs).
> However, such scenarios are unlikely to occur with a well-behaved guest,
> and at worst will result in sub-optimal performance.
> Fixes: b901b252b6cf ("drm/i915/gvt: Add 2M huge gtt support")
> Signed-off-by: Sean Christopherson <seanjc@google.com>
> ---
>  arch/x86/include/asm/kvm_page_track.h |  2 ++
>  arch/x86/kvm/mmu/page_track.c         | 18 ++++++++++++++++++
>  drivers/gpu/drm/i915/gvt/gtt.c        | 10 +++++++++-
>  3 files changed, 29 insertions(+), 1 deletion(-)
> 
> diff --git a/arch/x86/include/asm/kvm_page_track.h b/arch/x86/include/asm/kvm_page_track.h
> index eb186bc57f6a..3f72c7a172fc 100644
> --- a/arch/x86/include/asm/kvm_page_track.h
> +++ b/arch/x86/include/asm/kvm_page_track.h
> @@ -51,6 +51,8 @@ void kvm_page_track_cleanup(struct kvm *kvm);
>  
>  bool kvm_page_track_write_tracking_enabled(struct kvm *kvm);
>  int kvm_page_track_write_tracking_alloc(struct kvm_memory_slot *slot);
> +enum pg_level kvm_page_track_max_mapping_level(struct kvm *kvm, gfn_t gfn,
> +					       enum pg_level max_level);
>  
>  void kvm_page_track_free_memslot(struct kvm_memory_slot *slot);
>  int kvm_page_track_create_memslot(struct kvm *kvm,
> diff --git a/arch/x86/kvm/mmu/page_track.c b/arch/x86/kvm/mmu/page_track.c
> index 2e09d1b6249f..69ea16c31859 100644
> --- a/arch/x86/kvm/mmu/page_track.c
> +++ b/arch/x86/kvm/mmu/page_track.c
> @@ -300,3 +300,21 @@ void kvm_page_track_flush_slot(struct kvm *kvm, struct kvm_memory_slot *slot)
>  			n->track_flush_slot(kvm, slot, n);
>  	srcu_read_unlock(&head->track_srcu, idx);
>  }
> +
> +enum pg_level kvm_page_track_max_mapping_level(struct kvm *kvm, gfn_t gfn,
> +					       enum pg_level max_level)
> +{
> +	struct kvm_memory_slot *slot;
> +	int idx;
> +
> +	idx = srcu_read_lock(&kvm->srcu);
> +	slot = gfn_to_memslot(kvm, gfn);
> +	if (!slot || slot->flags & KVM_MEMSLOT_INVALID)
> +		max_level = PG_LEVEL_4K;
> +	else
> +		max_level = kvm_mmu_max_slot_mapping_level(slot, gfn, max_level);
> +	srcu_read_unlock(&kvm->srcu, idx);
> +
> +	return max_level;
> +}
> +EXPORT_SYMBOL_GPL(kvm_page_track_max_mapping_level);
> diff --git a/drivers/gpu/drm/i915/gvt/gtt.c b/drivers/gpu/drm/i915/gvt/gtt.c
> index d0fca53a3563..6736d7bd94ea 100644
> --- a/drivers/gpu/drm/i915/gvt/gtt.c
> +++ b/drivers/gpu/drm/i915/gvt/gtt.c
> @@ -1178,14 +1178,22 @@ static int is_2MB_gtt_possible(struct intel_vgpu *vgpu,
>  	struct intel_gvt_gtt_entry *entry)
>  {
>  	const struct intel_gvt_gtt_pte_ops *ops = vgpu->gvt->gtt.pte_ops;
> +	unsigned long gfn = ops->get_pfn(entry);
>  	kvm_pfn_t pfn;
> +	int max_level;
>  
>  	if (!HAS_PAGE_SIZES(vgpu->gvt->gt->i915, I915_GTT_PAGE_SIZE_2M))
>  		return 0;
>  
>  	if (!vgpu->attached)
>  		return -EINVAL;
> -	pfn = gfn_to_pfn(vgpu->vfio_device.kvm, ops->get_pfn(entry));
> +
> +	max_level = kvm_page_track_max_mapping_level(vgpu->vfio_device.kvm,
> +						     gfn, PG_LEVEL_2M);
> +	if (max_level < PG_LEVEL_2M)
> +		return 0;
> +
> +	pfn = gfn_to_pfn(vgpu->vfio_device.kvm, gfn);
>  	if (is_error_noslot_pfn(pfn))
>  		return -EINVAL;
>  
> -- 
> 2.39.0.314.g84b9a713c41-goog
>