Received-SPF: pass (google.com: domain of linux-kernel-owner@vger.kernel.org designates 2620:137:e000::1:20 as permitted sender) client-ip=2620:137:e000::1:20;
Date:   Thu, 30 Mar 2023 21:18:00 -0700
From:   Ira Weiny <ira.weiny@intel.com>
To:     Zhao Liu <zhao1.liu@linux.intel.com>,
        Jani Nikula <jani.nikula@linux.intel.com>,
        Joonas Lahtinen <joonas.lahtinen@linux.intel.com>,
        Rodrigo Vivi <rodrigo.vivi@intel.com>,
        Tvrtko Ursulin <tvrtko.ursulin@linux.intel.com>,
        David Airlie <airlied@gmail.com>,
        Daniel Vetter <daniel@ffwll.ch>,
        Matthew Auld <matthew.auld@intel.com>,
        Thomas =?iso-8859-1?Q?Hellstr=F6m?= 
        <thomas.hellstrom@linux.intel.com>,
        Nirmoy Das <nirmoy.das@intel.com>,
        Maarten Lankhorst <maarten.lankhorst@linux.intel.com>,
        Chris Wilson <chris@chris-wilson.co.uk>,
        Christian =?iso-8859-1?Q?K=F6nig?= <christian.koenig@amd.com>,
        <intel-gfx@lists.freedesktop.org>,
        <dri-devel@lists.freedesktop.org>, <linux-kernel@vger.kernel.org>
CC:     Ira Weiny <ira.weiny@intel.com>,
        "Fabio M . De Francesco" <fmdefrancesco@gmail.com>,
        Zhenyu Wang <zhenyu.z.wang@intel.com>,
        Zhao Liu <zhao1.liu@intel.com>
Subject: Re: [PATCH v2 9/9] drm/i915: Use kmap_local_page() in
 gem/i915_gem_execbuffer.c
Message-ID: <64265ef8725fe_375f7e294a@iweiny-mobl.notmuch>
References: <20230329073220.3982460-1-zhao1.liu@linux.intel.com>
 <20230329073220.3982460-10-zhao1.liu@linux.intel.com>
Content-Type: text/plain; charset="us-ascii"
Content-Disposition: inline
In-Reply-To: <20230329073220.3982460-10-zhao1.liu@linux.intel.com>
MIME-Version: 1.0
X-MS-Exchange-AntiSpam-MessageData-ChunkCount: 1
X-MS-Exchange-AntiSpam-MessageData-0: =?us-ascii?Q?kHb9TR0bMnSO12teK+Bz0IoMR978KLplOl2/WcNRJc9jFzWJ7oBPcw03By4t?=
 =?us-ascii?Q?PUpizDFOmMJ9tvAVfj+oWbsbDrsrWeMjvnHBjl936tQwuaZxX4YFWXQdjMKw?=
 =?us-ascii?Q?uTYaF2pT81GR7IJvms6cGut3zX0bipjDmtjFKBJ5jswJGUF7nWI3JPtOQxU5?=
 =?us-ascii?Q?eMjAEKmYDsSm8D/Ez3f2pFHGWJRMb9oBH5BPyr9+oylQCWLQHlRLReCHBc44?=
 =?us-ascii?Q?yuvmALdXe96ZRwk5caLQtDwdikc7x8Px1y4+Sa46uBlnnvQZwYLxYNWl8uvX?=
 =?us-ascii?Q?DJAcu4YvSp9WxxIjeyqXAtw/Wmbr9XHsExZSGN7685svcFlegxFXEu/jvqBb?=
 =?us-ascii?Q?RWmtCwEt0J8VsCQ/n1hqUYAqzGji5m3Oq78Fmw75cJrF3j3ZDd/jEjcGABJo?=
 =?us-ascii?Q?TTVmEW4CPjz28jcXVSS3Dp2NmDFigDlA/WWhNsDfWuO8+qWB0gXTJxPjD67h?=
 =?us-ascii?Q?hwkRmsdX+kPY2B6t1uAoR85Ot1N1vOOe8evJ9N6wQthGJhfWOgaZGTTcP0nh?=
 =?us-ascii?Q?v3Eody7GeH9AbiZJaknxhOteWlacFKY5UXOZYxBY0uo5GU6zWfByeXkjL8eX?=
 =?us-ascii?Q?074k3nWVB+471MEIKaYzSF4VVevymlqOpc9ov5bxOHWhqrmxgvJ/a53P4yVo?=
 =?us-ascii?Q?FNZ+WOyEc4FdGBbf5SbAOCvivPGnTkUMcPysDF/s6vhFnqUo8yWRMa3WqynJ?=
 =?us-ascii?Q?DQQB1n6afjlYFDxcocJDnAxKjl7ZNQKKrDZ24QKvCAPkTMgWNE7MDrzIsmj3?=
 =?us-ascii?Q?RhGO9++tB5ppWZ7Y822ZJOFny9m87DLKhM8AbIcSBcxuTHlwUW0bmq2jVKqh?=
 =?us-ascii?Q?svTmkUfcV6MjTTuNweFSHc8RhRshMZnUR+0k8us08RWcZSiB7XqRRYC+82Ou?=
 =?us-ascii?Q?ZqGZctlsUNxUSzokgZCwF+teT4bF4K5K370Q/ef6bnGSRlKgDACqMEkayANG?=
 =?us-ascii?Q?C2JV1RIoa7NtvP6KkUlMXKM8qdpYs3TtMlYTiBF8Qgnf4EvL5Biznkeqj6rt?=
 =?us-ascii?Q?IsFTcRfTQg+WnsFV33g54aBs5Hfl9VzhbXq1VYQEBd9u9sLpGCphVOaW2nNM?=
 =?us-ascii?Q?gBbDRBGLFjgMyQKHFy9SEmR7MCOVgsYvycH8sN71JODpGvrIdsp8COwuyy4e?=
 =?us-ascii?Q?yC3jD9qoRUlH48UIf/Tc5QxW63conLNhy+is1cZqjNbOsmX+bEoopzLcHjtM?=
 =?us-ascii?Q?JIzH9y6GH4ryOqwZhiEsa/PLi4bt+y3K1X9njrKulEi7e4sLERjkUJhpLkYi?=
 =?us-ascii?Q?yLMzsMKCsM5khTpOEnbGDDpzzcCSeyAuhodekBORU/WJekUG8DASKd6CzXUp?=
 =?us-ascii?Q?faLMnxjCkUjNTu4wfSTN2lSupQsVjMBnHdOSWVB7WOWEGfTr8oRya/ouO1rX?=
 =?us-ascii?Q?5nC6J2l54j00wiSa0MIDli7uIoRdXruNp0skFdT/oJWOJO4hin3cwzguRxe2?=
 =?us-ascii?Q?Q/QiaFOTq4ySrXxjl/aR61DfVWt0EQDby7fN3fo/kyyMZkTqjGa0xpvfFMo1?=
 =?us-ascii?Q?5yIHJey95DL/5tJPtevFt4mFW8YAN+Sfp/1JgTgmADoWYlmn0DVC1pXrMvZY?=
 =?us-ascii?Q?Nf3hCyoR8HpyKSlqIQ1AsBi6bNL2xIvut/awumBK?=
X-MS-Exchange-CrossTenant-Network-Message-Id: ac2573e3-15a6-45cd-a3ce-08db319eece9
X-MS-Exchange-CrossTenant-AuthSource: SA1PR11MB6733.namprd11.prod.outlook.com
X-MS-Exchange-CrossTenant-AuthAs: Internal
X-MS-Exchange-CrossTenant-OriginalArrivalTime: 31 Mar 2023 04:18:05.5729
 (UTC)
X-MS-Exchange-CrossTenant-FromEntityHeader: Hosted
X-MS-Exchange-CrossTenant-Id: 46c98d88-e344-4ed4-8496-4ed7712e255d
X-MS-Exchange-CrossTenant-MailboxType: HOSTED
X-MS-Exchange-CrossTenant-UserPrincipalName: KeaMWVqt58v4ajfJbeibqK8Cwc7yUrjWsLtPIN5Z+vW2o0yX8ac7o/mQIrX7CONAjqt0ptKfAPpNJ4v4dRDHfA==
X-MS-Exchange-Transport-CrossTenantHeadersStamped: MW4PR11MB7006
X-OriginatorOrg: intel.com
X-Spam-Status: No, score=-2.5 required=5.0 tests=DKIMWL_WL_HIGH,DKIM_SIGNED,
        DKIM_VALID,DKIM_VALID_AU,DKIM_VALID_EF,RCVD_IN_DNSWL_MED,SPF_HELO_NONE,
        SPF_NONE autolearn=unavailable autolearn_force=no version=3.4.6
X-Spam-Checker-Version: SpamAssassin 3.4.6 (2021-04-09) on
        lindbergh.monkeyblade.net
Precedence: bulk
List-ID: <linux-kernel.vger.kernel.org>
X-Mailing-List: linux-kernel@vger.kernel.org

Zhao Liu wrote:
> From: Zhao Liu <zhao1.liu@intel.com>
> 
> The use of kmap_atomic() is being deprecated in favor of
> kmap_local_page()[1], and this patch converts the calls from
> kmap_atomic() to kmap_local_page().
> 
> The main difference between atomic and local mappings is that local
> mappings doesn't disable page faults or preemption (the preemption is
> disabled for !PREEMPT_RT case, otherwise it only disables migration).
> 
> With kmap_local_page(), we can avoid the often unwanted side effect of
> unnecessary page faults and preemption disables.
> 
> In i915_gem_execbuffer.c, eb->reloc_cache.vaddr is mapped by
> kmap_atomic() in eb_relocate_entry(), and is unmapped by
> kunmap_atomic() in reloc_cache_reset().

First off thanks for the series and sticking with this.  That said this
patch kind of threw me for a loop because tracing the map/unmap calls did
not make sense to me.  See below.

> 
> And this mapping/unmapping occurs in two places: one is in
> eb_relocate_vma(), and another is in eb_relocate_vma_slow().
> 
> The function eb_relocate_vma() or eb_relocate_vma_slow() doesn't
> need to disable pagefaults and preemption during the above mapping/
> unmapping.
> 
> So it can simply use kmap_local_page() / kunmap_local() that can
> instead do the mapping / unmapping regardless of the context.
> 
> Convert the calls of kmap_atomic() / kunmap_atomic() to
> kmap_local_page() / kunmap_local().
> 
> [1]: https://lore.kernel.org/all/20220813220034.806698-1-ira.weiny@intel.com
> 
> v2: No code change since v1. Added description of the motivation of
>     using kmap_local_page() and "Suggested-by" tag of Fabio.
> 
> Suggested-by: Ira Weiny <ira.weiny@intel.com>
> Suggested-by: Fabio M. De Francesco <fmdefrancesco@gmail.com>
> Signed-off-by: Zhao Liu <zhao1.liu@intel.com>
> ---
> Suggested by credits:
>   Ira: Referred to his task document, review comments.
>   Fabio: Referred to his boiler plate commit message and his description
>          about why kmap_local_page() should be preferred.
> ---
>  drivers/gpu/drm/i915/gem/i915_gem_execbuffer.c | 10 +++++-----
>  1 file changed, 5 insertions(+), 5 deletions(-)
> 
> diff --git a/drivers/gpu/drm/i915/gem/i915_gem_execbuffer.c b/drivers/gpu/drm/i915/gem/i915_gem_execbuffer.c
> index 9dce2957b4e5..805565edd148 100644
> --- a/drivers/gpu/drm/i915/gem/i915_gem_execbuffer.c
> +++ b/drivers/gpu/drm/i915/gem/i915_gem_execbuffer.c
> @@ -1151,7 +1151,7 @@ static void reloc_cache_unmap(struct reloc_cache *cache)
>  
>  	vaddr = unmask_page(cache->vaddr);
>  	if (cache->vaddr & KMAP)
> -		kunmap_atomic(vaddr);
> +		kunmap_local(vaddr);

In the cover letter you don't mention this unmap path.  Rather you mention
only reloc_cache_reset().

After digging into this and considering these are kmap_atomic() calls I
_think_ what you have is ok.  But I think I'd like to see the call paths
documented a bit more clearly.  Or perhaps cleaned up a lot.

For example I see the following call possibility from a user ioctl.  In
this trace I see 2 examples where something is unmapped first.  I don't
understand why that is required?  I would assume reloc_cache_unmap() and
reloc_kmap() are helpers called from somewhere else requiring a remapping
of the cache but I don't see it.

i915_gem_execbuffer2_ioctl()
eb_relocate_parse()
eb_relocate_parse_slow()
eb_relocate_vma_slow()
	eb_relocate_entry()
		reloc_cache_unmap()
			kunmap_atomic()  <=== HERE!
		reloc_cache_remap()
			kmap_atomic()
		relocate_entry()
			reloc_vaddr()
				reloc_kmap()
					kunmap_atomic() <== HERE!
					kmap_atomic()

	reloc_cache_reset()
		kunmap_atomic()

Could these mappings be cleaned up a lot more?  Perhaps by removing some
of the helper functions which AFAICT are left over from older versions of
the code?

Also as an aside I think it is really bad that eb_relocate_entry() returns
negative errors in a u64.  Better to get the types right IMO.

Thanks for the series!
Ira

>  	else
>  		io_mapping_unmap_atomic((void __iomem *)vaddr);
>  }
> @@ -1167,7 +1167,7 @@ static void reloc_cache_remap(struct reloc_cache *cache,
>  	if (cache->vaddr & KMAP) {
>  		struct page *page = i915_gem_object_get_page(obj, cache->page);
>  
> -		vaddr = kmap_atomic(page);
> +		vaddr = kmap_local_page(page);
>  		cache->vaddr = unmask_flags(cache->vaddr) |
>  			(unsigned long)vaddr;
>  	} else {
> @@ -1197,7 +1197,7 @@ static void reloc_cache_reset(struct reloc_cache *cache, struct i915_execbuffer
>  		if (cache->vaddr & CLFLUSH_AFTER)
>  			mb();
>  
> -		kunmap_atomic(vaddr);
> +		kunmap_local(vaddr);
>  		i915_gem_object_finish_access(obj);
>  	} else {
>  		struct i915_ggtt *ggtt = cache_to_ggtt(cache);
> @@ -1229,7 +1229,7 @@ static void *reloc_kmap(struct drm_i915_gem_object *obj,
>  	struct page *page;
>  
>  	if (cache->vaddr) {
> -		kunmap_atomic(unmask_page(cache->vaddr));
> +		kunmap_local(unmask_page(cache->vaddr));
>  	} else {
>  		unsigned int flushes;
>  		int err;
> @@ -1251,7 +1251,7 @@ static void *reloc_kmap(struct drm_i915_gem_object *obj,
>  	if (!obj->mm.dirty)
>  		set_page_dirty(page);
>  
> -	vaddr = kmap_atomic(page);
> +	vaddr = kmap_local_page(page);
>  	cache->vaddr = unmask_flags(cache->vaddr) | (unsigned long)vaddr;
>  	cache->page = pageno;
>  
> -- 
> 2.34.1
>