Received: by 2002:a05:6a10:eb17:0:0:0:0 with SMTP id hx23csp726278pxb; Wed, 8 Sep 2021 10:50:17 -0700 (PDT) X-Google-Smtp-Source: ABdhPJw7uUr+sIpfGNjv4xECzpYvjn+e3Y+auePJTv0ZTzbjpVyx0fsbaPEPppGEr8STCbaZj8Q4 X-Received: by 2002:a05:6402:caa:: with SMTP id cn10mr4984209edb.202.1631123416776; Wed, 08 Sep 2021 10:50:16 -0700 (PDT) ARC-Seal: i=1; a=rsa-sha256; t=1631123416; cv=none; d=google.com; s=arc-20160816; b=VBqA68AxWe+DhO/w4VCopjyi9ZsLEDpaGXOoBO8D3Sg1gOebaxwUZ0j7CDuCk4FN7/ dQHn2KOL353XLlNpyIGXi/d0C+vRGJVORgl6JqDFCf29WclrnYWllZvyrBCK92U9AghE Twz3hduTveKwboQ1DqVLGWRRIw8OBop/O9wUqAnSnpQ+yU+EsqyKC002N1B5bCGtwVr/ FH//OvsXE8TonanO49hJC7OtAVmGeOPpB2QD37PTRaSqtunMAx1eCCcQTY88Vtn3shK5 YMq7P+if6Q61vp4oRAsL4R8CsKCliRh1EvnJl6VIQIb0E5dZy6uT2wJEgZHR9+/xcFLn K7XQ== ARC-Message-Signature: i=1; a=rsa-sha256; c=relaxed/relaxed; d=google.com; s=arc-20160816; h=list-id:precedence:in-reply-to:content-disposition:mime-version :references:mail-followup-to:message-id:subject:cc:to:from:date :dkim-signature; bh=+xEb27PMFQUN6a8DcLn87Q6DnS83FJhxmti0I/5v+c8=; b=DUM9cUuxz4sqjvV6B3KfnpEKWcPKZTyxmwyYrC9KpYjQwE4LniXR3lb8UBGNH2DzrF djgV1SzO7e8YdcNxXRcl37PLGdU4swyo/Oxejgpl7HqLiZebzhxHbay7HWUkCgNxvWZS uRDfhlX3YKBmSNcBGQl4nlnlklVnQm18dY1CAQ01wKFN72rYwtUtt/Y1iY8KYwLjz/en corMhQUh2RZR1eTULQzmis4heYTNupsHL3cCOAO0idoX6eO6LIrFhG0saKwo/3NBR1B4 3kDztdGKnwG4SbqiX7HMQcdAP7R0ArNF71kka/vPJF7/RxtW3LpIQqABnMO7rJZVyUU6 6yEw== ARC-Authentication-Results: i=1; mx.google.com; dkim=pass header.i=@ffwll.ch header.s=google header.b=NApzBmW3; spf=pass (google.com: domain of linux-kernel-owner@vger.kernel.org designates 23.128.96.18 as permitted sender) smtp.mailfrom=linux-kernel-owner@vger.kernel.org Return-Path: Received: from vger.kernel.org (vger.kernel.org. [23.128.96.18]) by mx.google.com with ESMTP id dp5si2704847ejc.590.2021.09.08.10.49.50; Wed, 08 Sep 2021 10:50:16 -0700 (PDT) Received-SPF: pass (google.com: domain of linux-kernel-owner@vger.kernel.org designates 23.128.96.18 as permitted sender) client-ip=23.128.96.18; Authentication-Results: mx.google.com; dkim=pass header.i=@ffwll.ch header.s=google header.b=NApzBmW3; spf=pass (google.com: domain of linux-kernel-owner@vger.kernel.org designates 23.128.96.18 as permitted sender) smtp.mailfrom=linux-kernel-owner@vger.kernel.org Received: (majordomo@vger.kernel.org) by vger.kernel.org via listexpand id S1349745AbhIHRqf (ORCPT + 99 others); Wed, 8 Sep 2021 13:46:35 -0400 Received: from lindbergh.monkeyblade.net ([23.128.96.19]:48400 "EHLO lindbergh.monkeyblade.net" rhost-flags-OK-OK-OK-OK) by vger.kernel.org with ESMTP id S235841AbhIHRqe (ORCPT ); Wed, 8 Sep 2021 13:46:34 -0400 Received: from mail-ed1-x535.google.com (mail-ed1-x535.google.com [IPv6:2a00:1450:4864:20::535]) by lindbergh.monkeyblade.net (Postfix) with ESMTPS id 79FA0C061575 for ; Wed, 8 Sep 2021 10:45:26 -0700 (PDT) Received: by mail-ed1-x535.google.com with SMTP id 9so2826623edx.11 for ; Wed, 08 Sep 2021 10:45:26 -0700 (PDT) DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=ffwll.ch; s=google; h=date:from:to:cc:subject:message-id:mail-followup-to:references :mime-version:content-disposition:in-reply-to; bh=+xEb27PMFQUN6a8DcLn87Q6DnS83FJhxmti0I/5v+c8=; b=NApzBmW3UHVNrriG9leoqpe0UPqlSwGG+HHEvMS6VrsAJ8Zk61JcNe2wN49FhKPnoj xp7yb2mjOQAw38ZaLHutQYGOZjV0CSU9Qo0X5R7hEJ23bnK8V9iPe1bEWcndMu/3PFf1 W9pHkI/23al4QTNYhOthpUxC7jqfJ3l2jmHWI= X-Google-DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=1e100.net; s=20210112; h=x-gm-message-state:date:from:to:cc:subject:message-id :mail-followup-to:references:mime-version:content-disposition :in-reply-to; bh=+xEb27PMFQUN6a8DcLn87Q6DnS83FJhxmti0I/5v+c8=; b=dRHpredHOsz+f545EQkSDeP51KrAQpSLNVVsnUBfEGM8ZKHe6wTqzjBoTXTVTLpnD5 UgnUoKwBBZEFCby1/Ov2ZgwaD7sNgYkxWbFYDaLL2q6BeBuMkJCAI5mbNsuGWsfRcxjg ruaebot+vpdtlEB2QXL6kiZ/r6UQk9YKBBFbky9wKeZen2KHp3b0IyppuyFV3qO+ITUQ vy0RZISIKvrY4yH22/542OxsMqzhwXwsWoiDEZzVNss/F3uwahqdLiPoBanR/rSvEYQ0 q5UxCH5lAcJaIoFjUdD2vu/362UzM2w6BwtGRmFNfHoEF4YRb6UFnj5uCT40ewR6Luj4 MGeA== X-Gm-Message-State: AOAM531zrb+uXIwuuSop+ZIssQ7ncAPHpwHHL163RbeoAEitWVEmvHvQ Em0eGg58OeiTRcZv5cz+KCyYIA== X-Received: by 2002:a50:d0d1:: with SMTP id g17mr2357446edf.96.1631123124210; Wed, 08 Sep 2021 10:45:24 -0700 (PDT) Received: from phenom.ffwll.local ([2a02:168:57f4:0:efd0:b9e5:5ae6:c2fa]) by smtp.gmail.com with ESMTPSA id r6sm1352603ejb.119.2021.09.08.10.45.22 (version=TLS1_3 cipher=TLS_AES_256_GCM_SHA384 bits=256/256); Wed, 08 Sep 2021 10:45:23 -0700 (PDT) Date: Wed, 8 Sep 2021 19:45:21 +0200 From: Daniel Vetter To: Rob Clark Cc: dri-devel@lists.freedesktop.org, linaro-mm-sig@lists.linaro.org, Daniel Vetter , Christian =?iso-8859-1?Q?K=F6nig?= , Michel =?iso-8859-1?Q?D=E4nzer?= , Pekka Paalanen , Rob Clark , David Airlie , Sumit Semwal , Christian =?iso-8859-1?Q?K=F6nig?= , Tian Tao , Steven Price , Melissa Wen , Luben Tuikov , Andrey Grodzovsky , Boris Brezillon , Jack Zhang , open list , "open list:DMA BUFFER SHARING FRAMEWORK" Subject: Re: [PATCH v3 4/9] drm/scheduler: Add fence deadline support Message-ID: Mail-Followup-To: Rob Clark , dri-devel@lists.freedesktop.org, linaro-mm-sig@lists.linaro.org, Christian =?iso-8859-1?Q?K=F6nig?= , Michel =?iso-8859-1?Q?D=E4nzer?= , Pekka Paalanen , Rob Clark , David Airlie , Sumit Semwal , Christian =?iso-8859-1?Q?K=F6nig?= , Tian Tao , Steven Price , Melissa Wen , Luben Tuikov , Andrey Grodzovsky , Boris Brezillon , Jack Zhang , open list , "open list:DMA BUFFER SHARING FRAMEWORK" References: <20210903184806.1680887-1-robdclark@gmail.com> <20210903184806.1680887-5-robdclark@gmail.com> MIME-Version: 1.0 Content-Type: text/plain; charset=us-ascii Content-Disposition: inline In-Reply-To: <20210903184806.1680887-5-robdclark@gmail.com> X-Operating-System: Linux phenom 5.10.0-8-amd64 Precedence: bulk List-ID: X-Mailing-List: linux-kernel@vger.kernel.org On Fri, Sep 03, 2021 at 11:47:55AM -0700, Rob Clark wrote: > From: Rob Clark > > As the finished fence is the one that is exposed to userspace, and > therefore the one that other operations, like atomic update, would > block on, we need to propagate the deadline from from the finished > fence to the actual hw fence. > > v2: Split into drm_sched_fence_set_parent() (ckoenig) > > Signed-off-by: Rob Clark > --- > drivers/gpu/drm/scheduler/sched_fence.c | 34 +++++++++++++++++++++++++ > drivers/gpu/drm/scheduler/sched_main.c | 2 +- > include/drm/gpu_scheduler.h | 8 ++++++ > 3 files changed, 43 insertions(+), 1 deletion(-) > > diff --git a/drivers/gpu/drm/scheduler/sched_fence.c b/drivers/gpu/drm/scheduler/sched_fence.c > index bcea035cf4c6..4fc41a71d1c7 100644 > --- a/drivers/gpu/drm/scheduler/sched_fence.c > +++ b/drivers/gpu/drm/scheduler/sched_fence.c > @@ -128,6 +128,30 @@ static void drm_sched_fence_release_finished(struct dma_fence *f) > dma_fence_put(&fence->scheduled); > } > > +static void drm_sched_fence_set_deadline_finished(struct dma_fence *f, > + ktime_t deadline) > +{ > + struct drm_sched_fence *fence = to_drm_sched_fence(f); > + unsigned long flags; > + > + spin_lock_irqsave(&fence->lock, flags); > + > + /* If we already have an earlier deadline, keep it: */ > + if (test_bit(DMA_FENCE_FLAG_HAS_DEADLINE_BIT, &f->flags) && > + ktime_before(fence->deadline, deadline)) { > + spin_unlock_irqrestore(&fence->lock, flags); > + return; > + } > + > + fence->deadline = deadline; > + set_bit(DMA_FENCE_FLAG_HAS_DEADLINE_BIT, &f->flags); > + > + spin_unlock_irqrestore(&fence->lock, flags); > + > + if (fence->parent) > + dma_fence_set_deadline(fence->parent, deadline); > +} > + > static const struct dma_fence_ops drm_sched_fence_ops_scheduled = { > .get_driver_name = drm_sched_fence_get_driver_name, > .get_timeline_name = drm_sched_fence_get_timeline_name, > @@ -138,6 +162,7 @@ static const struct dma_fence_ops drm_sched_fence_ops_finished = { > .get_driver_name = drm_sched_fence_get_driver_name, > .get_timeline_name = drm_sched_fence_get_timeline_name, > .release = drm_sched_fence_release_finished, > + .set_deadline = drm_sched_fence_set_deadline_finished, > }; > > struct drm_sched_fence *to_drm_sched_fence(struct dma_fence *f) > @@ -152,6 +177,15 @@ struct drm_sched_fence *to_drm_sched_fence(struct dma_fence *f) > } > EXPORT_SYMBOL(to_drm_sched_fence); > > +void drm_sched_fence_set_parent(struct drm_sched_fence *s_fence, > + struct dma_fence *fence) > +{ > + s_fence->parent = dma_fence_get(fence); > + if (test_bit(DMA_FENCE_FLAG_HAS_DEADLINE_BIT, > + &s_fence->finished.flags)) Don't you need the spinlock here too to avoid races? test_bit is very unordered, so guarantees nothing. Spinlock would need to be both around ->parent = and the test_bit. Entirely aside, but there's discussions going on to preallocate the hw fence somehow. If we do that we could make the deadline forwarding lockless here. Having a spinlock just to set the parent is a bit annoying ... Alternative is that you do this locklessly with barriers and a _lot_ of comments. Would be good to benchmark whether the overhead matters though first. -Daniel > + dma_fence_set_deadline(fence, s_fence->deadline); > +} > + > struct drm_sched_fence *drm_sched_fence_alloc(struct drm_sched_entity *entity, > void *owner) > { > diff --git a/drivers/gpu/drm/scheduler/sched_main.c b/drivers/gpu/drm/scheduler/sched_main.c > index 595e47ff7d06..27bf0ac0625f 100644 > --- a/drivers/gpu/drm/scheduler/sched_main.c > +++ b/drivers/gpu/drm/scheduler/sched_main.c > @@ -978,7 +978,7 @@ static int drm_sched_main(void *param) > drm_sched_fence_scheduled(s_fence); > > if (!IS_ERR_OR_NULL(fence)) { > - s_fence->parent = dma_fence_get(fence); > + drm_sched_fence_set_parent(s_fence, fence); > r = dma_fence_add_callback(fence, &sched_job->cb, > drm_sched_job_done_cb); > if (r == -ENOENT) > diff --git a/include/drm/gpu_scheduler.h b/include/drm/gpu_scheduler.h > index 7f77a455722c..158ddd662469 100644 > --- a/include/drm/gpu_scheduler.h > +++ b/include/drm/gpu_scheduler.h > @@ -238,6 +238,12 @@ struct drm_sched_fence { > */ > struct dma_fence finished; > > + /** > + * @deadline: deadline set on &drm_sched_fence.finished which > + * potentially needs to be propagated to &drm_sched_fence.parent > + */ > + ktime_t deadline; > + > /** > * @parent: the fence returned by &drm_sched_backend_ops.run_job > * when scheduling the job on hardware. We signal the > @@ -505,6 +511,8 @@ void drm_sched_entity_set_priority(struct drm_sched_entity *entity, > enum drm_sched_priority priority); > bool drm_sched_entity_is_ready(struct drm_sched_entity *entity); > > +void drm_sched_fence_set_parent(struct drm_sched_fence *s_fence, > + struct dma_fence *fence); > struct drm_sched_fence *drm_sched_fence_alloc( > struct drm_sched_entity *s_entity, void *owner); > void drm_sched_fence_init(struct drm_sched_fence *fence, > -- > 2.31.1 > -- Daniel Vetter Software Engineer, Intel Corporation http://blog.ffwll.ch