From: Rob Clark <[email protected]>
Inspired by https://lore.kernel.org/dri-devel/[email protected]/
it seemed like a good idea to get rid of memory allocation in job_run()
fence signaling path, and use lockdep annotations to yell at us about
anything that could deadlock against shrinker/reclaim. Anything that
can trigger reclaim, or block on any other thread that has triggered
reclaim, can block the GPU shrinker from releasing memory if it is
waiting the job to complete, causing deadlock.
The first patch pre-allocates the hw_fence, splitting allocation and
initialization, to avoid allocation in the job_run() path. The next
eight decouple the obj lock from job_run(), as the obj lock is required
to pin/unpin backing pages (ie. holding an obj lock in job_run() could
deadlock the shrinker by blocking forward progress towards pinned buffers
becoming idle). Followed by two so that we could idr_preload() in order
to avoid memory allocations under locks indirectly connected to the
shrinker path.
Next are three paths to decouple initialization (where allocations are
needed) from GPU runpm and devfreq, to avoid allocations in the fence
signaling path. Followed by various PM devfreq/qos and interconnect
locking fixes to decouple initialization (allocation) from runtime.
And finally, the last patch is a modified version of danvet's patch to
add lockdep annotations to gpu scheduler, but does so conditionally so
that drivers can opt-in.
v2: Switch from embedding hw_fence in submit/job object to preallocating
the hw_fence. Rework "fenced unpin" locking to drop obj lock from
fence signaling path (ie. the part that was still WIP in the first
iteration of the patchset). Adds the final patch to enable fence
signaling annotations now that job_run() and job_free() are safe.
The PM devfreq/QoS and interconnect patches are unchanged.
Rob Clark (23):
drm/msm: Pre-allocate hw_fence
drm/msm: Move submit bo flags update from obj lock
drm/msm/gem: Tidy up VMA API
drm/msm: Decouple vma tracking from obj lock
drm/msm/gem: Simplify vmap vs LRU tracking
drm/gem: Export drm_gem_lru_move_tail_locked()
drm/msm/gem: Move update_lru()
drm/msm/gem: Protect pin_count/madv by LRU lock
drm/msm/gem: Avoid obj lock in job_run()
drm/msm: Switch idr_lock to spinlock
drm/msm: Use idr_preload()
drm/msm/gpu: Move fw loading out of hw_init() path
drm/msm/gpu: Move BO allocation out of hw_init
drm/msm/a6xx: Move ioremap out of hw_init path
PM / devfreq: Drop unneed locking to appease lockdep
PM / devfreq: Teach lockdep about locking order
PM / QoS: Fix constraints alloc vs reclaim locking
PM / QoS: Decouple request alloc from dev_pm_qos_mtx
PM / QoS: Teach lockdep about dev_pm_qos_mtx locking order
soc: qcom: smd-rpm: Use GFP_ATOMIC in write path
interconnect: Fix locking for runpm vs reclaim
interconnect: Teach lockdep about icc_bw_lock order
drm/sched: Add (optional) fence signaling annotation
drivers/base/power/qos.c | 83 +++++++++---
drivers/devfreq/devfreq.c | 52 ++++----
drivers/gpu/drm/drm_gem.c | 11 +-
drivers/gpu/drm/msm/adreno/a5xx_gpu.c | 48 ++++---
drivers/gpu/drm/msm/adreno/a6xx_gmu.c | 18 ++-
drivers/gpu/drm/msm/adreno/a6xx_gpu.c | 46 ++++---
drivers/gpu/drm/msm/adreno/adreno_device.c | 6 +
drivers/gpu/drm/msm/adreno/adreno_gpu.c | 9 +-
drivers/gpu/drm/msm/msm_drv.c | 6 +-
drivers/gpu/drm/msm/msm_fence.c | 12 +-
drivers/gpu/drm/msm/msm_fence.h | 3 +-
drivers/gpu/drm/msm/msm_gem.c | 145 ++++++++++++++-------
drivers/gpu/drm/msm/msm_gem.h | 29 +++--
drivers/gpu/drm/msm/msm_gem_submit.c | 27 ++--
drivers/gpu/drm/msm/msm_gem_vma.c | 91 ++++++++++---
drivers/gpu/drm/msm/msm_gpu.h | 8 +-
drivers/gpu/drm/msm/msm_ringbuffer.c | 9 +-
drivers/gpu/drm/msm/msm_submitqueue.c | 2 +-
drivers/gpu/drm/scheduler/sched_main.c | 9 ++
drivers/interconnect/core.c | 18 ++-
drivers/soc/qcom/smd-rpm.c | 2 +-
include/drm/drm_gem.h | 1 +
include/drm/gpu_scheduler.h | 2 +
23 files changed, 416 insertions(+), 221 deletions(-)
--
2.39.2
From: Rob Clark <[email protected]>
Avoid allocating memory in job_run() by pre-allocating the hw_fence.
Signed-off-by: Rob Clark <[email protected]>
---
drivers/gpu/drm/msm/msm_fence.c | 12 +++++++++---
drivers/gpu/drm/msm/msm_fence.h | 3 ++-
drivers/gpu/drm/msm/msm_gem_submit.c | 7 +++++++
drivers/gpu/drm/msm/msm_ringbuffer.c | 2 +-
4 files changed, 19 insertions(+), 5 deletions(-)
diff --git a/drivers/gpu/drm/msm/msm_fence.c b/drivers/gpu/drm/msm/msm_fence.c
index 56641408ea74..bab3d84f1686 100644
--- a/drivers/gpu/drm/msm/msm_fence.c
+++ b/drivers/gpu/drm/msm/msm_fence.c
@@ -99,7 +99,7 @@ static const struct dma_fence_ops msm_fence_ops = {
};
struct dma_fence *
-msm_fence_alloc(struct msm_fence_context *fctx)
+msm_fence_alloc(void)
{
struct msm_fence *f;
@@ -107,10 +107,16 @@ msm_fence_alloc(struct msm_fence_context *fctx)
if (!f)
return ERR_PTR(-ENOMEM);
+ return &f->base;
+}
+
+void
+msm_fence_init(struct dma_fence *fence, struct msm_fence_context *fctx)
+{
+ struct msm_fence *f = to_msm_fence(fence);
+
f->fctx = fctx;
dma_fence_init(&f->base, &msm_fence_ops, &fctx->spinlock,
fctx->context, ++fctx->last_fence);
-
- return &f->base;
}
diff --git a/drivers/gpu/drm/msm/msm_fence.h b/drivers/gpu/drm/msm/msm_fence.h
index 7f1798c54cd1..f913fa22d8fe 100644
--- a/drivers/gpu/drm/msm/msm_fence.h
+++ b/drivers/gpu/drm/msm/msm_fence.h
@@ -61,7 +61,8 @@ void msm_fence_context_free(struct msm_fence_context *fctx);
bool msm_fence_completed(struct msm_fence_context *fctx, uint32_t fence);
void msm_update_fence(struct msm_fence_context *fctx, uint32_t fence);
-struct dma_fence * msm_fence_alloc(struct msm_fence_context *fctx);
+struct dma_fence * msm_fence_alloc(void);
+void msm_fence_init(struct dma_fence *fence, struct msm_fence_context *fctx);
static inline bool
fence_before(uint32_t a, uint32_t b)
diff --git a/drivers/gpu/drm/msm/msm_gem_submit.c b/drivers/gpu/drm/msm/msm_gem_submit.c
index be4bf77103cd..2570c018b0cb 100644
--- a/drivers/gpu/drm/msm/msm_gem_submit.c
+++ b/drivers/gpu/drm/msm/msm_gem_submit.c
@@ -41,6 +41,13 @@ static struct msm_gem_submit *submit_create(struct drm_device *dev,
if (!submit)
return ERR_PTR(-ENOMEM);
+ submit->hw_fence = msm_fence_alloc();
+ if (IS_ERR(submit->hw_fence)) {
+ ret = PTR_ERR(submit->hw_fence);
+ kfree(submit);
+ return ERR_PTR(ret);
+ }
+
ret = drm_sched_job_init(&submit->base, queue->entity, queue);
if (ret) {
kfree(submit);
diff --git a/drivers/gpu/drm/msm/msm_ringbuffer.c b/drivers/gpu/drm/msm/msm_ringbuffer.c
index 57a8e9564540..a62b45e5a8c3 100644
--- a/drivers/gpu/drm/msm/msm_ringbuffer.c
+++ b/drivers/gpu/drm/msm/msm_ringbuffer.c
@@ -18,7 +18,7 @@ static struct dma_fence *msm_job_run(struct drm_sched_job *job)
struct msm_gpu *gpu = submit->gpu;
int i;
- submit->hw_fence = msm_fence_alloc(fctx);
+ msm_fence_init(submit->hw_fence, fctx);
for (i = 0; i < submit->nr_bos; i++) {
struct drm_gem_object *obj = &submit->bos[i].obj->base;
--
2.39.2
From: Rob Clark <[email protected]>
The flags are only accessed (1) when submit is constructed, before
enqueuing to gpu sched (ie. when still visible to only the task calling
the submit ioctl), (2) here, where we own a reference to the submit and
are serialized on the gpu sched thread, and (3) after the submit is
retired and last reference is dropped, which is serialized on the
submit's reference count. Hence locking is unneeded here.
Signed-off-by: Rob Clark <[email protected]>
---
drivers/gpu/drm/msm/msm_ringbuffer.c | 2 +-
1 file changed, 1 insertion(+), 1 deletion(-)
diff --git a/drivers/gpu/drm/msm/msm_ringbuffer.c b/drivers/gpu/drm/msm/msm_ringbuffer.c
index a62b45e5a8c3..a80447c8764e 100644
--- a/drivers/gpu/drm/msm/msm_ringbuffer.c
+++ b/drivers/gpu/drm/msm/msm_ringbuffer.c
@@ -26,8 +26,8 @@ static struct dma_fence *msm_job_run(struct drm_sched_job *job)
msm_gem_lock(obj);
msm_gem_unpin_vma_fenced(submit->bos[i].vma, fctx);
msm_gem_unpin_locked(obj);
- submit->bos[i].flags &= ~(BO_VMA_PINNED | BO_OBJ_PINNED);
msm_gem_unlock(obj);
+ submit->bos[i].flags &= ~(BO_VMA_PINNED | BO_OBJ_PINNED);
}
/* TODO move submit path over to using a per-ring lock.. */
--
2.39.2
From: Rob Clark <[email protected]>
Stop open coding VMA construction, which will be needed in the next
commit. And since the VMA already has a ptr to the adress space, stop
passing that around everywhere. (Also, an aspace always has an mmu so
we can drop a couple pointless NULL checks.)
Signed-off-by: Rob Clark <[email protected]>
---
drivers/gpu/drm/msm/msm_gem.c | 18 +++++-----
drivers/gpu/drm/msm/msm_gem.h | 18 ++++------
drivers/gpu/drm/msm/msm_gem_submit.c | 2 +-
drivers/gpu/drm/msm/msm_gem_vma.c | 51 ++++++++++++++++++----------
drivers/gpu/drm/msm/msm_ringbuffer.c | 2 +-
5 files changed, 51 insertions(+), 40 deletions(-)
diff --git a/drivers/gpu/drm/msm/msm_gem.c b/drivers/gpu/drm/msm/msm_gem.c
index 1dee0d18abbb..6734aecf0703 100644
--- a/drivers/gpu/drm/msm/msm_gem.c
+++ b/drivers/gpu/drm/msm/msm_gem.c
@@ -309,12 +309,10 @@ static struct msm_gem_vma *add_vma(struct drm_gem_object *obj,
msm_gem_assert_locked(obj);
- vma = kzalloc(sizeof(*vma), GFP_KERNEL);
+ vma = msm_gem_vma_new(aspace);
if (!vma)
return ERR_PTR(-ENOMEM);
- vma->aspace = aspace;
-
list_add_tail(&vma->list, &msm_obj->vmas);
return vma;
@@ -361,9 +359,9 @@ put_iova_spaces(struct drm_gem_object *obj, bool close)
list_for_each_entry(vma, &msm_obj->vmas, list) {
if (vma->aspace) {
- msm_gem_purge_vma(vma->aspace, vma);
+ msm_gem_vma_purge(vma);
if (close)
- msm_gem_close_vma(vma->aspace, vma);
+ msm_gem_vma_close(vma);
}
}
}
@@ -399,7 +397,7 @@ static struct msm_gem_vma *get_vma_locked(struct drm_gem_object *obj,
if (IS_ERR(vma))
return vma;
- ret = msm_gem_init_vma(aspace, vma, obj->size,
+ ret = msm_gem_vma_init(vma, obj->size,
range_start, range_end);
if (ret) {
del_vma(vma);
@@ -437,7 +435,7 @@ int msm_gem_pin_vma_locked(struct drm_gem_object *obj, struct msm_gem_vma *vma)
if (IS_ERR(pages))
return PTR_ERR(pages);
- ret = msm_gem_map_vma(vma->aspace, vma, prot, msm_obj->sgt, obj->size);
+ ret = msm_gem_vma_map(vma, prot, msm_obj->sgt, obj->size);
if (ret)
msm_gem_unpin_locked(obj);
@@ -539,8 +537,8 @@ static int clear_iova(struct drm_gem_object *obj,
if (msm_gem_vma_inuse(vma))
return -EBUSY;
- msm_gem_purge_vma(vma->aspace, vma);
- msm_gem_close_vma(vma->aspace, vma);
+ msm_gem_vma_purge(vma);
+ msm_gem_vma_close(vma);
del_vma(vma);
return 0;
@@ -589,7 +587,7 @@ void msm_gem_unpin_iova(struct drm_gem_object *obj,
msm_gem_lock(obj);
vma = lookup_vma(obj, aspace);
if (!GEM_WARN_ON(!vma)) {
- msm_gem_unpin_vma(vma);
+ msm_gem_vma_unpin(vma);
msm_gem_unpin_locked(obj);
}
msm_gem_unlock(obj);
diff --git a/drivers/gpu/drm/msm/msm_gem.h b/drivers/gpu/drm/msm/msm_gem.h
index c4844cf3a585..d3219c523034 100644
--- a/drivers/gpu/drm/msm/msm_gem.h
+++ b/drivers/gpu/drm/msm/msm_gem.h
@@ -69,19 +69,15 @@ struct msm_gem_vma {
struct msm_fence_context *fctx[MSM_GPU_MAX_RINGS];
};
-int msm_gem_init_vma(struct msm_gem_address_space *aspace,
- struct msm_gem_vma *vma, int size,
+struct msm_gem_vma *msm_gem_vma_new(struct msm_gem_address_space *aspace);
+int msm_gem_vma_init(struct msm_gem_vma *vma, int size,
u64 range_start, u64 range_end);
bool msm_gem_vma_inuse(struct msm_gem_vma *vma);
-void msm_gem_purge_vma(struct msm_gem_address_space *aspace,
- struct msm_gem_vma *vma);
-void msm_gem_unpin_vma(struct msm_gem_vma *vma);
-void msm_gem_unpin_vma_fenced(struct msm_gem_vma *vma, struct msm_fence_context *fctx);
-int msm_gem_map_vma(struct msm_gem_address_space *aspace,
- struct msm_gem_vma *vma, int prot,
- struct sg_table *sgt, int size);
-void msm_gem_close_vma(struct msm_gem_address_space *aspace,
- struct msm_gem_vma *vma);
+void msm_gem_vma_purge(struct msm_gem_vma *vma);
+void msm_gem_vma_unpin(struct msm_gem_vma *vma);
+void msm_gem_vma_unpin_fenced(struct msm_gem_vma *vma, struct msm_fence_context *fctx);
+int msm_gem_vma_map(struct msm_gem_vma *vma, int prot, struct sg_table *sgt, int size);
+void msm_gem_vma_close(struct msm_gem_vma *vma);
struct msm_gem_object {
struct drm_gem_object base;
diff --git a/drivers/gpu/drm/msm/msm_gem_submit.c b/drivers/gpu/drm/msm/msm_gem_submit.c
index 2570c018b0cb..1d8e7c2a8024 100644
--- a/drivers/gpu/drm/msm/msm_gem_submit.c
+++ b/drivers/gpu/drm/msm/msm_gem_submit.c
@@ -249,7 +249,7 @@ static void submit_cleanup_bo(struct msm_gem_submit *submit, int i,
submit->bos[i].flags &= ~cleanup_flags;
if (flags & BO_VMA_PINNED)
- msm_gem_unpin_vma(submit->bos[i].vma);
+ msm_gem_vma_unpin(submit->bos[i].vma);
if (flags & BO_OBJ_PINNED)
msm_gem_unpin_locked(obj);
diff --git a/drivers/gpu/drm/msm/msm_gem_vma.c b/drivers/gpu/drm/msm/msm_gem_vma.c
index c471aebcdbab..2827679dc39a 100644
--- a/drivers/gpu/drm/msm/msm_gem_vma.c
+++ b/drivers/gpu/drm/msm/msm_gem_vma.c
@@ -56,9 +56,9 @@ bool msm_gem_vma_inuse(struct msm_gem_vma *vma)
}
/* Actually unmap memory for the vma */
-void msm_gem_purge_vma(struct msm_gem_address_space *aspace,
- struct msm_gem_vma *vma)
+void msm_gem_vma_purge(struct msm_gem_vma *vma)
{
+ struct msm_gem_address_space *aspace = vma->aspace;
unsigned size = vma->node.size;
/* Print a message if we try to purge a vma in use */
@@ -68,14 +68,13 @@ void msm_gem_purge_vma(struct msm_gem_address_space *aspace,
if (!vma->mapped)
return;
- if (aspace->mmu)
- aspace->mmu->funcs->unmap(aspace->mmu, vma->iova, size);
+ aspace->mmu->funcs->unmap(aspace->mmu, vma->iova, size);
vma->mapped = false;
}
/* Remove reference counts for the mapping */
-void msm_gem_unpin_vma(struct msm_gem_vma *vma)
+void msm_gem_vma_unpin(struct msm_gem_vma *vma)
{
if (GEM_WARN_ON(!vma->inuse))
return;
@@ -84,21 +83,21 @@ void msm_gem_unpin_vma(struct msm_gem_vma *vma)
}
/* Replace pin reference with fence: */
-void msm_gem_unpin_vma_fenced(struct msm_gem_vma *vma, struct msm_fence_context *fctx)
+void msm_gem_vma_unpin_fenced(struct msm_gem_vma *vma, struct msm_fence_context *fctx)
{
vma->fctx[fctx->index] = fctx;
vma->fence[fctx->index] = fctx->last_fence;
vma->fence_mask |= BIT(fctx->index);
- msm_gem_unpin_vma(vma);
+ msm_gem_vma_unpin(vma);
}
/* Map and pin vma: */
int
-msm_gem_map_vma(struct msm_gem_address_space *aspace,
- struct msm_gem_vma *vma, int prot,
+msm_gem_vma_map(struct msm_gem_vma *vma, int prot,
struct sg_table *sgt, int size)
{
- int ret = 0;
+ struct msm_gem_address_space *aspace = vma->aspace;
+ int ret;
if (GEM_WARN_ON(!vma->iova))
return -EINVAL;
@@ -111,9 +110,10 @@ msm_gem_map_vma(struct msm_gem_address_space *aspace,
vma->mapped = true;
- if (aspace && aspace->mmu)
- ret = aspace->mmu->funcs->map(aspace->mmu, vma->iova, sgt,
- size, prot);
+ if (!aspace)
+ return 0;
+
+ ret = aspace->mmu->funcs->map(aspace->mmu, vma->iova, sgt, size, prot);
if (ret) {
vma->mapped = false;
@@ -124,9 +124,10 @@ msm_gem_map_vma(struct msm_gem_address_space *aspace,
}
/* Close an iova. Warn if it is still in use */
-void msm_gem_close_vma(struct msm_gem_address_space *aspace,
- struct msm_gem_vma *vma)
+void msm_gem_vma_close(struct msm_gem_vma *vma)
{
+ struct msm_gem_address_space *aspace = vma->aspace;
+
GEM_WARN_ON(msm_gem_vma_inuse(vma) || vma->mapped);
spin_lock(&aspace->lock);
@@ -139,13 +140,29 @@ void msm_gem_close_vma(struct msm_gem_address_space *aspace,
msm_gem_address_space_put(aspace);
}
+struct msm_gem_vma *msm_gem_vma_new(struct msm_gem_address_space *aspace)
+{
+ struct msm_gem_vma *vma;
+
+ vma = kzalloc(sizeof(*vma), GFP_KERNEL);
+ if (!vma)
+ return NULL;
+
+ vma->aspace = aspace;
+
+ return vma;
+}
+
/* Initialize a new vma and allocate an iova for it */
-int msm_gem_init_vma(struct msm_gem_address_space *aspace,
- struct msm_gem_vma *vma, int size,
+int msm_gem_vma_init(struct msm_gem_vma *vma, int size,
u64 range_start, u64 range_end)
{
+ struct msm_gem_address_space *aspace = vma->aspace;
int ret;
+ if (GEM_WARN_ON(!aspace))
+ return -EINVAL;
+
if (GEM_WARN_ON(vma->iova))
return -EBUSY;
diff --git a/drivers/gpu/drm/msm/msm_ringbuffer.c b/drivers/gpu/drm/msm/msm_ringbuffer.c
index a80447c8764e..44a22b283730 100644
--- a/drivers/gpu/drm/msm/msm_ringbuffer.c
+++ b/drivers/gpu/drm/msm/msm_ringbuffer.c
@@ -24,7 +24,7 @@ static struct dma_fence *msm_job_run(struct drm_sched_job *job)
struct drm_gem_object *obj = &submit->bos[i].obj->base;
msm_gem_lock(obj);
- msm_gem_unpin_vma_fenced(submit->bos[i].vma, fctx);
+ msm_gem_vma_unpin_fenced(submit->bos[i].vma, fctx);
msm_gem_unpin_locked(obj);
msm_gem_unlock(obj);
submit->bos[i].flags &= ~(BO_VMA_PINNED | BO_OBJ_PINNED);
--
2.39.2
From: Rob Clark <[email protected]>
We need to use the inuse count to track that a BO is pinned until
we have the hw_fence. But we want to remove the obj lock from the
job_run() path as this could deadlock against reclaim/shrinker
(because it is blocking the hw_fence from eventually being signaled).
So split that tracking out into a per-vma lock with narrower scope.
Signed-off-by: Rob Clark <[email protected]>
---
drivers/gpu/drm/msm/msm_gem.h | 1 +
drivers/gpu/drm/msm/msm_gem_vma.c | 44 ++++++++++++++++++++++++----
drivers/gpu/drm/msm/msm_ringbuffer.c | 2 +-
3 files changed, 40 insertions(+), 7 deletions(-)
diff --git a/drivers/gpu/drm/msm/msm_gem.h b/drivers/gpu/drm/msm/msm_gem.h
index d3219c523034..1929f09c5b0d 100644
--- a/drivers/gpu/drm/msm/msm_gem.h
+++ b/drivers/gpu/drm/msm/msm_gem.h
@@ -59,6 +59,7 @@ struct msm_fence_context;
struct msm_gem_vma {
struct drm_mm_node node;
+ spinlock_t lock;
uint64_t iova;
struct msm_gem_address_space *aspace;
struct list_head list; /* node in msm_gem_object::vmas */
diff --git a/drivers/gpu/drm/msm/msm_gem_vma.c b/drivers/gpu/drm/msm/msm_gem_vma.c
index 2827679dc39a..98287ed99960 100644
--- a/drivers/gpu/drm/msm/msm_gem_vma.c
+++ b/drivers/gpu/drm/msm/msm_gem_vma.c
@@ -40,19 +40,28 @@ msm_gem_address_space_get(struct msm_gem_address_space *aspace)
bool msm_gem_vma_inuse(struct msm_gem_vma *vma)
{
+ bool ret = true;
+
+ spin_lock(&vma->lock);
+
if (vma->inuse > 0)
- return true;
+ goto out;
while (vma->fence_mask) {
unsigned idx = ffs(vma->fence_mask) - 1;
if (!msm_fence_completed(vma->fctx[idx], vma->fence[idx]))
- return true;
+ goto out;
vma->fence_mask &= ~BIT(idx);
}
- return false;
+ ret = false;
+
+out:
+ spin_unlock(&vma->lock);
+
+ return ret;
}
/* Actually unmap memory for the vma */
@@ -73,8 +82,7 @@ void msm_gem_vma_purge(struct msm_gem_vma *vma)
vma->mapped = false;
}
-/* Remove reference counts for the mapping */
-void msm_gem_vma_unpin(struct msm_gem_vma *vma)
+static void vma_unpin_locked(struct msm_gem_vma *vma)
{
if (GEM_WARN_ON(!vma->inuse))
return;
@@ -82,13 +90,23 @@ void msm_gem_vma_unpin(struct msm_gem_vma *vma)
vma->inuse--;
}
+/* Remove reference counts for the mapping */
+void msm_gem_vma_unpin(struct msm_gem_vma *vma)
+{
+ spin_lock(&vma->lock);
+ vma_unpin_locked(vma);
+ spin_unlock(&vma->lock);
+}
+
/* Replace pin reference with fence: */
void msm_gem_vma_unpin_fenced(struct msm_gem_vma *vma, struct msm_fence_context *fctx)
{
+ spin_lock(&vma->lock);
vma->fctx[fctx->index] = fctx;
vma->fence[fctx->index] = fctx->last_fence;
vma->fence_mask |= BIT(fctx->index);
- msm_gem_vma_unpin(vma);
+ vma_unpin_locked(vma);
+ spin_unlock(&vma->lock);
}
/* Map and pin vma: */
@@ -103,7 +121,9 @@ msm_gem_vma_map(struct msm_gem_vma *vma, int prot,
return -EINVAL;
/* Increase the usage counter */
+ spin_lock(&vma->lock);
vma->inuse++;
+ spin_unlock(&vma->lock);
if (vma->mapped)
return 0;
@@ -113,11 +133,22 @@ msm_gem_vma_map(struct msm_gem_vma *vma, int prot,
if (!aspace)
return 0;
+ /*
+ * NOTE: iommu/io-pgtable can allocate pages, so we cannot hold
+ * a lock across map/unmap which is also used in the job_run()
+ * path, as this can cause deadlock in job_run() vs shrinker/
+ * reclaim.
+ *
+ * Revisit this if we can come up with a scheme to pre-alloc pages
+ * for the pgtable in map/unmap ops.
+ */
ret = aspace->mmu->funcs->map(aspace->mmu, vma->iova, sgt, size, prot);
if (ret) {
vma->mapped = false;
+ spin_lock(&vma->lock);
vma->inuse--;
+ spin_unlock(&vma->lock);
}
return ret;
@@ -148,6 +179,7 @@ struct msm_gem_vma *msm_gem_vma_new(struct msm_gem_address_space *aspace)
if (!vma)
return NULL;
+ spin_lock_init(&vma->lock);
vma->aspace = aspace;
return vma;
diff --git a/drivers/gpu/drm/msm/msm_ringbuffer.c b/drivers/gpu/drm/msm/msm_ringbuffer.c
index 44a22b283730..31b4fbf96c36 100644
--- a/drivers/gpu/drm/msm/msm_ringbuffer.c
+++ b/drivers/gpu/drm/msm/msm_ringbuffer.c
@@ -23,8 +23,8 @@ static struct dma_fence *msm_job_run(struct drm_sched_job *job)
for (i = 0; i < submit->nr_bos; i++) {
struct drm_gem_object *obj = &submit->bos[i].obj->base;
- msm_gem_lock(obj);
msm_gem_vma_unpin_fenced(submit->bos[i].vma, fctx);
+ msm_gem_lock(obj);
msm_gem_unpin_locked(obj);
msm_gem_unlock(obj);
submit->bos[i].flags &= ~(BO_VMA_PINNED | BO_OBJ_PINNED);
--
2.39.2
From: Rob Clark <[email protected]>
vmap'ing is just pinning in disguise. So treat it as such and simplify
the LRU tracking.
Signed-off-by: Rob Clark <[email protected]>
---
drivers/gpu/drm/msm/msm_gem.c | 17 ++++++++---------
1 file changed, 8 insertions(+), 9 deletions(-)
diff --git a/drivers/gpu/drm/msm/msm_gem.c b/drivers/gpu/drm/msm/msm_gem.c
index 6734aecf0703..009a34b3a49b 100644
--- a/drivers/gpu/drm/msm/msm_gem.c
+++ b/drivers/gpu/drm/msm/msm_gem.c
@@ -626,6 +626,7 @@ int msm_gem_dumb_map_offset(struct drm_file *file, struct drm_device *dev,
static void *get_vaddr(struct drm_gem_object *obj, unsigned madv)
{
struct msm_gem_object *msm_obj = to_msm_bo(obj);
+ struct page **pages;
int ret = 0;
msm_gem_assert_locked(obj);
@@ -639,6 +640,10 @@ static void *get_vaddr(struct drm_gem_object *obj, unsigned madv)
return ERR_PTR(-EBUSY);
}
+ pages = msm_gem_pin_pages_locked(obj);
+ if (IS_ERR(pages))
+ return ERR_CAST(pages);
+
/* increment vmap_count *before* vmap() call, so shrinker can
* check vmap_count (is_vunmapable()) outside of msm_obj lock.
* This guarantees that we won't try to msm_gem_vunmap() this
@@ -648,25 +653,19 @@ static void *get_vaddr(struct drm_gem_object *obj, unsigned madv)
msm_obj->vmap_count++;
if (!msm_obj->vaddr) {
- struct page **pages = get_pages(obj);
- if (IS_ERR(pages)) {
- ret = PTR_ERR(pages);
- goto fail;
- }
msm_obj->vaddr = vmap(pages, obj->size >> PAGE_SHIFT,
VM_MAP, msm_gem_pgprot(msm_obj, PAGE_KERNEL));
if (msm_obj->vaddr == NULL) {
ret = -ENOMEM;
goto fail;
}
-
- update_lru(obj);
}
return msm_obj->vaddr;
fail:
msm_obj->vmap_count--;
+ msm_gem_unpin_locked(obj);
return ERR_PTR(ret);
}
@@ -705,6 +704,7 @@ void msm_gem_put_vaddr_locked(struct drm_gem_object *obj)
GEM_WARN_ON(msm_obj->vmap_count < 1);
msm_obj->vmap_count--;
+ msm_gem_unpin_locked(obj);
}
void msm_gem_put_vaddr(struct drm_gem_object *obj)
@@ -813,10 +813,9 @@ static void update_lru(struct drm_gem_object *obj)
if (!msm_obj->pages) {
GEM_WARN_ON(msm_obj->pin_count);
- GEM_WARN_ON(msm_obj->vmap_count);
drm_gem_lru_move_tail(&priv->lru.unbacked, obj);
- } else if (msm_obj->pin_count || msm_obj->vmap_count) {
+ } else if (msm_obj->pin_count) {
drm_gem_lru_move_tail(&priv->lru.pinned, obj);
} else if (msm_obj->madv == MSM_MADV_WILLNEED) {
drm_gem_lru_move_tail(&priv->lru.willneed, obj);
--
2.39.2
From: Rob Clark <[email protected]>
Export the locked version or lru's move_tail().
Signed-off-by: Rob Clark <[email protected]>
---
drivers/gpu/drm/drm_gem.c | 11 ++++++++++-
include/drm/drm_gem.h | 1 +
2 files changed, 11 insertions(+), 1 deletion(-)
diff --git a/drivers/gpu/drm/drm_gem.c b/drivers/gpu/drm/drm_gem.c
index 59a0bb5ebd85..693f7f35a7bd 100644
--- a/drivers/gpu/drm/drm_gem.c
+++ b/drivers/gpu/drm/drm_gem.c
@@ -1344,7 +1344,15 @@ drm_gem_lru_remove(struct drm_gem_object *obj)
}
EXPORT_SYMBOL(drm_gem_lru_remove);
-static void
+/**
+ * drm_gem_lru_move_tail_locked - move the object to the tail of the LRU
+ *
+ * Like &drm_gem_lru_move_tail but lru lock must be held
+ *
+ * @lru: The LRU to move the object into.
+ * @obj: The GEM object to move into this LRU
+ */
+void
drm_gem_lru_move_tail_locked(struct drm_gem_lru *lru, struct drm_gem_object *obj)
{
lockdep_assert_held_once(lru->lock);
@@ -1356,6 +1364,7 @@ drm_gem_lru_move_tail_locked(struct drm_gem_lru *lru, struct drm_gem_object *obj
list_add_tail(&obj->lru_node, &lru->list);
obj->lru = lru;
}
+EXPORT_SYMBOL(drm_gem_lru_move_tail_locked);
/**
* drm_gem_lru_move_tail - move the object to the tail of the LRU
diff --git a/include/drm/drm_gem.h b/include/drm/drm_gem.h
index 772a4adf5287..a811c7e346ec 100644
--- a/include/drm/drm_gem.h
+++ b/include/drm/drm_gem.h
@@ -475,6 +475,7 @@ int drm_gem_dumb_map_offset(struct drm_file *file, struct drm_device *dev,
void drm_gem_lru_init(struct drm_gem_lru *lru, struct mutex *lock);
void drm_gem_lru_remove(struct drm_gem_object *obj);
+void drm_gem_lru_move_tail_locked(struct drm_gem_lru *lru, struct drm_gem_object *obj);
void drm_gem_lru_move_tail(struct drm_gem_lru *lru, struct drm_gem_object *obj);
unsigned long drm_gem_lru_scan(struct drm_gem_lru *lru, unsigned nr_to_scan,
bool (*shrink)(struct drm_gem_object *obj));
--
2.39.2
From: Rob Clark <[email protected]>
Just code-motion.
Signed-off-by: Rob Clark <[email protected]>
---
drivers/gpu/drm/msm/msm_gem.c | 46 +++++++++++++++++------------------
1 file changed, 22 insertions(+), 24 deletions(-)
diff --git a/drivers/gpu/drm/msm/msm_gem.c b/drivers/gpu/drm/msm/msm_gem.c
index 009a34b3a49b..c97dddf3e2f2 100644
--- a/drivers/gpu/drm/msm/msm_gem.c
+++ b/drivers/gpu/drm/msm/msm_gem.c
@@ -19,8 +19,6 @@
#include "msm_gpu.h"
#include "msm_mmu.h"
-static void update_lru(struct drm_gem_object *obj);
-
static dma_addr_t physaddr(struct drm_gem_object *obj)
{
struct msm_gem_object *msm_obj = to_msm_bo(obj);
@@ -63,6 +61,28 @@ static void sync_for_cpu(struct msm_gem_object *msm_obj)
dma_unmap_sgtable(dev, msm_obj->sgt, DMA_BIDIRECTIONAL, 0);
}
+static void update_lru(struct drm_gem_object *obj)
+{
+ struct msm_drm_private *priv = obj->dev->dev_private;
+ struct msm_gem_object *msm_obj = to_msm_bo(obj);
+
+ msm_gem_assert_locked(&msm_obj->base);
+
+ if (!msm_obj->pages) {
+ GEM_WARN_ON(msm_obj->pin_count);
+
+ drm_gem_lru_move_tail(&priv->lru.unbacked, obj);
+ } else if (msm_obj->pin_count) {
+ drm_gem_lru_move_tail(&priv->lru.pinned, obj);
+ } else if (msm_obj->madv == MSM_MADV_WILLNEED) {
+ drm_gem_lru_move_tail(&priv->lru.willneed, obj);
+ } else {
+ GEM_WARN_ON(msm_obj->madv != MSM_MADV_DONTNEED);
+
+ drm_gem_lru_move_tail(&priv->lru.dontneed, obj);
+ }
+}
+
/* allocate pages from VRAM carveout, used when no IOMMU: */
static struct page **get_pages_vram(struct drm_gem_object *obj, int npages)
{
@@ -804,28 +824,6 @@ void msm_gem_vunmap(struct drm_gem_object *obj)
msm_obj->vaddr = NULL;
}
-static void update_lru(struct drm_gem_object *obj)
-{
- struct msm_drm_private *priv = obj->dev->dev_private;
- struct msm_gem_object *msm_obj = to_msm_bo(obj);
-
- msm_gem_assert_locked(&msm_obj->base);
-
- if (!msm_obj->pages) {
- GEM_WARN_ON(msm_obj->pin_count);
-
- drm_gem_lru_move_tail(&priv->lru.unbacked, obj);
- } else if (msm_obj->pin_count) {
- drm_gem_lru_move_tail(&priv->lru.pinned, obj);
- } else if (msm_obj->madv == MSM_MADV_WILLNEED) {
- drm_gem_lru_move_tail(&priv->lru.willneed, obj);
- } else {
- GEM_WARN_ON(msm_obj->madv != MSM_MADV_DONTNEED);
-
- drm_gem_lru_move_tail(&priv->lru.dontneed, obj);
- }
-}
-
bool msm_gem_active(struct drm_gem_object *obj)
{
msm_gem_assert_locked(obj);
--
2.39.2
From: Rob Clark <[email protected]>
Since the LRU lock is already acquired when moving an obj between LRUs,
we can use it to protect pin_count and madv, without any significant
change in locking (ie. it just expands the scope of the lock by a hand-
ful of instructions). This prepares the way to decrement the pin_count
in the job_run() path without needing to hold the obj lock, to avoid a
potential deadlock (or rather stall) caused by the fence-signaling path
(job_run()) blocking on shrinker/reclaim. (Only a stall because the
wait for fence signaling wait_for_idle() is not infinite.)
Signed-off-by: Rob Clark <[email protected]>
---
drivers/gpu/drm/msm/msm_gem.c | 48 ++++++++++++++++++++++++++---------
drivers/gpu/drm/msm/msm_gem.h | 9 ++++++-
2 files changed, 44 insertions(+), 13 deletions(-)
diff --git a/drivers/gpu/drm/msm/msm_gem.c b/drivers/gpu/drm/msm/msm_gem.c
index c97dddf3e2f2..d0ac3e704b66 100644
--- a/drivers/gpu/drm/msm/msm_gem.c
+++ b/drivers/gpu/drm/msm/msm_gem.c
@@ -61,7 +61,7 @@ static void sync_for_cpu(struct msm_gem_object *msm_obj)
dma_unmap_sgtable(dev, msm_obj->sgt, DMA_BIDIRECTIONAL, 0);
}
-static void update_lru(struct drm_gem_object *obj)
+static void update_lru_locked(struct drm_gem_object *obj)
{
struct msm_drm_private *priv = obj->dev->dev_private;
struct msm_gem_object *msm_obj = to_msm_bo(obj);
@@ -71,18 +71,27 @@ static void update_lru(struct drm_gem_object *obj)
if (!msm_obj->pages) {
GEM_WARN_ON(msm_obj->pin_count);
- drm_gem_lru_move_tail(&priv->lru.unbacked, obj);
+ drm_gem_lru_move_tail_locked(&priv->lru.unbacked, obj);
} else if (msm_obj->pin_count) {
- drm_gem_lru_move_tail(&priv->lru.pinned, obj);
+ drm_gem_lru_move_tail_locked(&priv->lru.pinned, obj);
} else if (msm_obj->madv == MSM_MADV_WILLNEED) {
- drm_gem_lru_move_tail(&priv->lru.willneed, obj);
+ drm_gem_lru_move_tail_locked(&priv->lru.willneed, obj);
} else {
GEM_WARN_ON(msm_obj->madv != MSM_MADV_DONTNEED);
- drm_gem_lru_move_tail(&priv->lru.dontneed, obj);
+ drm_gem_lru_move_tail_locked(&priv->lru.dontneed, obj);
}
}
+static void update_lru(struct drm_gem_object *obj)
+{
+ struct msm_drm_private *priv = obj->dev->dev_private;
+
+ mutex_lock(&priv->lru.lock);
+ update_lru_locked(obj);
+ mutex_unlock(&priv->lru.lock);
+}
+
/* allocate pages from VRAM carveout, used when no IOMMU: */
static struct page **get_pages_vram(struct drm_gem_object *obj, int npages)
{
@@ -200,6 +209,7 @@ static void put_pages(struct drm_gem_object *obj)
static struct page **msm_gem_pin_pages_locked(struct drm_gem_object *obj)
{
+ struct msm_drm_private *priv = obj->dev->dev_private;
struct msm_gem_object *msm_obj = to_msm_bo(obj);
struct page **p;
@@ -210,10 +220,13 @@ static struct page **msm_gem_pin_pages_locked(struct drm_gem_object *obj)
}
p = get_pages(obj);
- if (!IS_ERR(p)) {
- to_msm_bo(obj)->pin_count++;
- update_lru(obj);
- }
+ if (IS_ERR(p))
+ return p;
+
+ mutex_lock(&priv->lru.lock);
+ msm_obj->pin_count++;
+ update_lru_locked(obj);
+ mutex_unlock(&priv->lru.lock);
return p;
}
@@ -464,14 +477,16 @@ int msm_gem_pin_vma_locked(struct drm_gem_object *obj, struct msm_gem_vma *vma)
void msm_gem_unpin_locked(struct drm_gem_object *obj)
{
+ struct msm_drm_private *priv = obj->dev->dev_private;
struct msm_gem_object *msm_obj = to_msm_bo(obj);
msm_gem_assert_locked(obj);
+ mutex_lock(&priv->lru.lock);
msm_obj->pin_count--;
GEM_WARN_ON(msm_obj->pin_count < 0);
-
- update_lru(obj);
+ update_lru_locked(obj);
+ mutex_unlock(&priv->lru.lock);
}
struct msm_gem_vma *msm_gem_get_vma_locked(struct drm_gem_object *obj,
@@ -739,10 +754,13 @@ void msm_gem_put_vaddr(struct drm_gem_object *obj)
*/
int msm_gem_madvise(struct drm_gem_object *obj, unsigned madv)
{
+ struct msm_drm_private *priv = obj->dev->dev_private;
struct msm_gem_object *msm_obj = to_msm_bo(obj);
msm_gem_lock(obj);
+ mutex_lock(&priv->lru.lock);
+
if (msm_obj->madv != __MSM_MADV_PURGED)
msm_obj->madv = madv;
@@ -751,7 +769,9 @@ int msm_gem_madvise(struct drm_gem_object *obj, unsigned madv)
/* If the obj is inactive, we might need to move it
* between inactive lists
*/
- update_lru(obj);
+ update_lru_locked(obj);
+
+ mutex_unlock(&priv->lru.lock);
msm_gem_unlock(obj);
@@ -761,6 +781,7 @@ int msm_gem_madvise(struct drm_gem_object *obj, unsigned madv)
void msm_gem_purge(struct drm_gem_object *obj)
{
struct drm_device *dev = obj->dev;
+ struct msm_drm_private *priv = obj->dev->dev_private;
struct msm_gem_object *msm_obj = to_msm_bo(obj);
msm_gem_assert_locked(obj);
@@ -777,7 +798,10 @@ void msm_gem_purge(struct drm_gem_object *obj)
put_iova_vmas(obj);
+ mutex_lock(&priv->lru.lock);
+ /* A one-way transition: */
msm_obj->madv = __MSM_MADV_PURGED;
+ mutex_unlock(&priv->lru.lock);
drm_gem_free_mmap_offset(obj);
diff --git a/drivers/gpu/drm/msm/msm_gem.h b/drivers/gpu/drm/msm/msm_gem.h
index 1929f09c5b0d..0057e8e8fa13 100644
--- a/drivers/gpu/drm/msm/msm_gem.h
+++ b/drivers/gpu/drm/msm/msm_gem.h
@@ -86,7 +86,9 @@ struct msm_gem_object {
uint32_t flags;
/**
- * Advice: are the backing pages purgeable?
+ * madv: are the backing pages purgeable?
+ *
+ * Protected by obj lock and LRU lock
*/
uint8_t madv;
@@ -114,6 +116,11 @@ struct msm_gem_object {
char name[32]; /* Identifier to print for the debugfs files */
+ /**
+ * pin_count: Number of times the pages are pinned
+ *
+ * Protected by LRU lock.
+ */
int pin_count;
};
#define to_msm_bo(x) container_of(x, struct msm_gem_object, base)
--
2.39.2
From: Rob Clark <[email protected]>
Needed to idr_preload() which returns with preemption disabled.
Signed-off-by: Rob Clark <[email protected]>
---
drivers/gpu/drm/msm/msm_drv.c | 6 ++----
drivers/gpu/drm/msm/msm_gem_submit.c | 10 +++++-----
drivers/gpu/drm/msm/msm_gpu.h | 2 +-
drivers/gpu/drm/msm/msm_submitqueue.c | 2 +-
4 files changed, 9 insertions(+), 11 deletions(-)
diff --git a/drivers/gpu/drm/msm/msm_drv.c b/drivers/gpu/drm/msm/msm_drv.c
index aca48c868c14..ce1a77b607d1 100644
--- a/drivers/gpu/drm/msm/msm_drv.c
+++ b/drivers/gpu/drm/msm/msm_drv.c
@@ -918,13 +918,11 @@ static int wait_fence(struct msm_gpu_submitqueue *queue, uint32_t fence_id,
* retired, so if the fence is not found it means there is nothing
* to wait for
*/
- ret = mutex_lock_interruptible(&queue->idr_lock);
- if (ret)
- return ret;
+ spin_lock(&queue->idr_lock);
fence = idr_find(&queue->fence_idr, fence_id);
if (fence)
fence = dma_fence_get_rcu(fence);
- mutex_unlock(&queue->idr_lock);
+ spin_unlock(&queue->idr_lock);
if (!fence)
return 0;
diff --git a/drivers/gpu/drm/msm/msm_gem_submit.c b/drivers/gpu/drm/msm/msm_gem_submit.c
index 1d8e7c2a8024..b9d81e5acb42 100644
--- a/drivers/gpu/drm/msm/msm_gem_submit.c
+++ b/drivers/gpu/drm/msm/msm_gem_submit.c
@@ -79,9 +79,9 @@ void __msm_gem_submit_destroy(struct kref *kref)
unsigned i;
if (submit->fence_id) {
- mutex_lock(&submit->queue->idr_lock);
+ spin_lock(&submit->queue->idr_lock);
idr_remove(&submit->queue->fence_idr, submit->fence_id);
- mutex_unlock(&submit->queue->idr_lock);
+ spin_unlock(&submit->queue->idr_lock);
}
dma_fence_put(submit->user_fence);
@@ -882,7 +882,7 @@ int msm_ioctl_gem_submit(struct drm_device *dev, void *data,
submit->nr_cmds = i;
- mutex_lock(&queue->idr_lock);
+ spin_lock(&queue->idr_lock);
/*
* If using userspace provided seqno fence, validate that the id
@@ -892,7 +892,7 @@ int msm_ioctl_gem_submit(struct drm_device *dev, void *data,
*/
if ((args->flags & MSM_SUBMIT_FENCE_SN_IN) &&
idr_find(&queue->fence_idr, args->fence)) {
- mutex_unlock(&queue->idr_lock);
+ spin_unlock(&queue->idr_lock);
ret = -EINVAL;
goto out;
}
@@ -926,7 +926,7 @@ int msm_ioctl_gem_submit(struct drm_device *dev, void *data,
INT_MAX, GFP_KERNEL);
}
- mutex_unlock(&queue->idr_lock);
+ spin_unlock(&queue->idr_lock);
if (submit->fence_id < 0) {
ret = submit->fence_id;
diff --git a/drivers/gpu/drm/msm/msm_gpu.h b/drivers/gpu/drm/msm/msm_gpu.h
index fc1c0d8611a8..5929ecaa1fcd 100644
--- a/drivers/gpu/drm/msm/msm_gpu.h
+++ b/drivers/gpu/drm/msm/msm_gpu.h
@@ -499,7 +499,7 @@ struct msm_gpu_submitqueue {
struct msm_file_private *ctx;
struct list_head node;
struct idr fence_idr;
- struct mutex idr_lock;
+ struct spinlock idr_lock;
struct mutex lock;
struct kref ref;
struct drm_sched_entity *entity;
diff --git a/drivers/gpu/drm/msm/msm_submitqueue.c b/drivers/gpu/drm/msm/msm_submitqueue.c
index c6929e205b51..0e803125a325 100644
--- a/drivers/gpu/drm/msm/msm_submitqueue.c
+++ b/drivers/gpu/drm/msm/msm_submitqueue.c
@@ -200,7 +200,7 @@ int msm_submitqueue_create(struct drm_device *drm, struct msm_file_private *ctx,
*id = queue->id;
idr_init(&queue->fence_idr);
- mutex_init(&queue->idr_lock);
+ spin_lock_init(&queue->idr_lock);
mutex_init(&queue->lock);
list_add_tail(&queue->node, &ctx->submitqueues);
--
2.39.2
From: Rob Clark <[email protected]>
Now that everything that controls which LRU an obj lives in *except* the
backing pages is protected by the LRU lock, add a special path to unpin
in the job_run() path, we we are assured that we already have backing
pages and will not be racing against eviction (because the GEM object's
dma_resv contains the fence that will be signaled when the submit/job
completes).
Signed-off-by: Rob Clark <[email protected]>
---
drivers/gpu/drm/msm/msm_gem.c | 44 +++++++++++++++++++++++-----
drivers/gpu/drm/msm/msm_gem.h | 1 +
drivers/gpu/drm/msm/msm_ringbuffer.c | 4 +--
3 files changed, 39 insertions(+), 10 deletions(-)
diff --git a/drivers/gpu/drm/msm/msm_gem.c b/drivers/gpu/drm/msm/msm_gem.c
index d0ac3e704b66..9628e8d8dd02 100644
--- a/drivers/gpu/drm/msm/msm_gem.c
+++ b/drivers/gpu/drm/msm/msm_gem.c
@@ -61,18 +61,14 @@ static void sync_for_cpu(struct msm_gem_object *msm_obj)
dma_unmap_sgtable(dev, msm_obj->sgt, DMA_BIDIRECTIONAL, 0);
}
-static void update_lru_locked(struct drm_gem_object *obj)
+static void update_lru_active(struct drm_gem_object *obj)
{
struct msm_drm_private *priv = obj->dev->dev_private;
struct msm_gem_object *msm_obj = to_msm_bo(obj);
- msm_gem_assert_locked(&msm_obj->base);
-
- if (!msm_obj->pages) {
- GEM_WARN_ON(msm_obj->pin_count);
+ GEM_WARN_ON(!msm_obj->pages);
- drm_gem_lru_move_tail_locked(&priv->lru.unbacked, obj);
- } else if (msm_obj->pin_count) {
+ if (msm_obj->pin_count) {
drm_gem_lru_move_tail_locked(&priv->lru.pinned, obj);
} else if (msm_obj->madv == MSM_MADV_WILLNEED) {
drm_gem_lru_move_tail_locked(&priv->lru.willneed, obj);
@@ -83,6 +79,22 @@ static void update_lru_locked(struct drm_gem_object *obj)
}
}
+static void update_lru_locked(struct drm_gem_object *obj)
+{
+ struct msm_drm_private *priv = obj->dev->dev_private;
+ struct msm_gem_object *msm_obj = to_msm_bo(obj);
+
+ msm_gem_assert_locked(&msm_obj->base);
+
+ if (!msm_obj->pages) {
+ GEM_WARN_ON(msm_obj->pin_count);
+
+ drm_gem_lru_move_tail_locked(&priv->lru.unbacked, obj);
+ } else {
+ update_lru_active(obj);
+ }
+}
+
static void update_lru(struct drm_gem_object *obj)
{
struct msm_drm_private *priv = obj->dev->dev_private;
@@ -489,6 +501,24 @@ void msm_gem_unpin_locked(struct drm_gem_object *obj)
mutex_unlock(&priv->lru.lock);
}
+/* Special unpin path for use in fence-signaling path, avoiding the need
+ * to hold the obj lock by only depending on things that a protected by
+ * the LRU lock. In particular we know that that we already have backing
+ * and and that the object's dma_resv has the fence for the current
+ * submit/job which will prevent us racing against page eviction.
+ */
+void msm_gem_unpin_active(struct drm_gem_object *obj)
+{
+ struct msm_drm_private *priv = obj->dev->dev_private;
+ struct msm_gem_object *msm_obj = to_msm_bo(obj);
+
+ mutex_lock(&priv->lru.lock);
+ msm_obj->pin_count--;
+ GEM_WARN_ON(msm_obj->pin_count < 0);
+ update_lru_active(obj);
+ mutex_unlock(&priv->lru.lock);
+}
+
struct msm_gem_vma *msm_gem_get_vma_locked(struct drm_gem_object *obj,
struct msm_gem_address_space *aspace)
{
diff --git a/drivers/gpu/drm/msm/msm_gem.h b/drivers/gpu/drm/msm/msm_gem.h
index 0057e8e8fa13..2bd6846c83a9 100644
--- a/drivers/gpu/drm/msm/msm_gem.h
+++ b/drivers/gpu/drm/msm/msm_gem.h
@@ -128,6 +128,7 @@ struct msm_gem_object {
uint64_t msm_gem_mmap_offset(struct drm_gem_object *obj);
int msm_gem_pin_vma_locked(struct drm_gem_object *obj, struct msm_gem_vma *vma);
void msm_gem_unpin_locked(struct drm_gem_object *obj);
+void msm_gem_unpin_active(struct drm_gem_object *obj);
struct msm_gem_vma *msm_gem_get_vma_locked(struct drm_gem_object *obj,
struct msm_gem_address_space *aspace);
int msm_gem_get_iova(struct drm_gem_object *obj,
diff --git a/drivers/gpu/drm/msm/msm_ringbuffer.c b/drivers/gpu/drm/msm/msm_ringbuffer.c
index 31b4fbf96c36..b60199184409 100644
--- a/drivers/gpu/drm/msm/msm_ringbuffer.c
+++ b/drivers/gpu/drm/msm/msm_ringbuffer.c
@@ -24,9 +24,7 @@ static struct dma_fence *msm_job_run(struct drm_sched_job *job)
struct drm_gem_object *obj = &submit->bos[i].obj->base;
msm_gem_vma_unpin_fenced(submit->bos[i].vma, fctx);
- msm_gem_lock(obj);
- msm_gem_unpin_locked(obj);
- msm_gem_unlock(obj);
+ msm_gem_unpin_active(obj);
submit->bos[i].flags &= ~(BO_VMA_PINNED | BO_OBJ_PINNED);
}
--
2.39.2
From: Rob Clark <[email protected]>
Avoid allocation under idr_lock, to prevent deadlock against the
job_free() path (which runs on same thread as job_run(), which makes
it also part of the fence-signaling path.
Signed-off-by: Rob Clark <[email protected]>
---
drivers/gpu/drm/msm/msm_gem_submit.c | 8 ++++++--
1 file changed, 6 insertions(+), 2 deletions(-)
diff --git a/drivers/gpu/drm/msm/msm_gem_submit.c b/drivers/gpu/drm/msm/msm_gem_submit.c
index b9d81e5acb42..0ab62cb4ed69 100644
--- a/drivers/gpu/drm/msm/msm_gem_submit.c
+++ b/drivers/gpu/drm/msm/msm_gem_submit.c
@@ -882,6 +882,8 @@ int msm_ioctl_gem_submit(struct drm_device *dev, void *data,
submit->nr_cmds = i;
+ idr_preload(GFP_KERNEL);
+
spin_lock(&queue->idr_lock);
/*
@@ -893,6 +895,7 @@ int msm_ioctl_gem_submit(struct drm_device *dev, void *data,
if ((args->flags & MSM_SUBMIT_FENCE_SN_IN) &&
idr_find(&queue->fence_idr, args->fence)) {
spin_unlock(&queue->idr_lock);
+ idr_preload_end();
ret = -EINVAL;
goto out;
}
@@ -910,7 +913,7 @@ int msm_ioctl_gem_submit(struct drm_device *dev, void *data,
submit->fence_id = args->fence;
ret = idr_alloc_u32(&queue->fence_idr, submit->user_fence,
&submit->fence_id, submit->fence_id,
- GFP_KERNEL);
+ GFP_NOWAIT);
/*
* We've already validated that the fence_id slot is valid,
* so if idr_alloc_u32 failed, it is a kernel bug
@@ -923,10 +926,11 @@ int msm_ioctl_gem_submit(struct drm_device *dev, void *data,
*/
submit->fence_id = idr_alloc_cyclic(&queue->fence_idr,
submit->user_fence, 1,
- INT_MAX, GFP_KERNEL);
+ INT_MAX, GFP_NOWAIT);
}
spin_unlock(&queue->idr_lock);
+ idr_preload_end();
if (submit->fence_id < 0) {
ret = submit->fence_id;
--
2.39.2
From: Rob Clark <[email protected]>
It is already a no-op, since we've already loaded the fw from
adreno_load_gpu(), so drop the redundant call.
Signed-off-by: Rob Clark <[email protected]>
---
drivers/gpu/drm/msm/adreno/adreno_gpu.c | 9 +--------
1 file changed, 1 insertion(+), 8 deletions(-)
diff --git a/drivers/gpu/drm/msm/adreno/adreno_gpu.c b/drivers/gpu/drm/msm/adreno/adreno_gpu.c
index 817599766329..28cc5685ba96 100644
--- a/drivers/gpu/drm/msm/adreno/adreno_gpu.c
+++ b/drivers/gpu/drm/msm/adreno/adreno_gpu.c
@@ -503,16 +503,9 @@ struct drm_gem_object *adreno_fw_create_bo(struct msm_gpu *gpu,
int adreno_hw_init(struct msm_gpu *gpu)
{
- struct adreno_gpu *adreno_gpu = to_adreno_gpu(gpu);
- int ret, i;
-
VERB("%s", gpu->name);
- ret = adreno_load_fw(adreno_gpu);
- if (ret)
- return ret;
-
- for (i = 0; i < gpu->nr_rings; i++) {
+ for (int i = 0; i < gpu->nr_rings; i++) {
struct msm_ringbuffer *ring = gpu->rb[i];
if (!ring)
--
2.39.2
From: Rob Clark <[email protected]>
These allocations are only done the first (successful) time through
hw_init() so they won't actually happen in the job_run() path. But
lockdep doesn't know this. So dis-entangle them from the hw_init()
path.
Signed-off-by: Rob Clark <[email protected]>
---
drivers/gpu/drm/msm/adreno/a5xx_gpu.c | 48 +++++++++++-----------
drivers/gpu/drm/msm/adreno/a6xx_gpu.c | 46 ++++++++++-----------
drivers/gpu/drm/msm/adreno/adreno_device.c | 6 +++
drivers/gpu/drm/msm/msm_gpu.h | 6 +++
4 files changed, 57 insertions(+), 49 deletions(-)
diff --git a/drivers/gpu/drm/msm/adreno/a5xx_gpu.c b/drivers/gpu/drm/msm/adreno/a5xx_gpu.c
index 660ba0db8900..f8e278d46dcf 100644
--- a/drivers/gpu/drm/msm/adreno/a5xx_gpu.c
+++ b/drivers/gpu/drm/msm/adreno/a5xx_gpu.c
@@ -567,7 +567,7 @@ static void a5xx_ucode_check_version(struct a5xx_gpu *a5xx_gpu,
msm_gem_put_vaddr(obj);
}
-static int a5xx_ucode_init(struct msm_gpu *gpu)
+static int a5xx_ucode_load(struct msm_gpu *gpu)
{
struct adreno_gpu *adreno_gpu = to_adreno_gpu(gpu);
struct a5xx_gpu *a5xx_gpu = to_a5xx_gpu(adreno_gpu);
@@ -605,9 +605,24 @@ static int a5xx_ucode_init(struct msm_gpu *gpu)
a5xx_ucode_check_version(a5xx_gpu, a5xx_gpu->pfp_bo);
}
- gpu_write64(gpu, REG_A5XX_CP_ME_INSTR_BASE_LO, a5xx_gpu->pm4_iova);
+ if (a5xx_gpu->has_whereami) {
+ if (!a5xx_gpu->shadow_bo) {
+ a5xx_gpu->shadow = msm_gem_kernel_new(gpu->dev,
+ sizeof(u32) * gpu->nr_rings,
+ MSM_BO_WC | MSM_BO_MAP_PRIV,
+ gpu->aspace, &a5xx_gpu->shadow_bo,
+ &a5xx_gpu->shadow_iova);
- gpu_write64(gpu, REG_A5XX_CP_PFP_INSTR_BASE_LO, a5xx_gpu->pfp_iova);
+ if (IS_ERR(a5xx_gpu->shadow))
+ return PTR_ERR(a5xx_gpu->shadow);
+
+ msm_gem_object_set_name(a5xx_gpu->shadow_bo, "shadow");
+ }
+ } else if (gpu->nr_rings > 1) {
+ /* Disable preemption if WHERE_AM_I isn't available */
+ a5xx_preempt_fini(gpu);
+ gpu->nr_rings = 1;
+ }
return 0;
}
@@ -900,9 +915,8 @@ static int a5xx_hw_init(struct msm_gpu *gpu)
if (adreno_is_a530(adreno_gpu) || adreno_is_a540(adreno_gpu))
a5xx_gpmu_ucode_init(gpu);
- ret = a5xx_ucode_init(gpu);
- if (ret)
- return ret;
+ gpu_write64(gpu, REG_A5XX_CP_ME_INSTR_BASE_LO, a5xx_gpu->pm4_iova);
+ gpu_write64(gpu, REG_A5XX_CP_PFP_INSTR_BASE_LO, a5xx_gpu->pfp_iova);
/* Set the ringbuffer address */
gpu_write64(gpu, REG_A5XX_CP_RB_BASE, gpu->rb[0]->iova);
@@ -916,27 +930,10 @@ static int a5xx_hw_init(struct msm_gpu *gpu)
gpu_write(gpu, REG_A5XX_CP_RB_CNTL,
MSM_GPU_RB_CNTL_DEFAULT | AXXX_CP_RB_CNTL_NO_UPDATE);
- /* Create a privileged buffer for the RPTR shadow */
- if (a5xx_gpu->has_whereami) {
- if (!a5xx_gpu->shadow_bo) {
- a5xx_gpu->shadow = msm_gem_kernel_new(gpu->dev,
- sizeof(u32) * gpu->nr_rings,
- MSM_BO_WC | MSM_BO_MAP_PRIV,
- gpu->aspace, &a5xx_gpu->shadow_bo,
- &a5xx_gpu->shadow_iova);
-
- if (IS_ERR(a5xx_gpu->shadow))
- return PTR_ERR(a5xx_gpu->shadow);
-
- msm_gem_object_set_name(a5xx_gpu->shadow_bo, "shadow");
- }
-
+ /* Configure the RPTR shadow if needed: */
+ if (a5xx_gpu->shadow_bo) {
gpu_write64(gpu, REG_A5XX_CP_RB_RPTR_ADDR,
shadowptr(a5xx_gpu, gpu->rb[0]));
- } else if (gpu->nr_rings > 1) {
- /* Disable preemption if WHERE_AM_I isn't available */
- a5xx_preempt_fini(gpu);
- gpu->nr_rings = 1;
}
a5xx_preempt_hw_init(gpu);
@@ -1682,6 +1679,7 @@ static const struct adreno_gpu_funcs funcs = {
.get_param = adreno_get_param,
.set_param = adreno_set_param,
.hw_init = a5xx_hw_init,
+ .ucode_load = a5xx_ucode_load,
.pm_suspend = a5xx_pm_suspend,
.pm_resume = a5xx_pm_resume,
.recover = a5xx_recover,
diff --git a/drivers/gpu/drm/msm/adreno/a6xx_gpu.c b/drivers/gpu/drm/msm/adreno/a6xx_gpu.c
index aae60cbd9164..89049094a242 100644
--- a/drivers/gpu/drm/msm/adreno/a6xx_gpu.c
+++ b/drivers/gpu/drm/msm/adreno/a6xx_gpu.c
@@ -917,7 +917,7 @@ static bool a6xx_ucode_check_version(struct a6xx_gpu *a6xx_gpu,
return ret;
}
-static int a6xx_ucode_init(struct msm_gpu *gpu)
+static int a6xx_ucode_load(struct msm_gpu *gpu)
{
struct adreno_gpu *adreno_gpu = to_adreno_gpu(gpu);
struct a6xx_gpu *a6xx_gpu = to_a6xx_gpu(adreno_gpu);
@@ -946,7 +946,23 @@ static int a6xx_ucode_init(struct msm_gpu *gpu)
}
}
- gpu_write64(gpu, REG_A6XX_CP_SQE_INSTR_BASE, a6xx_gpu->sqe_iova);
+ /*
+ * Expanded APRIV and targets that support WHERE_AM_I both need a
+ * privileged buffer to store the RPTR shadow
+ */
+ if ((adreno_gpu->base.hw_apriv || a6xx_gpu->has_whereami) &&
+ !a6xx_gpu->shadow_bo) {
+ a6xx_gpu->shadow = msm_gem_kernel_new(gpu->dev,
+ sizeof(u32) * gpu->nr_rings,
+ MSM_BO_WC | MSM_BO_MAP_PRIV,
+ gpu->aspace, &a6xx_gpu->shadow_bo,
+ &a6xx_gpu->shadow_iova);
+
+ if (IS_ERR(a6xx_gpu->shadow))
+ return PTR_ERR(a6xx_gpu->shadow);
+
+ msm_gem_object_set_name(a6xx_gpu->shadow_bo, "shadow");
+ }
return 0;
}
@@ -1135,9 +1151,7 @@ static int hw_init(struct msm_gpu *gpu)
if (ret)
goto out;
- ret = a6xx_ucode_init(gpu);
- if (ret)
- goto out;
+ gpu_write64(gpu, REG_A6XX_CP_SQE_INSTR_BASE, a6xx_gpu->sqe_iova);
/* Set the ringbuffer address */
gpu_write64(gpu, REG_A6XX_CP_RB_BASE, gpu->rb[0]->iova);
@@ -1152,25 +1166,8 @@ static int hw_init(struct msm_gpu *gpu)
gpu_write(gpu, REG_A6XX_CP_RB_CNTL,
MSM_GPU_RB_CNTL_DEFAULT | AXXX_CP_RB_CNTL_NO_UPDATE);
- /*
- * Expanded APRIV and targets that support WHERE_AM_I both need a
- * privileged buffer to store the RPTR shadow
- */
-
- if (adreno_gpu->base.hw_apriv || a6xx_gpu->has_whereami) {
- if (!a6xx_gpu->shadow_bo) {
- a6xx_gpu->shadow = msm_gem_kernel_new(gpu->dev,
- sizeof(u32) * gpu->nr_rings,
- MSM_BO_WC | MSM_BO_MAP_PRIV,
- gpu->aspace, &a6xx_gpu->shadow_bo,
- &a6xx_gpu->shadow_iova);
-
- if (IS_ERR(a6xx_gpu->shadow))
- return PTR_ERR(a6xx_gpu->shadow);
-
- msm_gem_object_set_name(a6xx_gpu->shadow_bo, "shadow");
- }
-
+ /* Configure the RPTR shadow if needed: */
+ if (a6xx_gpu->shadow_bo) {
gpu_write64(gpu, REG_A6XX_CP_RB_RPTR_ADDR_LO,
shadowptr(a6xx_gpu, gpu->rb[0]));
}
@@ -1952,6 +1949,7 @@ static const struct adreno_gpu_funcs funcs = {
.get_param = adreno_get_param,
.set_param = adreno_set_param,
.hw_init = a6xx_hw_init,
+ .ucode_load = a6xx_ucode_load,
.pm_suspend = a6xx_pm_suspend,
.pm_resume = a6xx_pm_resume,
.recover = a6xx_recover,
diff --git a/drivers/gpu/drm/msm/adreno/adreno_device.c b/drivers/gpu/drm/msm/adreno/adreno_device.c
index 36f062c7582f..83d89b8d93e4 100644
--- a/drivers/gpu/drm/msm/adreno/adreno_device.c
+++ b/drivers/gpu/drm/msm/adreno/adreno_device.c
@@ -432,6 +432,12 @@ struct msm_gpu *adreno_load_gpu(struct drm_device *dev)
if (ret)
return NULL;
+ if (gpu->funcs->ucode_load) {
+ ret = gpu->funcs->ucode_load(gpu);
+ if (ret)
+ return NULL;
+ }
+
/*
* Now that we have firmware loaded, and are ready to begin
* booting the gpu, go ahead and enable runpm:
diff --git a/drivers/gpu/drm/msm/msm_gpu.h b/drivers/gpu/drm/msm/msm_gpu.h
index 5929ecaa1fcd..f84de0e8afac 100644
--- a/drivers/gpu/drm/msm/msm_gpu.h
+++ b/drivers/gpu/drm/msm/msm_gpu.h
@@ -50,6 +50,12 @@ struct msm_gpu_funcs {
int (*set_param)(struct msm_gpu *gpu, struct msm_file_private *ctx,
uint32_t param, uint64_t value, uint32_t len);
int (*hw_init)(struct msm_gpu *gpu);
+
+ /**
+ * @ucode_load: Optional hook to upload fw to GEM objs
+ */
+ int (*ucode_load)(struct msm_gpu *gpu);
+
int (*pm_suspend)(struct msm_gpu *gpu);
int (*pm_resume)(struct msm_gpu *gpu);
void (*submit)(struct msm_gpu *gpu, struct msm_gem_submit *submit);
--
2.39.2
From: Rob Clark <[email protected]>
In the process of adding lockdep annotation for GPU job_run() path to
catch potential deadlocks against the shrinker/reclaim path, I turned
up this lockdep splat:
======================================================
WARNING: possible circular locking dependency detected
6.2.0-rc8-debug+ #556 Not tainted
------------------------------------------------------
ring0/123 is trying to acquire lock:
ffffff8087219078 (&devfreq->lock){+.+.}-{3:3}, at: devfreq_monitor_resume+0x3c/0xf0
but task is already holding lock:
ffffffd6f64e57e8 (dma_fence_map){++++}-{0:0}, at: msm_job_run+0x68/0x150
which lock already depends on the new lock.
the existing dependency chain (in reverse order) is:
-> #3 (dma_fence_map){++++}-{0:0}:
__dma_fence_might_wait+0x74/0xc0
dma_resv_lockdep+0x1f4/0x2f4
do_one_initcall+0x104/0x2bc
kernel_init_freeable+0x344/0x34c
kernel_init+0x30/0x134
ret_from_fork+0x10/0x20
-> #2 (mmu_notifier_invalidate_range_start){+.+.}-{0:0}:
fs_reclaim_acquire+0x80/0xa8
slab_pre_alloc_hook.constprop.0+0x40/0x25c
__kmem_cache_alloc_node+0x60/0x1cc
__kmalloc+0xd8/0x100
topology_parse_cpu_capacity+0x8c/0x178
get_cpu_for_node+0x88/0xc4
parse_cluster+0x1b0/0x28c
parse_cluster+0x8c/0x28c
init_cpu_topology+0x168/0x188
smp_prepare_cpus+0x24/0xf8
kernel_init_freeable+0x18c/0x34c
kernel_init+0x30/0x134
ret_from_fork+0x10/0x20
-> #1 (fs_reclaim){+.+.}-{0:0}:
__fs_reclaim_acquire+0x3c/0x48
fs_reclaim_acquire+0x54/0xa8
slab_pre_alloc_hook.constprop.0+0x40/0x25c
__kmem_cache_alloc_node+0x60/0x1cc
__kmalloc_node_track_caller+0xb8/0xe0
kstrdup+0x70/0x90
kstrdup_const+0x38/0x48
kvasprintf_const+0x48/0xbc
kobject_set_name_vargs+0x40/0xb0
dev_set_name+0x64/0x8c
devfreq_add_device+0x31c/0x55c
devm_devfreq_add_device+0x6c/0xb8
msm_devfreq_init+0xa8/0x16c
msm_gpu_init+0x38c/0x570
adreno_gpu_init+0x1b4/0x2b4
a6xx_gpu_init+0x15c/0x3e4
adreno_bind+0x218/0x254
component_bind_all+0x114/0x1ec
msm_drm_bind+0x2b8/0x608
try_to_bring_up_aggregate_device+0x88/0x1a4
__component_add+0xec/0x13c
component_add+0x1c/0x28
dsi_dev_attach+0x28/0x34
dsi_host_attach+0xdc/0x124
mipi_dsi_attach+0x30/0x44
devm_mipi_dsi_attach+0x2c/0x70
ti_sn_bridge_probe+0x298/0x2c4
auxiliary_bus_probe+0x7c/0x94
really_probe+0x158/0x290
__driver_probe_device+0xc8/0xe0
driver_probe_device+0x44/0x100
__device_attach_driver+0x64/0xdc
bus_for_each_drv+0xa0/0xc8
__device_attach+0xd8/0x168
device_initial_probe+0x1c/0x28
bus_probe_device+0x38/0xa0
deferred_probe_work_func+0xc8/0xe0
process_one_work+0x2d8/0x478
process_scheduled_works+0x4c/0x50
worker_thread+0x218/0x274
kthread+0xf0/0x100
ret_from_fork+0x10/0x20
-> #0 (&devfreq->lock){+.+.}-{3:3}:
__lock_acquire+0xe00/0x1060
lock_acquire+0x1e0/0x2f8
__mutex_lock+0xcc/0x3c8
mutex_lock_nested+0x30/0x44
devfreq_monitor_resume+0x3c/0xf0
devfreq_simple_ondemand_handler+0x54/0x7c
devfreq_resume_device+0xa4/0xe8
msm_devfreq_resume+0x78/0xa8
a6xx_pm_resume+0x110/0x234
adreno_runtime_resume+0x2c/0x38
pm_generic_runtime_resume+0x30/0x44
__rpm_callback+0x15c/0x174
rpm_callback+0x78/0x7c
rpm_resume+0x318/0x524
__pm_runtime_resume+0x78/0xbc
pm_runtime_get_sync.isra.0+0x14/0x20
msm_gpu_submit+0x58/0x178
msm_job_run+0x78/0x150
drm_sched_main+0x290/0x370
kthread+0xf0/0x100
ret_from_fork+0x10/0x20
other info that might help us debug this:
Chain exists of:
&devfreq->lock --> mmu_notifier_invalidate_range_start --> dma_fence_map
Possible unsafe locking scenario:
CPU0 CPU1
---- ----
lock(dma_fence_map);
lock(mmu_notifier_invalidate_range_start);
lock(dma_fence_map);
lock(&devfreq->lock);
*** DEADLOCK ***
2 locks held by ring0/123:
#0: ffffff8087201170 (&gpu->lock){+.+.}-{3:3}, at: msm_job_run+0x64/0x150
#1: ffffffd6f64e57e8 (dma_fence_map){++++}-{0:0}, at: msm_job_run+0x68/0x150
stack backtrace:
CPU: 6 PID: 123 Comm: ring0 Not tainted 6.2.0-rc8-debug+ #556
Hardware name: Google Lazor (rev1 - 2) with LTE (DT)
Call trace:
dump_backtrace.part.0+0xb4/0xf8
show_stack+0x20/0x38
dump_stack_lvl+0x9c/0xd0
dump_stack+0x18/0x34
print_circular_bug+0x1b4/0x1f0
check_noncircular+0x78/0xac
__lock_acquire+0xe00/0x1060
lock_acquire+0x1e0/0x2f8
__mutex_lock+0xcc/0x3c8
mutex_lock_nested+0x30/0x44
devfreq_monitor_resume+0x3c/0xf0
devfreq_simple_ondemand_handler+0x54/0x7c
devfreq_resume_device+0xa4/0xe8
msm_devfreq_resume+0x78/0xa8
a6xx_pm_resume+0x110/0x234
adreno_runtime_resume+0x2c/0x38
pm_generic_runtime_resume+0x30/0x44
__rpm_callback+0x15c/0x174
rpm_callback+0x78/0x7c
rpm_resume+0x318/0x524
__pm_runtime_resume+0x78/0xbc
pm_runtime_get_sync.isra.0+0x14/0x20
msm_gpu_submit+0x58/0x178
msm_job_run+0x78/0x150
drm_sched_main+0x290/0x370
kthread+0xf0/0x100
ret_from_fork+0x10/0x20
The issue is that we cannot be holding any lock while doing memory
allocations that is also needed in the job_run (and in the case of
devfreq, this means runpm_resume()) because lockdep sees this as a
potential dependency.
Fortunately there is really no reason to hold the devfreq lock when
we are creating the devfreq device, as it is not yet visible to any
other task. The only reason it was needed was for a lockdep assert
in devfreq_get_freq_range(). Instead, split this up into an internal
fxn that is used in the devfreq_add_device() (where the lock is not
required).
Signed-off-by: Rob Clark <[email protected]>
---
drivers/devfreq/devfreq.c | 46 ++++++++++++++++++---------------------
1 file changed, 21 insertions(+), 25 deletions(-)
diff --git a/drivers/devfreq/devfreq.c b/drivers/devfreq/devfreq.c
index 817c71da391a..11b774048bd2 100644
--- a/drivers/devfreq/devfreq.c
+++ b/drivers/devfreq/devfreq.c
@@ -111,23 +111,13 @@ static unsigned long find_available_max_freq(struct devfreq *devfreq)
return max_freq;
}
-/**
- * devfreq_get_freq_range() - Get the current freq range
- * @devfreq: the devfreq instance
- * @min_freq: the min frequency
- * @max_freq: the max frequency
- *
- * This takes into consideration all constraints.
- */
-void devfreq_get_freq_range(struct devfreq *devfreq,
- unsigned long *min_freq,
- unsigned long *max_freq)
+static void __get_freq_range(struct devfreq *devfreq,
+ unsigned long *min_freq,
+ unsigned long *max_freq)
{
unsigned long *freq_table = devfreq->freq_table;
s32 qos_min_freq, qos_max_freq;
- lockdep_assert_held(&devfreq->lock);
-
/*
* Initialize minimum/maximum frequency from freq table.
* The devfreq drivers can initialize this in either ascending or
@@ -158,6 +148,23 @@ void devfreq_get_freq_range(struct devfreq *devfreq,
if (*min_freq > *max_freq)
*min_freq = *max_freq;
}
+
+/**
+ * devfreq_get_freq_range() - Get the current freq range
+ * @devfreq: the devfreq instance
+ * @min_freq: the min frequency
+ * @max_freq: the max frequency
+ *
+ * This takes into consideration all constraints.
+ */
+void devfreq_get_freq_range(struct devfreq *devfreq,
+ unsigned long *min_freq,
+ unsigned long *max_freq)
+{
+ lockdep_assert_held(&devfreq->lock);
+
+ __get_freq_range(devfreq, min_freq, max_freq);
+}
EXPORT_SYMBOL(devfreq_get_freq_range);
/**
@@ -810,7 +817,6 @@ struct devfreq *devfreq_add_device(struct device *dev,
}
mutex_init(&devfreq->lock);
- mutex_lock(&devfreq->lock);
devfreq->dev.parent = dev;
devfreq->dev.class = devfreq_class;
devfreq->dev.release = devfreq_dev_release;
@@ -823,17 +829,14 @@ struct devfreq *devfreq_add_device(struct device *dev,
if (devfreq->profile->timer < 0
|| devfreq->profile->timer >= DEVFREQ_TIMER_NUM) {
- mutex_unlock(&devfreq->lock);
err = -EINVAL;
goto err_dev;
}
if (!devfreq->profile->max_state || !devfreq->profile->freq_table) {
- mutex_unlock(&devfreq->lock);
err = set_freq_table(devfreq);
if (err < 0)
goto err_dev;
- mutex_lock(&devfreq->lock);
} else {
devfreq->freq_table = devfreq->profile->freq_table;
devfreq->max_state = devfreq->profile->max_state;
@@ -841,19 +844,17 @@ struct devfreq *devfreq_add_device(struct device *dev,
devfreq->scaling_min_freq = find_available_min_freq(devfreq);
if (!devfreq->scaling_min_freq) {
- mutex_unlock(&devfreq->lock);
err = -EINVAL;
goto err_dev;
}
devfreq->scaling_max_freq = find_available_max_freq(devfreq);
if (!devfreq->scaling_max_freq) {
- mutex_unlock(&devfreq->lock);
err = -EINVAL;
goto err_dev;
}
- devfreq_get_freq_range(devfreq, &min_freq, &max_freq);
+ __get_freq_range(devfreq, &min_freq, &max_freq);
devfreq->suspend_freq = dev_pm_opp_get_suspend_opp_freq(dev);
devfreq->opp_table = dev_pm_opp_get_opp_table(dev);
@@ -865,7 +866,6 @@ struct devfreq *devfreq_add_device(struct device *dev,
dev_set_name(&devfreq->dev, "%s", dev_name(dev));
err = device_register(&devfreq->dev);
if (err) {
- mutex_unlock(&devfreq->lock);
put_device(&devfreq->dev);
goto err_out;
}
@@ -876,7 +876,6 @@ struct devfreq *devfreq_add_device(struct device *dev,
devfreq->max_state),
GFP_KERNEL);
if (!devfreq->stats.trans_table) {
- mutex_unlock(&devfreq->lock);
err = -ENOMEM;
goto err_devfreq;
}
@@ -886,7 +885,6 @@ struct devfreq *devfreq_add_device(struct device *dev,
sizeof(*devfreq->stats.time_in_state),
GFP_KERNEL);
if (!devfreq->stats.time_in_state) {
- mutex_unlock(&devfreq->lock);
err = -ENOMEM;
goto err_devfreq;
}
@@ -896,8 +894,6 @@ struct devfreq *devfreq_add_device(struct device *dev,
srcu_init_notifier_head(&devfreq->transition_notifier_list);
- mutex_unlock(&devfreq->lock);
-
err = dev_pm_qos_add_request(dev, &devfreq->user_min_freq_req,
DEV_PM_QOS_MIN_FREQUENCY, 0);
if (err < 0)
--
2.39.2
From: Rob Clark <[email protected]>
Move the one-time RPMh setup to a6xx_gmu_init(). To get rid of the hack
for one-time init vs start, add in an extra a6xx_rpmh_stop() at the end
of the init sequence.
Signed-off-by: Rob Clark <[email protected]>
---
drivers/gpu/drm/msm/adreno/a6xx_gmu.c | 18 ++++++++----------
1 file changed, 8 insertions(+), 10 deletions(-)
diff --git a/drivers/gpu/drm/msm/adreno/a6xx_gmu.c b/drivers/gpu/drm/msm/adreno/a6xx_gmu.c
index f3c9600221d4..30a1bf39ea83 100644
--- a/drivers/gpu/drm/msm/adreno/a6xx_gmu.c
+++ b/drivers/gpu/drm/msm/adreno/a6xx_gmu.c
@@ -621,6 +621,8 @@ static void a6xx_gmu_rpmh_init(struct a6xx_gmu *gmu)
/* ensure no writes happen before the uCode is fully written */
wmb();
+ a6xx_rpmh_stop(gmu);
+
err:
if (!IS_ERR_OR_NULL(pdcptr))
iounmap(pdcptr);
@@ -753,7 +755,6 @@ static int a6xx_gmu_fw_load(struct a6xx_gmu *gmu)
static int a6xx_gmu_fw_start(struct a6xx_gmu *gmu, unsigned int state)
{
- static bool rpmh_init;
struct a6xx_gpu *a6xx_gpu = container_of(gmu, struct a6xx_gpu, gmu);
struct adreno_gpu *adreno_gpu = &a6xx_gpu->base;
int ret;
@@ -776,15 +777,9 @@ static int a6xx_gmu_fw_start(struct a6xx_gmu *gmu, unsigned int state)
/* Turn on register retention */
gmu_write(gmu, REG_A6XX_GMU_GENERAL_7, 1);
- /* We only need to load the RPMh microcode once */
- if (!rpmh_init) {
- a6xx_gmu_rpmh_init(gmu);
- rpmh_init = true;
- } else {
- ret = a6xx_rpmh_start(gmu);
- if (ret)
- return ret;
- }
+ ret = a6xx_rpmh_start(gmu);
+ if (ret)
+ return ret;
ret = a6xx_gmu_fw_load(gmu);
if (ret)
@@ -1633,6 +1628,9 @@ int a6xx_gmu_init(struct a6xx_gpu *a6xx_gpu, struct device_node *node)
/* Set up the HFI queues */
a6xx_hfi_init(gmu);
+ /* Initialize RPMh */
+ a6xx_gmu_rpmh_init(gmu);
+
gmu->initialized = true;
return 0;
--
2.39.2
From: Rob Clark <[email protected]>
This will make it easier to catch places doing allocations that can
trigger reclaim under devfreq->lock.
Because devfreq->lock is held over various devfreq_dev_profile
callbacks, there might be some fallout if those callbacks do allocations
that can trigger reclaim, but I've looked through the various callback
implementations and don't see anything obvious. If it does trigger any
lockdep splats, those should be fixed.
Signed-off-by: Rob Clark <[email protected]>
---
drivers/devfreq/devfreq.c | 6 ++++++
1 file changed, 6 insertions(+)
diff --git a/drivers/devfreq/devfreq.c b/drivers/devfreq/devfreq.c
index 11b774048bd2..5ce3bf9b59e7 100644
--- a/drivers/devfreq/devfreq.c
+++ b/drivers/devfreq/devfreq.c
@@ -817,6 +817,12 @@ struct devfreq *devfreq_add_device(struct device *dev,
}
mutex_init(&devfreq->lock);
+
+ /* Teach lockdep about lock ordering wrt. shrinker: */
+ fs_reclaim_acquire(GFP_KERNEL);
+ might_lock(&devfreq->lock);
+ fs_reclaim_release(GFP_KERNEL);
+
devfreq->dev.parent = dev;
devfreq->dev.class = devfreq_class;
devfreq->dev.release = devfreq_dev_release;
--
2.39.2
From: Rob Clark <[email protected]>
Annotate dev_pm_qos_mtx to teach lockdep to scream about allocations
that could trigger reclaim under dev_pm_qos_mtx.
Signed-off-by: Rob Clark <[email protected]>
---
drivers/base/power/qos.c | 11 +++++++++++
1 file changed, 11 insertions(+)
diff --git a/drivers/base/power/qos.c b/drivers/base/power/qos.c
index 9cba334b3729..d4addda3944a 100644
--- a/drivers/base/power/qos.c
+++ b/drivers/base/power/qos.c
@@ -1012,3 +1012,14 @@ void dev_pm_qos_hide_latency_tolerance(struct device *dev)
pm_runtime_put(dev);
}
EXPORT_SYMBOL_GPL(dev_pm_qos_hide_latency_tolerance);
+
+static int __init dev_pm_qos_init(void)
+{
+ /* Teach lockdep about lock ordering wrt. shrinker: */
+ fs_reclaim_acquire(GFP_KERNEL);
+ might_lock(&dev_pm_qos_mtx);
+ fs_reclaim_release(GFP_KERNEL);
+
+ return 0;
+}
+early_initcall(dev_pm_qos_init);
--
2.39.2
From: Rob Clark <[email protected]>
Preparing for better lockdep annotations for things that happen in runpm
suspend/resume path vs shrinker/reclaim in the following patches, we
need to avoid allocations that can trigger reclaim in the icc_set_bw()
path. In the RPMh case, rpmh_write_batch() already uses GFP_ATOMIC, so
it should be reasonable to use in the smd-rpm case as well.
Alternatively, 256bytes is small enough for a function that isn't called
recursively to allocate on-stack.
Signed-off-by: Rob Clark <[email protected]>
---
drivers/soc/qcom/smd-rpm.c | 2 +-
1 file changed, 1 insertion(+), 1 deletion(-)
diff --git a/drivers/soc/qcom/smd-rpm.c b/drivers/soc/qcom/smd-rpm.c
index 7e3b6a7ea34c..478da981d9fb 100644
--- a/drivers/soc/qcom/smd-rpm.c
+++ b/drivers/soc/qcom/smd-rpm.c
@@ -113,7 +113,7 @@ int qcom_rpm_smd_write(struct qcom_smd_rpm *rpm,
if (WARN_ON(size >= 256))
return -EINVAL;
- pkt = kmalloc(size, GFP_KERNEL);
+ pkt = kmalloc(size, GFP_ATOMIC);
if (!pkt)
return -ENOMEM;
--
2.39.2
From: Rob Clark <[email protected]>
In the process of adding lockdep annotation for drm GPU scheduler's
job_run() to detect potential deadlock against shrinker/reclaim, I hit
this lockdep splat:
======================================================
WARNING: possible circular locking dependency detected
6.2.0-rc8-debug+ #558 Tainted: G W
------------------------------------------------------
ring0/125 is trying to acquire lock:
ffffffd6d6ce0f28 (dev_pm_qos_mtx){+.+.}-{3:3}, at: dev_pm_qos_update_request+0x38/0x68
but task is already holding lock:
ffffff8087239208 (&gpu->active_lock){+.+.}-{3:3}, at: msm_gpu_submit+0xec/0x178
which lock already depends on the new lock.
the existing dependency chain (in reverse order) is:
-> #4 (&gpu->active_lock){+.+.}-{3:3}:
__mutex_lock+0xcc/0x3c8
mutex_lock_nested+0x30/0x44
msm_gpu_submit+0xec/0x178
msm_job_run+0x78/0x150
drm_sched_main+0x290/0x370
kthread+0xf0/0x100
ret_from_fork+0x10/0x20
-> #3 (dma_fence_map){++++}-{0:0}:
__dma_fence_might_wait+0x74/0xc0
dma_resv_lockdep+0x1f4/0x2f4
do_one_initcall+0x104/0x2bc
kernel_init_freeable+0x344/0x34c
kernel_init+0x30/0x134
ret_from_fork+0x10/0x20
-> #2 (mmu_notifier_invalidate_range_start){+.+.}-{0:0}:
fs_reclaim_acquire+0x80/0xa8
slab_pre_alloc_hook.constprop.0+0x40/0x25c
__kmem_cache_alloc_node+0x60/0x1cc
__kmalloc+0xd8/0x100
topology_parse_cpu_capacity+0x8c/0x178
get_cpu_for_node+0x88/0xc4
parse_cluster+0x1b0/0x28c
parse_cluster+0x8c/0x28c
init_cpu_topology+0x168/0x188
smp_prepare_cpus+0x24/0xf8
kernel_init_freeable+0x18c/0x34c
kernel_init+0x30/0x134
ret_from_fork+0x10/0x20
-> #1 (fs_reclaim){+.+.}-{0:0}:
__fs_reclaim_acquire+0x3c/0x48
fs_reclaim_acquire+0x54/0xa8
slab_pre_alloc_hook.constprop.0+0x40/0x25c
__kmem_cache_alloc_node+0x60/0x1cc
kmalloc_trace+0x50/0xa8
dev_pm_qos_constraints_allocate+0x38/0x100
__dev_pm_qos_add_request+0xb0/0x1e8
dev_pm_qos_add_request+0x58/0x80
dev_pm_qos_expose_latency_limit+0x60/0x13c
register_cpu+0x12c/0x130
topology_init+0xac/0xbc
do_one_initcall+0x104/0x2bc
kernel_init_freeable+0x344/0x34c
kernel_init+0x30/0x134
ret_from_fork+0x10/0x20
-> #0 (dev_pm_qos_mtx){+.+.}-{3:3}:
__lock_acquire+0xe00/0x1060
lock_acquire+0x1e0/0x2f8
__mutex_lock+0xcc/0x3c8
mutex_lock_nested+0x30/0x44
dev_pm_qos_update_request+0x38/0x68
msm_devfreq_boost+0x40/0x70
msm_devfreq_active+0xc0/0xf0
msm_gpu_submit+0x10c/0x178
msm_job_run+0x78/0x150
drm_sched_main+0x290/0x370
kthread+0xf0/0x100
ret_from_fork+0x10/0x20
other info that might help us debug this:
Chain exists of:
dev_pm_qos_mtx --> dma_fence_map --> &gpu->active_lock
Possible unsafe locking scenario:
CPU0 CPU1
---- ----
lock(&gpu->active_lock);
lock(dma_fence_map);
lock(&gpu->active_lock);
lock(dev_pm_qos_mtx);
*** DEADLOCK ***
3 locks held by ring0/123:
#0: ffffff8087251170 (&gpu->lock){+.+.}-{3:3}, at: msm_job_run+0x64/0x150
#1: ffffffd00b0e57e8 (dma_fence_map){++++}-{0:0}, at: msm_job_run+0x68/0x150
#2: ffffff8087251208 (&gpu->active_lock){+.+.}-{3:3}, at: msm_gpu_submit+0xec/0x178
stack backtrace:
CPU: 6 PID: 123 Comm: ring0 Not tainted 6.2.0-rc8-debug+ #559
Hardware name: Google Lazor (rev1 - 2) with LTE (DT)
Call trace:
dump_backtrace.part.0+0xb4/0xf8
show_stack+0x20/0x38
dump_stack_lvl+0x9c/0xd0
dump_stack+0x18/0x34
print_circular_bug+0x1b4/0x1f0
check_noncircular+0x78/0xac
__lock_acquire+0xe00/0x1060
lock_acquire+0x1e0/0x2f8
__mutex_lock+0xcc/0x3c8
mutex_lock_nested+0x30/0x44
dev_pm_qos_update_request+0x38/0x68
msm_devfreq_boost+0x40/0x70
msm_devfreq_active+0xc0/0xf0
msm_gpu_submit+0x10c/0x178
msm_job_run+0x78/0x150
drm_sched_main+0x290/0x370
kthread+0xf0/0x100
ret_from_fork+0x10/0x20
The issue is that dev_pm_qos_mtx is held in the runpm suspend/resume (or
freq change) path, but it is also held across allocations that could
recurse into shrinker.
Solve this by changing dev_pm_qos_constraints_allocate() into a function
that can be called unconditionally before the device qos object is
needed and before aquiring dev_pm_qos_mtx. This way the allocations can
be done without holding the mutex. In the case that we raced with
another thread to allocate the qos object, detect this *after* acquiring
the dev_pm_qos_mtx and simply free the redundant allocations.
Signed-off-by: Rob Clark <[email protected]>
---
drivers/base/power/qos.c | 60 +++++++++++++++++++++++++++-------------
1 file changed, 41 insertions(+), 19 deletions(-)
diff --git a/drivers/base/power/qos.c b/drivers/base/power/qos.c
index 8e93167f1783..f3e0c6b65635 100644
--- a/drivers/base/power/qos.c
+++ b/drivers/base/power/qos.c
@@ -185,18 +185,24 @@ static int apply_constraint(struct dev_pm_qos_request *req,
}
/*
- * dev_pm_qos_constraints_allocate
+ * dev_pm_qos_constraints_ensure_allocated
* @dev: device to allocate data for
*
- * Called at the first call to add_request, for constraint data allocation
- * Must be called with the dev_pm_qos_mtx mutex held
+ * Called to ensure that devices qos is allocated, before acquiring
+ * dev_pm_qos_mtx.
*/
-static int dev_pm_qos_constraints_allocate(struct device *dev)
+static int dev_pm_qos_constraints_ensure_allocated(struct device *dev)
{
struct dev_pm_qos *qos;
struct pm_qos_constraints *c;
struct blocking_notifier_head *n;
+ if (!dev)
+ return -ENODEV;
+
+ if (!IS_ERR_OR_NULL(dev->power.qos))
+ return 0;
+
qos = kzalloc(sizeof(*qos), GFP_KERNEL);
if (!qos)
return -ENOMEM;
@@ -227,10 +233,26 @@ static int dev_pm_qos_constraints_allocate(struct device *dev)
INIT_LIST_HEAD(&qos->flags.list);
+ mutex_lock(&dev_pm_qos_mtx);
+
+ if (!IS_ERR_OR_NULL(dev->power.qos)) {
+ /*
+ * We have raced with another task to create the qos.
+ * No biggie, just free the resources we've allocated
+ * outside of dev_pm_qos_mtx and move on with life.
+ */
+ kfree(n);
+ kfree(qos);
+ goto unlock;
+ }
+
spin_lock_irq(&dev->power.lock);
dev->power.qos = qos;
spin_unlock_irq(&dev->power.lock);
+unlock:
+ mutex_unlock(&dev_pm_qos_mtx);
+
return 0;
}
@@ -331,17 +353,15 @@ static int __dev_pm_qos_add_request(struct device *dev,
{
int ret = 0;
- if (!dev || !req || dev_pm_qos_invalid_req_type(dev, type))
+ if (!req || dev_pm_qos_invalid_req_type(dev, type))
return -EINVAL;
if (WARN(dev_pm_qos_request_active(req),
"%s() called for already added request\n", __func__))
return -EINVAL;
- if (IS_ERR(dev->power.qos))
+ if (IS_ERR_OR_NULL(dev->power.qos))
ret = -ENODEV;
- else if (!dev->power.qos)
- ret = dev_pm_qos_constraints_allocate(dev);
trace_dev_pm_qos_add_request(dev_name(dev), type, value);
if (ret)
@@ -390,6 +410,10 @@ int dev_pm_qos_add_request(struct device *dev, struct dev_pm_qos_request *req,
{
int ret;
+ ret = dev_pm_qos_constraints_ensure_allocated(dev);
+ if (ret)
+ return ret;
+
mutex_lock(&dev_pm_qos_mtx);
ret = __dev_pm_qos_add_request(dev, req, type, value);
mutex_unlock(&dev_pm_qos_mtx);
@@ -537,15 +561,11 @@ int dev_pm_qos_add_notifier(struct device *dev, struct notifier_block *notifier,
{
int ret = 0;
- mutex_lock(&dev_pm_qos_mtx);
-
- if (IS_ERR(dev->power.qos))
- ret = -ENODEV;
- else if (!dev->power.qos)
- ret = dev_pm_qos_constraints_allocate(dev);
-
+ ret = dev_pm_qos_constraints_ensure_allocated(dev);
if (ret)
- goto unlock;
+ return ret;
+
+ mutex_lock(&dev_pm_qos_mtx);
switch (type) {
case DEV_PM_QOS_RESUME_LATENCY:
@@ -565,7 +585,6 @@ int dev_pm_qos_add_notifier(struct device *dev, struct notifier_block *notifier,
ret = -EINVAL;
}
-unlock:
mutex_unlock(&dev_pm_qos_mtx);
return ret;
}
@@ -905,10 +924,13 @@ int dev_pm_qos_update_user_latency_tolerance(struct device *dev, s32 val)
{
int ret;
+ ret = dev_pm_qos_constraints_ensure_allocated(dev);
+ if (ret)
+ return ret;
+
mutex_lock(&dev_pm_qos_mtx);
- if (IS_ERR_OR_NULL(dev->power.qos)
- || !dev->power.qos->latency_tolerance_req) {
+ if (!dev->power.qos->latency_tolerance_req) {
struct dev_pm_qos_request *req;
if (val < 0) {
--
2.39.2
From: Rob Clark <[email protected]>
Similar to the previous patch, move the allocation out from under
dev_pm_qos_mtx, by speculatively doing the allocation and handle
any race after acquiring dev_pm_qos_mtx by freeing the redundant
allocation.
Signed-off-by: Rob Clark <[email protected]>
---
drivers/base/power/qos.c | 12 +++++++++++-
1 file changed, 11 insertions(+), 1 deletion(-)
diff --git a/drivers/base/power/qos.c b/drivers/base/power/qos.c
index f3e0c6b65635..9cba334b3729 100644
--- a/drivers/base/power/qos.c
+++ b/drivers/base/power/qos.c
@@ -922,12 +922,16 @@ s32 dev_pm_qos_get_user_latency_tolerance(struct device *dev)
*/
int dev_pm_qos_update_user_latency_tolerance(struct device *dev, s32 val)
{
+ struct dev_pm_qos_request *req = NULL;
int ret;
ret = dev_pm_qos_constraints_ensure_allocated(dev);
if (ret)
return ret;
+ if (!dev->power.qos->latency_tolerance_req)
+ req = kzalloc(sizeof(*req), GFP_KERNEL);
+
mutex_lock(&dev_pm_qos_mtx);
if (!dev->power.qos->latency_tolerance_req) {
@@ -940,7 +944,6 @@ int dev_pm_qos_update_user_latency_tolerance(struct device *dev, s32 val)
ret = -EINVAL;
goto out;
}
- req = kzalloc(sizeof(*req), GFP_KERNEL);
if (!req) {
ret = -ENOMEM;
goto out;
@@ -952,6 +955,13 @@ int dev_pm_qos_update_user_latency_tolerance(struct device *dev, s32 val)
}
dev->power.qos->latency_tolerance_req = req;
} else {
+ /*
+ * If we raced with another thread to allocate the request,
+ * simply free the redundant allocation and move on.
+ */
+ if (req)
+ kfree(req);
+
if (val < 0) {
__dev_pm_qos_drop_user_request(dev, DEV_PM_QOS_LATENCY_TOLERANCE);
ret = 0;
--
2.39.2
From: Rob Clark <[email protected]>
For cases where icc_bw_set() can be called in callbaths that could
deadlock against shrinker/reclaim, such as runpm resume, we need to
decouple the icc locking. Introduce a new icc_bw_lock for cases where
we need to serialize bw aggregation and update to decouple that from
paths that require memory allocation such as node/link creation/
destruction.
Fixes this lockdep splat:
======================================================
WARNING: possible circular locking dependency detected
6.2.0-rc8-debug+ #554 Not tainted
------------------------------------------------------
ring0/132 is trying to acquire lock:
ffffff80871916d0 (&gmu->lock){+.+.}-{3:3}, at: a6xx_pm_resume+0xf0/0x234
but task is already holding lock:
ffffffdb5aee57e8 (dma_fence_map){++++}-{0:0}, at: msm_job_run+0x68/0x150
which lock already depends on the new lock.
the existing dependency chain (in reverse order) is:
-> #4 (dma_fence_map){++++}-{0:0}:
__dma_fence_might_wait+0x74/0xc0
dma_resv_lockdep+0x1f4/0x2f4
do_one_initcall+0x104/0x2bc
kernel_init_freeable+0x344/0x34c
kernel_init+0x30/0x134
ret_from_fork+0x10/0x20
-> #3 (mmu_notifier_invalidate_range_start){+.+.}-{0:0}:
fs_reclaim_acquire+0x80/0xa8
slab_pre_alloc_hook.constprop.0+0x40/0x25c
__kmem_cache_alloc_node+0x60/0x1cc
__kmalloc+0xd8/0x100
topology_parse_cpu_capacity+0x8c/0x178
get_cpu_for_node+0x88/0xc4
parse_cluster+0x1b0/0x28c
parse_cluster+0x8c/0x28c
init_cpu_topology+0x168/0x188
smp_prepare_cpus+0x24/0xf8
kernel_init_freeable+0x18c/0x34c
kernel_init+0x30/0x134
ret_from_fork+0x10/0x20
-> #2 (fs_reclaim){+.+.}-{0:0}:
__fs_reclaim_acquire+0x3c/0x48
fs_reclaim_acquire+0x54/0xa8
slab_pre_alloc_hook.constprop.0+0x40/0x25c
__kmem_cache_alloc_node+0x60/0x1cc
__kmalloc+0xd8/0x100
kzalloc.constprop.0+0x14/0x20
icc_node_create_nolock+0x4c/0xc4
icc_node_create+0x38/0x58
qcom_icc_rpmh_probe+0x1b8/0x248
platform_probe+0x70/0xc4
really_probe+0x158/0x290
__driver_probe_device+0xc8/0xe0
driver_probe_device+0x44/0x100
__driver_attach+0xf8/0x108
bus_for_each_dev+0x78/0xc4
driver_attach+0x2c/0x38
bus_add_driver+0xd0/0x1d8
driver_register+0xbc/0xf8
__platform_driver_register+0x30/0x3c
qnoc_driver_init+0x24/0x30
do_one_initcall+0x104/0x2bc
kernel_init_freeable+0x344/0x34c
kernel_init+0x30/0x134
ret_from_fork+0x10/0x20
-> #1 (icc_lock){+.+.}-{3:3}:
__mutex_lock+0xcc/0x3c8
mutex_lock_nested+0x30/0x44
icc_set_bw+0x88/0x2b4
_set_opp_bw+0x8c/0xd8
_set_opp+0x19c/0x300
dev_pm_opp_set_opp+0x84/0x94
a6xx_gmu_resume+0x18c/0x804
a6xx_pm_resume+0xf8/0x234
adreno_runtime_resume+0x2c/0x38
pm_generic_runtime_resume+0x30/0x44
__rpm_callback+0x15c/0x174
rpm_callback+0x78/0x7c
rpm_resume+0x318/0x524
__pm_runtime_resume+0x78/0xbc
adreno_load_gpu+0xc4/0x17c
msm_open+0x50/0x120
drm_file_alloc+0x17c/0x228
drm_open_helper+0x74/0x118
drm_open+0xa0/0x144
drm_stub_open+0xd4/0xe4
chrdev_open+0x1b8/0x1e4
do_dentry_open+0x2f8/0x38c
vfs_open+0x34/0x40
path_openat+0x64c/0x7b4
do_filp_open+0x54/0xc4
do_sys_openat2+0x9c/0x100
do_sys_open+0x50/0x7c
__arm64_sys_openat+0x28/0x34
invoke_syscall+0x8c/0x128
el0_svc_common.constprop.0+0xa0/0x11c
do_el0_svc+0xac/0xbc
el0_svc+0x48/0xa0
el0t_64_sync_handler+0xac/0x13c
el0t_64_sync+0x190/0x194
-> #0 (&gmu->lock){+.+.}-{3:3}:
__lock_acquire+0xe00/0x1060
lock_acquire+0x1e0/0x2f8
__mutex_lock+0xcc/0x3c8
mutex_lock_nested+0x30/0x44
a6xx_pm_resume+0xf0/0x234
adreno_runtime_resume+0x2c/0x38
pm_generic_runtime_resume+0x30/0x44
__rpm_callback+0x15c/0x174
rpm_callback+0x78/0x7c
rpm_resume+0x318/0x524
__pm_runtime_resume+0x78/0xbc
pm_runtime_get_sync.isra.0+0x14/0x20
msm_gpu_submit+0x58/0x178
msm_job_run+0x78/0x150
drm_sched_main+0x290/0x370
kthread+0xf0/0x100
ret_from_fork+0x10/0x20
other info that might help us debug this:
Chain exists of:
&gmu->lock --> mmu_notifier_invalidate_range_start --> dma_fence_map
Possible unsafe locking scenario:
CPU0 CPU1
---- ----
lock(dma_fence_map);
lock(mmu_notifier_invalidate_range_start);
lock(dma_fence_map);
lock(&gmu->lock);
*** DEADLOCK ***
2 locks held by ring0/132:
#0: ffffff8087191170 (&gpu->lock){+.+.}-{3:3}, at: msm_job_run+0x64/0x150
#1: ffffffdb5aee57e8 (dma_fence_map){++++}-{0:0}, at: msm_job_run+0x68/0x150
stack backtrace:
CPU: 7 PID: 132 Comm: ring0 Not tainted 6.2.0-rc8-debug+ #554
Hardware name: Google Lazor (rev1 - 2) with LTE (DT)
Call trace:
dump_backtrace.part.0+0xb4/0xf8
show_stack+0x20/0x38
dump_stack_lvl+0x9c/0xd0
dump_stack+0x18/0x34
print_circular_bug+0x1b4/0x1f0
check_noncircular+0x78/0xac
__lock_acquire+0xe00/0x1060
lock_acquire+0x1e0/0x2f8
__mutex_lock+0xcc/0x3c8
mutex_lock_nested+0x30/0x44
a6xx_pm_resume+0xf0/0x234
adreno_runtime_resume+0x2c/0x38
pm_generic_runtime_resume+0x30/0x44
__rpm_callback+0x15c/0x174
rpm_callback+0x78/0x7c
rpm_resume+0x318/0x524
__pm_runtime_resume+0x78/0xbc
pm_runtime_get_sync.isra.0+0x14/0x20
msm_gpu_submit+0x58/0x178
msm_job_run+0x78/0x150
drm_sched_main+0x290/0x370
kthread+0xf0/0x100
ret_from_fork+0x10/0x20
Signed-off-by: Rob Clark <[email protected]>
---
drivers/interconnect/core.c | 8 ++++++--
1 file changed, 6 insertions(+), 2 deletions(-)
diff --git a/drivers/interconnect/core.c b/drivers/interconnect/core.c
index 25debded65a8..f7251784765f 100644
--- a/drivers/interconnect/core.c
+++ b/drivers/interconnect/core.c
@@ -29,6 +29,7 @@ static LIST_HEAD(icc_providers);
static int providers_count;
static bool synced_state;
static DEFINE_MUTEX(icc_lock);
+static DEFINE_MUTEX(icc_bw_lock);
static struct dentry *icc_debugfs_dir;
static void icc_summary_show_one(struct seq_file *s, struct icc_node *n)
@@ -632,7 +633,7 @@ int icc_set_bw(struct icc_path *path, u32 avg_bw, u32 peak_bw)
if (WARN_ON(IS_ERR(path) || !path->num_nodes))
return -EINVAL;
- mutex_lock(&icc_lock);
+ mutex_lock(&icc_bw_lock);
old_avg = path->reqs[0].avg_bw;
old_peak = path->reqs[0].peak_bw;
@@ -664,7 +665,7 @@ int icc_set_bw(struct icc_path *path, u32 avg_bw, u32 peak_bw)
apply_constraints(path);
}
- mutex_unlock(&icc_lock);
+ mutex_unlock(&icc_bw_lock);
trace_icc_set_bw_end(path, ret);
@@ -963,6 +964,7 @@ void icc_node_add(struct icc_node *node, struct icc_provider *provider)
return;
mutex_lock(&icc_lock);
+ mutex_lock(&icc_bw_lock);
node->provider = provider;
list_add_tail(&node->node_list, &provider->nodes);
@@ -988,6 +990,7 @@ void icc_node_add(struct icc_node *node, struct icc_provider *provider)
node->avg_bw = 0;
node->peak_bw = 0;
+ mutex_unlock(&icc_bw_lock);
mutex_unlock(&icc_lock);
}
EXPORT_SYMBOL_GPL(icc_node_add);
@@ -1111,6 +1114,7 @@ void icc_sync_state(struct device *dev)
return;
mutex_lock(&icc_lock);
+ mutex_lock(&icc_bw_lock);
synced_state = true;
list_for_each_entry(p, &icc_providers, provider_list) {
dev_dbg(p->dev, "interconnect provider is in synced state\n");
--
2.39.2
From: Rob Clark <[email protected]>
Teach lockdep that icc_bw_lock is needed in code paths that could
deadlock if they trigger reclaim.
Signed-off-by: Rob Clark <[email protected]>
---
drivers/interconnect/core.c | 10 +++++++++-
1 file changed, 9 insertions(+), 1 deletion(-)
diff --git a/drivers/interconnect/core.c b/drivers/interconnect/core.c
index f7251784765f..5619963ee85c 100644
--- a/drivers/interconnect/core.c
+++ b/drivers/interconnect/core.c
@@ -1127,13 +1127,21 @@ void icc_sync_state(struct device *dev)
}
}
}
+ mutex_unlock(&icc_bw_lock);
mutex_unlock(&icc_lock);
}
EXPORT_SYMBOL_GPL(icc_sync_state);
static int __init icc_init(void)
{
- struct device_node *root = of_find_node_by_path("/");
+ struct device_node *root;
+
+ /* Teach lockdep about lock ordering wrt. shrinker: */
+ fs_reclaim_acquire(GFP_KERNEL);
+ might_lock(&icc_bw_lock);
+ fs_reclaim_release(GFP_KERNEL);
+
+ root = of_find_node_by_path("/");
providers_count = of_count_icc_providers(root);
of_node_put(root);
--
2.39.2
From: Rob Clark <[email protected]>
Based on
https://lore.kernel.org/dri-devel/[email protected]/
but made to be optional.
Signed-off-by: Rob Clark <[email protected]>
---
drivers/gpu/drm/msm/msm_ringbuffer.c | 1 +
drivers/gpu/drm/scheduler/sched_main.c | 9 +++++++++
include/drm/gpu_scheduler.h | 2 ++
3 files changed, 12 insertions(+)
diff --git a/drivers/gpu/drm/msm/msm_ringbuffer.c b/drivers/gpu/drm/msm/msm_ringbuffer.c
index b60199184409..7e42baf16cd0 100644
--- a/drivers/gpu/drm/msm/msm_ringbuffer.c
+++ b/drivers/gpu/drm/msm/msm_ringbuffer.c
@@ -93,6 +93,7 @@ struct msm_ringbuffer *msm_ringbuffer_new(struct msm_gpu *gpu, int id,
/* currently managing hangcheck ourselves: */
sched_timeout = MAX_SCHEDULE_TIMEOUT;
+ ring->sched.fence_signaling = true;
ret = drm_sched_init(&ring->sched, &msm_sched_ops,
num_hw_submissions, 0, sched_timeout,
NULL, NULL, to_msm_bo(ring->bo)->name, gpu->dev->dev);
diff --git a/drivers/gpu/drm/scheduler/sched_main.c b/drivers/gpu/drm/scheduler/sched_main.c
index 4e6ad6e122bc..c2ee44d6224b 100644
--- a/drivers/gpu/drm/scheduler/sched_main.c
+++ b/drivers/gpu/drm/scheduler/sched_main.c
@@ -978,10 +978,15 @@ static bool drm_sched_blocked(struct drm_gpu_scheduler *sched)
static int drm_sched_main(void *param)
{
struct drm_gpu_scheduler *sched = (struct drm_gpu_scheduler *)param;
+ const bool fence_signaling = sched->fence_signaling;
+ bool fence_cookie;
int r;
sched_set_fifo_low(current);
+ if (fence_signaling)
+ fence_cookie = dma_fence_begin_signalling();
+
while (!kthread_should_stop()) {
struct drm_sched_entity *entity = NULL;
struct drm_sched_fence *s_fence;
@@ -1039,6 +1044,10 @@ static int drm_sched_main(void *param)
wake_up(&sched->job_scheduled);
}
+
+ if (fence_signaling)
+ dma_fence_end_signalling(fence_cookie);
+
return 0;
}
diff --git a/include/drm/gpu_scheduler.h b/include/drm/gpu_scheduler.h
index 9db9e5e504ee..8f23ea522e22 100644
--- a/include/drm/gpu_scheduler.h
+++ b/include/drm/gpu_scheduler.h
@@ -483,6 +483,7 @@ struct drm_sched_backend_ops {
* @ready: marks if the underlying HW is ready to work
* @free_guilty: A hit to time out handler to free the guilty job.
* @dev: system &struct device
+ * @fence_signaling: Opt in to fence signaling annotations
*
* One scheduler is implemented for each hardware ring.
*/
@@ -507,6 +508,7 @@ struct drm_gpu_scheduler {
bool ready;
bool free_guilty;
struct device *dev;
+ bool fence_signaling;
};
int drm_sched_init(struct drm_gpu_scheduler *sched,
--
2.39.2
Am 20.03.23 um 15:43 schrieb Rob Clark:
> From: Rob Clark <[email protected]>
>
> Avoid allocating memory in job_run() by pre-allocating the hw_fence.
>
> Signed-off-by: Rob Clark <[email protected]>
> ---
> drivers/gpu/drm/msm/msm_fence.c | 12 +++++++++---
> drivers/gpu/drm/msm/msm_fence.h | 3 ++-
> drivers/gpu/drm/msm/msm_gem_submit.c | 7 +++++++
> drivers/gpu/drm/msm/msm_ringbuffer.c | 2 +-
> 4 files changed, 19 insertions(+), 5 deletions(-)
>
> diff --git a/drivers/gpu/drm/msm/msm_fence.c b/drivers/gpu/drm/msm/msm_fence.c
> index 56641408ea74..bab3d84f1686 100644
> --- a/drivers/gpu/drm/msm/msm_fence.c
> +++ b/drivers/gpu/drm/msm/msm_fence.c
> @@ -99,7 +99,7 @@ static const struct dma_fence_ops msm_fence_ops = {
> };
>
> struct dma_fence *
> -msm_fence_alloc(struct msm_fence_context *fctx)
> +msm_fence_alloc(void)
> {
> struct msm_fence *f;
>
> @@ -107,10 +107,16 @@ msm_fence_alloc(struct msm_fence_context *fctx)
> if (!f)
> return ERR_PTR(-ENOMEM);
>
> + return &f->base;
> +}
> +
> +void
> +msm_fence_init(struct dma_fence *fence, struct msm_fence_context *fctx)
> +{
> + struct msm_fence *f = to_msm_fence(fence);
> +
> f->fctx = fctx;
>
> dma_fence_init(&f->base, &msm_fence_ops, &fctx->spinlock,
> fctx->context, ++fctx->last_fence);
> -
> - return &f->base;
> }
> diff --git a/drivers/gpu/drm/msm/msm_fence.h b/drivers/gpu/drm/msm/msm_fence.h
> index 7f1798c54cd1..f913fa22d8fe 100644
> --- a/drivers/gpu/drm/msm/msm_fence.h
> +++ b/drivers/gpu/drm/msm/msm_fence.h
> @@ -61,7 +61,8 @@ void msm_fence_context_free(struct msm_fence_context *fctx);
> bool msm_fence_completed(struct msm_fence_context *fctx, uint32_t fence);
> void msm_update_fence(struct msm_fence_context *fctx, uint32_t fence);
>
> -struct dma_fence * msm_fence_alloc(struct msm_fence_context *fctx);
> +struct dma_fence * msm_fence_alloc(void);
> +void msm_fence_init(struct dma_fence *fence, struct msm_fence_context *fctx);
>
> static inline bool
> fence_before(uint32_t a, uint32_t b)
> diff --git a/drivers/gpu/drm/msm/msm_gem_submit.c b/drivers/gpu/drm/msm/msm_gem_submit.c
> index be4bf77103cd..2570c018b0cb 100644
> --- a/drivers/gpu/drm/msm/msm_gem_submit.c
> +++ b/drivers/gpu/drm/msm/msm_gem_submit.c
> @@ -41,6 +41,13 @@ static struct msm_gem_submit *submit_create(struct drm_device *dev,
> if (!submit)
> return ERR_PTR(-ENOMEM);
>
> + submit->hw_fence = msm_fence_alloc();
> + if (IS_ERR(submit->hw_fence)) {
> + ret = PTR_ERR(submit->hw_fence);
> + kfree(submit);
> + return ERR_PTR(ret);
> + }
> +
> ret = drm_sched_job_init(&submit->base, queue->entity, queue);
> if (ret) {
> kfree(submit);
You probably need some error handling here or otherwise leak
submit->hw_fence.
Apart from that looks good to me.
Christian.
> diff --git a/drivers/gpu/drm/msm/msm_ringbuffer.c b/drivers/gpu/drm/msm/msm_ringbuffer.c
> index 57a8e9564540..a62b45e5a8c3 100644
> --- a/drivers/gpu/drm/msm/msm_ringbuffer.c
> +++ b/drivers/gpu/drm/msm/msm_ringbuffer.c
> @@ -18,7 +18,7 @@ static struct dma_fence *msm_job_run(struct drm_sched_job *job)
> struct msm_gpu *gpu = submit->gpu;
> int i;
>
> - submit->hw_fence = msm_fence_alloc(fctx);
> + msm_fence_init(submit->hw_fence, fctx);
>
> for (i = 0; i < submit->nr_bos; i++) {
> struct drm_gem_object *obj = &submit->bos[i].obj->base;
Hi Rob,
I love your patch! Perhaps something to improve:
[auto build test WARNING on drm-misc/drm-misc-next]
[also build test WARNING on rafael-pm/linux-next drm-intel/for-linux-next drm-intel/for-linux-next-fixes drm-tip/drm-tip linus/master v6.3-rc3 next-20230320]
[cannot apply to chanwoo/devfreq-testing]
[If your patch is applied to the wrong git tree, kindly drop us a note.
And when submitting patch, we suggest to use '--base' as documented in
https://git-scm.com/docs/git-format-patch#_base_tree_information]
url: https://github.com/intel-lab-lkp/linux/commits/Rob-Clark/drm-msm-Pre-allocate-hw_fence/20230320-224826
base: git://anongit.freedesktop.org/drm/drm-misc drm-misc-next
patch link: https://lore.kernel.org/r/20230320144356.803762-19-robdclark%40gmail.com
patch subject: [PATCH v2 18/23] PM / QoS: Decouple request alloc from dev_pm_qos_mtx
config: arm-randconfig-r005-20230319 (https://download.01.org/0day-ci/archive/20230321/[email protected]/config)
compiler: clang version 17.0.0 (https://github.com/llvm/llvm-project 67409911353323ca5edf2049ef0df54132fa1ca7)
reproduce (this is a W=1 build):
wget https://raw.githubusercontent.com/intel/lkp-tests/master/sbin/make.cross -O ~/bin/make.cross
chmod +x ~/bin/make.cross
# install arm cross compiling tool for clang build
# apt-get install binutils-arm-linux-gnueabi
# https://github.com/intel-lab-lkp/linux/commit/2d7e4629d7265d7e77fc72d01e84d27d805b7485
git remote add linux-review https://github.com/intel-lab-lkp/linux
git fetch --no-tags linux-review Rob-Clark/drm-msm-Pre-allocate-hw_fence/20230320-224826
git checkout 2d7e4629d7265d7e77fc72d01e84d27d805b7485
# save the config file
mkdir build_dir && cp config build_dir/.config
COMPILER_INSTALL_PATH=$HOME/0day COMPILER=clang make.cross W=1 O=build_dir ARCH=arm olddefconfig
COMPILER_INSTALL_PATH=$HOME/0day COMPILER=clang make.cross W=1 O=build_dir ARCH=arm SHELL=/bin/bash drivers/base/power/
If you fix the issue, kindly add following tag where applicable
| Reported-by: kernel test robot <[email protected]>
| Link: https://lore.kernel.org/oe-kbuild-all/[email protected]/
All warnings (new ones prefixed by >>):
>> drivers/base/power/qos.c:947:8: warning: variable 'req' is uninitialized when used here [-Wuninitialized]
if (!req) {
^~~
drivers/base/power/qos.c:938:33: note: initialize the variable 'req' to silence this warning
struct dev_pm_qos_request *req;
^
= NULL
1 warning generated.
vim +/req +947 drivers/base/power/qos.c
2d984ad132a87c Rafael J. Wysocki 2014-02-11 917
2d984ad132a87c Rafael J. Wysocki 2014-02-11 918 /**
2d984ad132a87c Rafael J. Wysocki 2014-02-11 919 * dev_pm_qos_update_user_latency_tolerance - Update user space latency tolerance.
2d984ad132a87c Rafael J. Wysocki 2014-02-11 920 * @dev: Device to update the user space latency tolerance for.
2d984ad132a87c Rafael J. Wysocki 2014-02-11 921 * @val: New user space latency tolerance for @dev (negative values disable).
2d984ad132a87c Rafael J. Wysocki 2014-02-11 922 */
2d984ad132a87c Rafael J. Wysocki 2014-02-11 923 int dev_pm_qos_update_user_latency_tolerance(struct device *dev, s32 val)
2d984ad132a87c Rafael J. Wysocki 2014-02-11 924 {
2d7e4629d7265d Rob Clark 2023-03-20 925 struct dev_pm_qos_request *req = NULL;
2d984ad132a87c Rafael J. Wysocki 2014-02-11 926 int ret;
2d984ad132a87c Rafael J. Wysocki 2014-02-11 927
00dd582e52a535 Rob Clark 2023-03-20 928 ret = dev_pm_qos_constraints_ensure_allocated(dev);
00dd582e52a535 Rob Clark 2023-03-20 929 if (ret)
00dd582e52a535 Rob Clark 2023-03-20 930 return ret;
00dd582e52a535 Rob Clark 2023-03-20 931
2d7e4629d7265d Rob Clark 2023-03-20 932 if (!dev->power.qos->latency_tolerance_req)
2d7e4629d7265d Rob Clark 2023-03-20 933 req = kzalloc(sizeof(*req), GFP_KERNEL);
2d7e4629d7265d Rob Clark 2023-03-20 934
2d984ad132a87c Rafael J. Wysocki 2014-02-11 935 mutex_lock(&dev_pm_qos_mtx);
2d984ad132a87c Rafael J. Wysocki 2014-02-11 936
00dd582e52a535 Rob Clark 2023-03-20 937 if (!dev->power.qos->latency_tolerance_req) {
2d984ad132a87c Rafael J. Wysocki 2014-02-11 938 struct dev_pm_qos_request *req;
2d984ad132a87c Rafael J. Wysocki 2014-02-11 939
2d984ad132a87c Rafael J. Wysocki 2014-02-11 940 if (val < 0) {
80a6f7c79b7822 Andrew Lutomirski 2016-11-29 941 if (val == PM_QOS_LATENCY_TOLERANCE_NO_CONSTRAINT)
80a6f7c79b7822 Andrew Lutomirski 2016-11-29 942 ret = 0;
80a6f7c79b7822 Andrew Lutomirski 2016-11-29 943 else
2d984ad132a87c Rafael J. Wysocki 2014-02-11 944 ret = -EINVAL;
2d984ad132a87c Rafael J. Wysocki 2014-02-11 945 goto out;
2d984ad132a87c Rafael J. Wysocki 2014-02-11 946 }
2d984ad132a87c Rafael J. Wysocki 2014-02-11 @947 if (!req) {
2d984ad132a87c Rafael J. Wysocki 2014-02-11 948 ret = -ENOMEM;
2d984ad132a87c Rafael J. Wysocki 2014-02-11 949 goto out;
2d984ad132a87c Rafael J. Wysocki 2014-02-11 950 }
2d984ad132a87c Rafael J. Wysocki 2014-02-11 951 ret = __dev_pm_qos_add_request(dev, req, DEV_PM_QOS_LATENCY_TOLERANCE, val);
2d984ad132a87c Rafael J. Wysocki 2014-02-11 952 if (ret < 0) {
2d984ad132a87c Rafael J. Wysocki 2014-02-11 953 kfree(req);
2d984ad132a87c Rafael J. Wysocki 2014-02-11 954 goto out;
2d984ad132a87c Rafael J. Wysocki 2014-02-11 955 }
2d984ad132a87c Rafael J. Wysocki 2014-02-11 956 dev->power.qos->latency_tolerance_req = req;
2d984ad132a87c Rafael J. Wysocki 2014-02-11 957 } else {
2d7e4629d7265d Rob Clark 2023-03-20 958 /*
2d7e4629d7265d Rob Clark 2023-03-20 959 * If we raced with another thread to allocate the request,
2d7e4629d7265d Rob Clark 2023-03-20 960 * simply free the redundant allocation and move on.
2d7e4629d7265d Rob Clark 2023-03-20 961 */
2d7e4629d7265d Rob Clark 2023-03-20 962 if (req)
2d7e4629d7265d Rob Clark 2023-03-20 963 kfree(req);
2d7e4629d7265d Rob Clark 2023-03-20 964
2d984ad132a87c Rafael J. Wysocki 2014-02-11 965 if (val < 0) {
2d984ad132a87c Rafael J. Wysocki 2014-02-11 966 __dev_pm_qos_drop_user_request(dev, DEV_PM_QOS_LATENCY_TOLERANCE);
2d984ad132a87c Rafael J. Wysocki 2014-02-11 967 ret = 0;
2d984ad132a87c Rafael J. Wysocki 2014-02-11 968 } else {
2d984ad132a87c Rafael J. Wysocki 2014-02-11 969 ret = __dev_pm_qos_update_request(dev->power.qos->latency_tolerance_req, val);
2d984ad132a87c Rafael J. Wysocki 2014-02-11 970 }
2d984ad132a87c Rafael J. Wysocki 2014-02-11 971 }
2d984ad132a87c Rafael J. Wysocki 2014-02-11 972
2d984ad132a87c Rafael J. Wysocki 2014-02-11 973 out:
2d984ad132a87c Rafael J. Wysocki 2014-02-11 974 mutex_unlock(&dev_pm_qos_mtx);
2d984ad132a87c Rafael J. Wysocki 2014-02-11 975 return ret;
2d984ad132a87c Rafael J. Wysocki 2014-02-11 976 }
034e7906211c18 Andrew Lutomirski 2016-11-29 977 EXPORT_SYMBOL_GPL(dev_pm_qos_update_user_latency_tolerance);
13b2c4a0c3b1cd Mika Westerberg 2015-07-27 978
--
0-DAY CI Kernel Test Service
https://github.com/intel/lkp-tests
On Mon, Mar 20, 2023 at 9:52 AM Christian König
<[email protected]> wrote:
>
>
>
> Am 20.03.23 um 15:43 schrieb Rob Clark:
> > From: Rob Clark <[email protected]>
> >
> > Avoid allocating memory in job_run() by pre-allocating the hw_fence.
> >
> > Signed-off-by: Rob Clark <[email protected]>
> > ---
> > drivers/gpu/drm/msm/msm_fence.c | 12 +++++++++---
> > drivers/gpu/drm/msm/msm_fence.h | 3 ++-
> > drivers/gpu/drm/msm/msm_gem_submit.c | 7 +++++++
> > drivers/gpu/drm/msm/msm_ringbuffer.c | 2 +-
> > 4 files changed, 19 insertions(+), 5 deletions(-)
> >
> > diff --git a/drivers/gpu/drm/msm/msm_fence.c b/drivers/gpu/drm/msm/msm_fence.c
> > index 56641408ea74..bab3d84f1686 100644
> > --- a/drivers/gpu/drm/msm/msm_fence.c
> > +++ b/drivers/gpu/drm/msm/msm_fence.c
> > @@ -99,7 +99,7 @@ static const struct dma_fence_ops msm_fence_ops = {
> > };
> >
> > struct dma_fence *
> > -msm_fence_alloc(struct msm_fence_context *fctx)
> > +msm_fence_alloc(void)
> > {
> > struct msm_fence *f;
> >
> > @@ -107,10 +107,16 @@ msm_fence_alloc(struct msm_fence_context *fctx)
> > if (!f)
> > return ERR_PTR(-ENOMEM);
> >
> > + return &f->base;
> > +}
> > +
> > +void
> > +msm_fence_init(struct dma_fence *fence, struct msm_fence_context *fctx)
> > +{
> > + struct msm_fence *f = to_msm_fence(fence);
> > +
> > f->fctx = fctx;
> >
> > dma_fence_init(&f->base, &msm_fence_ops, &fctx->spinlock,
> > fctx->context, ++fctx->last_fence);
> > -
> > - return &f->base;
> > }
> > diff --git a/drivers/gpu/drm/msm/msm_fence.h b/drivers/gpu/drm/msm/msm_fence.h
> > index 7f1798c54cd1..f913fa22d8fe 100644
> > --- a/drivers/gpu/drm/msm/msm_fence.h
> > +++ b/drivers/gpu/drm/msm/msm_fence.h
> > @@ -61,7 +61,8 @@ void msm_fence_context_free(struct msm_fence_context *fctx);
> > bool msm_fence_completed(struct msm_fence_context *fctx, uint32_t fence);
> > void msm_update_fence(struct msm_fence_context *fctx, uint32_t fence);
> >
> > -struct dma_fence * msm_fence_alloc(struct msm_fence_context *fctx);
> > +struct dma_fence * msm_fence_alloc(void);
> > +void msm_fence_init(struct dma_fence *fence, struct msm_fence_context *fctx);
> >
> > static inline bool
> > fence_before(uint32_t a, uint32_t b)
> > diff --git a/drivers/gpu/drm/msm/msm_gem_submit.c b/drivers/gpu/drm/msm/msm_gem_submit.c
> > index be4bf77103cd..2570c018b0cb 100644
> > --- a/drivers/gpu/drm/msm/msm_gem_submit.c
> > +++ b/drivers/gpu/drm/msm/msm_gem_submit.c
> > @@ -41,6 +41,13 @@ static struct msm_gem_submit *submit_create(struct drm_device *dev,
> > if (!submit)
> > return ERR_PTR(-ENOMEM);
> >
> > + submit->hw_fence = msm_fence_alloc();
> > + if (IS_ERR(submit->hw_fence)) {
> > + ret = PTR_ERR(submit->hw_fence);
> > + kfree(submit);
> > + return ERR_PTR(ret);
> > + }
> > +
> > ret = drm_sched_job_init(&submit->base, queue->entity, queue);
> > if (ret) {
> > kfree(submit);
>
> You probably need some error handling here or otherwise leak
> submit->hw_fence.
ah, right.. thx
BR,
-R
> Apart from that looks good to me.
>
> Christian.
>
> > diff --git a/drivers/gpu/drm/msm/msm_ringbuffer.c b/drivers/gpu/drm/msm/msm_ringbuffer.c
> > index 57a8e9564540..a62b45e5a8c3 100644
> > --- a/drivers/gpu/drm/msm/msm_ringbuffer.c
> > +++ b/drivers/gpu/drm/msm/msm_ringbuffer.c
> > @@ -18,7 +18,7 @@ static struct dma_fence *msm_job_run(struct drm_sched_job *job)
> > struct msm_gpu *gpu = submit->gpu;
> > int i;
> >
> > - submit->hw_fence = msm_fence_alloc(fctx);
> > + msm_fence_init(submit->hw_fence, fctx);
> >
> > for (i = 0; i < submit->nr_bos; i++) {
> > struct drm_gem_object *obj = &submit->bos[i].obj->base;
>
Hi Rob,
I love your patch! Perhaps something to improve:
[auto build test WARNING on drm-misc/drm-misc-next]
[also build test WARNING on rafael-pm/linux-next drm-intel/for-linux-next drm-intel/for-linux-next-fixes drm-tip/drm-tip linus/master v6.3-rc3 next-20230320]
[cannot apply to chanwoo/devfreq-testing]
[If your patch is applied to the wrong git tree, kindly drop us a note.
And when submitting patch, we suggest to use '--base' as documented in
https://git-scm.com/docs/git-format-patch#_base_tree_information]
url: https://github.com/intel-lab-lkp/linux/commits/Rob-Clark/drm-msm-Pre-allocate-hw_fence/20230320-224826
base: git://anongit.freedesktop.org/drm/drm-misc drm-misc-next
patch link: https://lore.kernel.org/r/20230320144356.803762-19-robdclark%40gmail.com
patch subject: [PATCH v2 18/23] PM / QoS: Decouple request alloc from dev_pm_qos_mtx
config: x86_64-randconfig-a014 (https://download.01.org/0day-ci/archive/20230321/[email protected]/config)
compiler: clang version 14.0.6 (https://github.com/llvm/llvm-project f28c006a5895fc0e329fe15fead81e37457cb1d1)
reproduce (this is a W=1 build):
wget https://raw.githubusercontent.com/intel/lkp-tests/master/sbin/make.cross -O ~/bin/make.cross
chmod +x ~/bin/make.cross
# https://github.com/intel-lab-lkp/linux/commit/2d7e4629d7265d7e77fc72d01e84d27d805b7485
git remote add linux-review https://github.com/intel-lab-lkp/linux
git fetch --no-tags linux-review Rob-Clark/drm-msm-Pre-allocate-hw_fence/20230320-224826
git checkout 2d7e4629d7265d7e77fc72d01e84d27d805b7485
# save the config file
mkdir build_dir && cp config build_dir/.config
COMPILER_INSTALL_PATH=$HOME/0day COMPILER=clang make.cross W=1 O=build_dir ARCH=x86_64 olddefconfig
COMPILER_INSTALL_PATH=$HOME/0day COMPILER=clang make.cross W=1 O=build_dir ARCH=x86_64 SHELL=/bin/bash drivers/base/power/ drivers/char/tpm/
If you fix the issue, kindly add following tag where applicable
| Reported-by: kernel test robot <[email protected]>
| Link: https://lore.kernel.org/oe-kbuild-all/[email protected]/
All warnings (new ones prefixed by >>):
>> drivers/base/power/qos.c:947:8: warning: variable 'req' is uninitialized when used here [-Wuninitialized]
if (!req) {
^~~
drivers/base/power/qos.c:938:33: note: initialize the variable 'req' to silence this warning
struct dev_pm_qos_request *req;
^
= NULL
1 warning generated.
vim +/req +947 drivers/base/power/qos.c
2d984ad132a87c Rafael J. Wysocki 2014-02-11 917
2d984ad132a87c Rafael J. Wysocki 2014-02-11 918 /**
2d984ad132a87c Rafael J. Wysocki 2014-02-11 919 * dev_pm_qos_update_user_latency_tolerance - Update user space latency tolerance.
2d984ad132a87c Rafael J. Wysocki 2014-02-11 920 * @dev: Device to update the user space latency tolerance for.
2d984ad132a87c Rafael J. Wysocki 2014-02-11 921 * @val: New user space latency tolerance for @dev (negative values disable).
2d984ad132a87c Rafael J. Wysocki 2014-02-11 922 */
2d984ad132a87c Rafael J. Wysocki 2014-02-11 923 int dev_pm_qos_update_user_latency_tolerance(struct device *dev, s32 val)
2d984ad132a87c Rafael J. Wysocki 2014-02-11 924 {
2d7e4629d7265d Rob Clark 2023-03-20 925 struct dev_pm_qos_request *req = NULL;
2d984ad132a87c Rafael J. Wysocki 2014-02-11 926 int ret;
2d984ad132a87c Rafael J. Wysocki 2014-02-11 927
00dd582e52a535 Rob Clark 2023-03-20 928 ret = dev_pm_qos_constraints_ensure_allocated(dev);
00dd582e52a535 Rob Clark 2023-03-20 929 if (ret)
00dd582e52a535 Rob Clark 2023-03-20 930 return ret;
00dd582e52a535 Rob Clark 2023-03-20 931
2d7e4629d7265d Rob Clark 2023-03-20 932 if (!dev->power.qos->latency_tolerance_req)
2d7e4629d7265d Rob Clark 2023-03-20 933 req = kzalloc(sizeof(*req), GFP_KERNEL);
2d7e4629d7265d Rob Clark 2023-03-20 934
2d984ad132a87c Rafael J. Wysocki 2014-02-11 935 mutex_lock(&dev_pm_qos_mtx);
2d984ad132a87c Rafael J. Wysocki 2014-02-11 936
00dd582e52a535 Rob Clark 2023-03-20 937 if (!dev->power.qos->latency_tolerance_req) {
2d984ad132a87c Rafael J. Wysocki 2014-02-11 938 struct dev_pm_qos_request *req;
2d984ad132a87c Rafael J. Wysocki 2014-02-11 939
2d984ad132a87c Rafael J. Wysocki 2014-02-11 940 if (val < 0) {
80a6f7c79b7822 Andrew Lutomirski 2016-11-29 941 if (val == PM_QOS_LATENCY_TOLERANCE_NO_CONSTRAINT)
80a6f7c79b7822 Andrew Lutomirski 2016-11-29 942 ret = 0;
80a6f7c79b7822 Andrew Lutomirski 2016-11-29 943 else
2d984ad132a87c Rafael J. Wysocki 2014-02-11 944 ret = -EINVAL;
2d984ad132a87c Rafael J. Wysocki 2014-02-11 945 goto out;
2d984ad132a87c Rafael J. Wysocki 2014-02-11 946 }
2d984ad132a87c Rafael J. Wysocki 2014-02-11 @947 if (!req) {
2d984ad132a87c Rafael J. Wysocki 2014-02-11 948 ret = -ENOMEM;
2d984ad132a87c Rafael J. Wysocki 2014-02-11 949 goto out;
2d984ad132a87c Rafael J. Wysocki 2014-02-11 950 }
2d984ad132a87c Rafael J. Wysocki 2014-02-11 951 ret = __dev_pm_qos_add_request(dev, req, DEV_PM_QOS_LATENCY_TOLERANCE, val);
2d984ad132a87c Rafael J. Wysocki 2014-02-11 952 if (ret < 0) {
2d984ad132a87c Rafael J. Wysocki 2014-02-11 953 kfree(req);
2d984ad132a87c Rafael J. Wysocki 2014-02-11 954 goto out;
2d984ad132a87c Rafael J. Wysocki 2014-02-11 955 }
2d984ad132a87c Rafael J. Wysocki 2014-02-11 956 dev->power.qos->latency_tolerance_req = req;
2d984ad132a87c Rafael J. Wysocki 2014-02-11 957 } else {
2d7e4629d7265d Rob Clark 2023-03-20 958 /*
2d7e4629d7265d Rob Clark 2023-03-20 959 * If we raced with another thread to allocate the request,
2d7e4629d7265d Rob Clark 2023-03-20 960 * simply free the redundant allocation and move on.
2d7e4629d7265d Rob Clark 2023-03-20 961 */
2d7e4629d7265d Rob Clark 2023-03-20 962 if (req)
2d7e4629d7265d Rob Clark 2023-03-20 963 kfree(req);
2d7e4629d7265d Rob Clark 2023-03-20 964
2d984ad132a87c Rafael J. Wysocki 2014-02-11 965 if (val < 0) {
2d984ad132a87c Rafael J. Wysocki 2014-02-11 966 __dev_pm_qos_drop_user_request(dev, DEV_PM_QOS_LATENCY_TOLERANCE);
2d984ad132a87c Rafael J. Wysocki 2014-02-11 967 ret = 0;
2d984ad132a87c Rafael J. Wysocki 2014-02-11 968 } else {
2d984ad132a87c Rafael J. Wysocki 2014-02-11 969 ret = __dev_pm_qos_update_request(dev->power.qos->latency_tolerance_req, val);
2d984ad132a87c Rafael J. Wysocki 2014-02-11 970 }
2d984ad132a87c Rafael J. Wysocki 2014-02-11 971 }
2d984ad132a87c Rafael J. Wysocki 2014-02-11 972
2d984ad132a87c Rafael J. Wysocki 2014-02-11 973 out:
2d984ad132a87c Rafael J. Wysocki 2014-02-11 974 mutex_unlock(&dev_pm_qos_mtx);
2d984ad132a87c Rafael J. Wysocki 2014-02-11 975 return ret;
2d984ad132a87c Rafael J. Wysocki 2014-02-11 976 }
034e7906211c18 Andrew Lutomirski 2016-11-29 977 EXPORT_SYMBOL_GPL(dev_pm_qos_update_user_latency_tolerance);
13b2c4a0c3b1cd Mika Westerberg 2015-07-27 978
--
0-DAY CI Kernel Test Service
https://github.com/intel/lkp-tests
This is good. Hopefully it helps us catch lockdep bugs.
Reviewed-by: Luben Tuikov <[email protected]>
Regards,
Luben
On 2023-03-20 10:43, Rob Clark wrote:
> From: Rob Clark <[email protected]>
>
> Based on
> https://lore.kernel.org/dri-devel/[email protected]/
> but made to be optional.
>
> Signed-off-by: Rob Clark <[email protected]>
> ---
> drivers/gpu/drm/msm/msm_ringbuffer.c | 1 +
> drivers/gpu/drm/scheduler/sched_main.c | 9 +++++++++
> include/drm/gpu_scheduler.h | 2 ++
> 3 files changed, 12 insertions(+)
>
> diff --git a/drivers/gpu/drm/msm/msm_ringbuffer.c b/drivers/gpu/drm/msm/msm_ringbuffer.c
> index b60199184409..7e42baf16cd0 100644
> --- a/drivers/gpu/drm/msm/msm_ringbuffer.c
> +++ b/drivers/gpu/drm/msm/msm_ringbuffer.c
> @@ -93,6 +93,7 @@ struct msm_ringbuffer *msm_ringbuffer_new(struct msm_gpu *gpu, int id,
> /* currently managing hangcheck ourselves: */
> sched_timeout = MAX_SCHEDULE_TIMEOUT;
>
> + ring->sched.fence_signaling = true;
> ret = drm_sched_init(&ring->sched, &msm_sched_ops,
> num_hw_submissions, 0, sched_timeout,
> NULL, NULL, to_msm_bo(ring->bo)->name, gpu->dev->dev);
> diff --git a/drivers/gpu/drm/scheduler/sched_main.c b/drivers/gpu/drm/scheduler/sched_main.c
> index 4e6ad6e122bc..c2ee44d6224b 100644
> --- a/drivers/gpu/drm/scheduler/sched_main.c
> +++ b/drivers/gpu/drm/scheduler/sched_main.c
> @@ -978,10 +978,15 @@ static bool drm_sched_blocked(struct drm_gpu_scheduler *sched)
> static int drm_sched_main(void *param)
> {
> struct drm_gpu_scheduler *sched = (struct drm_gpu_scheduler *)param;
> + const bool fence_signaling = sched->fence_signaling;
> + bool fence_cookie;
> int r;
>
> sched_set_fifo_low(current);
>
> + if (fence_signaling)
> + fence_cookie = dma_fence_begin_signalling();
> +
> while (!kthread_should_stop()) {
> struct drm_sched_entity *entity = NULL;
> struct drm_sched_fence *s_fence;
> @@ -1039,6 +1044,10 @@ static int drm_sched_main(void *param)
>
> wake_up(&sched->job_scheduled);
> }
> +
> + if (fence_signaling)
> + dma_fence_end_signalling(fence_cookie);
> +
> return 0;
> }
>
> diff --git a/include/drm/gpu_scheduler.h b/include/drm/gpu_scheduler.h
> index 9db9e5e504ee..8f23ea522e22 100644
> --- a/include/drm/gpu_scheduler.h
> +++ b/include/drm/gpu_scheduler.h
> @@ -483,6 +483,7 @@ struct drm_sched_backend_ops {
> * @ready: marks if the underlying HW is ready to work
> * @free_guilty: A hit to time out handler to free the guilty job.
> * @dev: system &struct device
> + * @fence_signaling: Opt in to fence signaling annotations
> *
> * One scheduler is implemented for each hardware ring.
> */
> @@ -507,6 +508,7 @@ struct drm_gpu_scheduler {
> bool ready;
> bool free_guilty;
> struct device *dev;
> + bool fence_signaling;
> };
>
> int drm_sched_init(struct drm_gpu_scheduler *sched,
Hi Rob,
https://git-scm.com/docs/git-format-patch#_base_tree_information]
url: https://github.com/intel-lab-lkp/linux/commits/Rob-Clark/drm-msm-Pre-allocate-hw_fence/20230320-224826
base: git://anongit.freedesktop.org/drm/drm-misc drm-misc-next
patch link: https://lore.kernel.org/r/20230320144356.803762-19-robdclark%40gmail.com
patch subject: [PATCH v2 18/23] PM / QoS: Decouple request alloc from dev_pm_qos_mtx
config: arm64-randconfig-m041-20230319 (https://download.01.org/0day-ci/archive/20230321/[email protected]/config)
compiler: aarch64-linux-gcc (GCC) 12.1.0
If you fix the issue, kindly add following tag where applicable
| Reported-by: kernel test robot <[email protected]>
| Reported-by: Dan Carpenter <[email protected]>
| Link: https://lore.kernel.org/r/[email protected]/
smatch warnings:
drivers/base/power/qos.c:947 dev_pm_qos_update_user_latency_tolerance() error: uninitialized symbol 'req'.
drivers/base/power/qos.c:975 dev_pm_qos_update_user_latency_tolerance() warn: possible memory leak of 'req'
vim +/req +947 drivers/base/power/qos.c
2d984ad132a87c Rafael J. Wysocki 2014-02-11 923 int dev_pm_qos_update_user_latency_tolerance(struct device *dev, s32 val)
2d984ad132a87c Rafael J. Wysocki 2014-02-11 924 {
2d7e4629d7265d Rob Clark 2023-03-20 925 struct dev_pm_qos_request *req = NULL;
2d984ad132a87c Rafael J. Wysocki 2014-02-11 926 int ret;
2d984ad132a87c Rafael J. Wysocki 2014-02-11 927
00dd582e52a535 Rob Clark 2023-03-20 928 ret = dev_pm_qos_constraints_ensure_allocated(dev);
00dd582e52a535 Rob Clark 2023-03-20 929 if (ret)
00dd582e52a535 Rob Clark 2023-03-20 930 return ret;
00dd582e52a535 Rob Clark 2023-03-20 931
2d7e4629d7265d Rob Clark 2023-03-20 932 if (!dev->power.qos->latency_tolerance_req)
2d7e4629d7265d Rob Clark 2023-03-20 933 req = kzalloc(sizeof(*req), GFP_KERNEL);
2d7e4629d7265d Rob Clark 2023-03-20 934
2d984ad132a87c Rafael J. Wysocki 2014-02-11 935 mutex_lock(&dev_pm_qos_mtx);
2d984ad132a87c Rafael J. Wysocki 2014-02-11 936
00dd582e52a535 Rob Clark 2023-03-20 937 if (!dev->power.qos->latency_tolerance_req) {
2d984ad132a87c Rafael J. Wysocki 2014-02-11 938 struct dev_pm_qos_request *req;
This "req" shadows the ealier req.
2d984ad132a87c Rafael J. Wysocki 2014-02-11 939
2d984ad132a87c Rafael J. Wysocki 2014-02-11 940 if (val < 0) {
80a6f7c79b7822 Andrew Lutomirski 2016-11-29 941 if (val == PM_QOS_LATENCY_TOLERANCE_NO_CONSTRAINT)
80a6f7c79b7822 Andrew Lutomirski 2016-11-29 942 ret = 0;
80a6f7c79b7822 Andrew Lutomirski 2016-11-29 943 else
2d984ad132a87c Rafael J. Wysocki 2014-02-11 944 ret = -EINVAL;
2d984ad132a87c Rafael J. Wysocki 2014-02-11 945 goto out;
2d984ad132a87c Rafael J. Wysocki 2014-02-11 946 }
2d984ad132a87c Rafael J. Wysocki 2014-02-11 @947 if (!req) {
So it leads to an unintialized variable and a leak.
2d984ad132a87c Rafael J. Wysocki 2014-02-11 948 ret = -ENOMEM;
2d984ad132a87c Rafael J. Wysocki 2014-02-11 949 goto out;
2d984ad132a87c Rafael J. Wysocki 2014-02-11 950 }
2d984ad132a87c Rafael J. Wysocki 2014-02-11 951 ret = __dev_pm_qos_add_request(dev, req, DEV_PM_QOS_LATENCY_TOLERANCE, val);
2d984ad132a87c Rafael J. Wysocki 2014-02-11 952 if (ret < 0) {
2d984ad132a87c Rafael J. Wysocki 2014-02-11 953 kfree(req);
2d984ad132a87c Rafael J. Wysocki 2014-02-11 954 goto out;
2d984ad132a87c Rafael J. Wysocki 2014-02-11 955 }
2d984ad132a87c Rafael J. Wysocki 2014-02-11 956 dev->power.qos->latency_tolerance_req = req;
2d984ad132a87c Rafael J. Wysocki 2014-02-11 957 } else {
2d7e4629d7265d Rob Clark 2023-03-20 958 /*
2d7e4629d7265d Rob Clark 2023-03-20 959 * If we raced with another thread to allocate the request,
2d7e4629d7265d Rob Clark 2023-03-20 960 * simply free the redundant allocation and move on.
2d7e4629d7265d Rob Clark 2023-03-20 961 */
2d7e4629d7265d Rob Clark 2023-03-20 962 if (req)
2d7e4629d7265d Rob Clark 2023-03-20 963 kfree(req);
2d7e4629d7265d Rob Clark 2023-03-20 964
2d984ad132a87c Rafael J. Wysocki 2014-02-11 965 if (val < 0) {
2d984ad132a87c Rafael J. Wysocki 2014-02-11 966 __dev_pm_qos_drop_user_request(dev, DEV_PM_QOS_LATENCY_TOLERANCE);
2d984ad132a87c Rafael J. Wysocki 2014-02-11 967 ret = 0;
2d984ad132a87c Rafael J. Wysocki 2014-02-11 968 } else {
2d984ad132a87c Rafael J. Wysocki 2014-02-11 969 ret = __dev_pm_qos_update_request(dev->power.qos->latency_tolerance_req, val);
2d984ad132a87c Rafael J. Wysocki 2014-02-11 970 }
2d984ad132a87c Rafael J. Wysocki 2014-02-11 971 }
2d984ad132a87c Rafael J. Wysocki 2014-02-11 972
2d984ad132a87c Rafael J. Wysocki 2014-02-11 973 out:
2d984ad132a87c Rafael J. Wysocki 2014-02-11 974 mutex_unlock(&dev_pm_qos_mtx);
2d984ad132a87c Rafael J. Wysocki 2014-02-11 @975 return ret;
2d984ad132a87c Rafael J. Wysocki 2014-02-11 976 }
--
0-DAY CI Kernel Test Service
https://github.com/intel/lkp-tests
On Mon, Mar 20, 2023 at 3:45 PM Rob Clark <[email protected]> wrote:
>
> From: Rob Clark <[email protected]>
>
> In the process of adding lockdep annotation for drm GPU scheduler's
> job_run() to detect potential deadlock against shrinker/reclaim, I hit
> this lockdep splat:
>
> ======================================================
> WARNING: possible circular locking dependency detected
> 6.2.0-rc8-debug+ #558 Tainted: G W
> ------------------------------------------------------
> ring0/125 is trying to acquire lock:
> ffffffd6d6ce0f28 (dev_pm_qos_mtx){+.+.}-{3:3}, at: dev_pm_qos_update_request+0x38/0x68
>
> but task is already holding lock:
> ffffff8087239208 (&gpu->active_lock){+.+.}-{3:3}, at: msm_gpu_submit+0xec/0x178
>
> which lock already depends on the new lock.
>
> the existing dependency chain (in reverse order) is:
>
> -> #4 (&gpu->active_lock){+.+.}-{3:3}:
> __mutex_lock+0xcc/0x3c8
> mutex_lock_nested+0x30/0x44
> msm_gpu_submit+0xec/0x178
> msm_job_run+0x78/0x150
> drm_sched_main+0x290/0x370
> kthread+0xf0/0x100
> ret_from_fork+0x10/0x20
>
> -> #3 (dma_fence_map){++++}-{0:0}:
> __dma_fence_might_wait+0x74/0xc0
> dma_resv_lockdep+0x1f4/0x2f4
> do_one_initcall+0x104/0x2bc
> kernel_init_freeable+0x344/0x34c
> kernel_init+0x30/0x134
> ret_from_fork+0x10/0x20
>
> -> #2 (mmu_notifier_invalidate_range_start){+.+.}-{0:0}:
> fs_reclaim_acquire+0x80/0xa8
> slab_pre_alloc_hook.constprop.0+0x40/0x25c
> __kmem_cache_alloc_node+0x60/0x1cc
> __kmalloc+0xd8/0x100
> topology_parse_cpu_capacity+0x8c/0x178
> get_cpu_for_node+0x88/0xc4
> parse_cluster+0x1b0/0x28c
> parse_cluster+0x8c/0x28c
> init_cpu_topology+0x168/0x188
> smp_prepare_cpus+0x24/0xf8
> kernel_init_freeable+0x18c/0x34c
> kernel_init+0x30/0x134
> ret_from_fork+0x10/0x20
>
> -> #1 (fs_reclaim){+.+.}-{0:0}:
> __fs_reclaim_acquire+0x3c/0x48
> fs_reclaim_acquire+0x54/0xa8
> slab_pre_alloc_hook.constprop.0+0x40/0x25c
> __kmem_cache_alloc_node+0x60/0x1cc
> kmalloc_trace+0x50/0xa8
> dev_pm_qos_constraints_allocate+0x38/0x100
> __dev_pm_qos_add_request+0xb0/0x1e8
> dev_pm_qos_add_request+0x58/0x80
> dev_pm_qos_expose_latency_limit+0x60/0x13c
> register_cpu+0x12c/0x130
> topology_init+0xac/0xbc
> do_one_initcall+0x104/0x2bc
> kernel_init_freeable+0x344/0x34c
> kernel_init+0x30/0x134
> ret_from_fork+0x10/0x20
>
> -> #0 (dev_pm_qos_mtx){+.+.}-{3:3}:
> __lock_acquire+0xe00/0x1060
> lock_acquire+0x1e0/0x2f8
> __mutex_lock+0xcc/0x3c8
> mutex_lock_nested+0x30/0x44
> dev_pm_qos_update_request+0x38/0x68
> msm_devfreq_boost+0x40/0x70
> msm_devfreq_active+0xc0/0xf0
> msm_gpu_submit+0x10c/0x178
> msm_job_run+0x78/0x150
> drm_sched_main+0x290/0x370
> kthread+0xf0/0x100
> ret_from_fork+0x10/0x20
>
> other info that might help us debug this:
>
> Chain exists of:
> dev_pm_qos_mtx --> dma_fence_map --> &gpu->active_lock
>
> Possible unsafe locking scenario:
>
> CPU0 CPU1
> ---- ----
> lock(&gpu->active_lock);
> lock(dma_fence_map);
> lock(&gpu->active_lock);
> lock(dev_pm_qos_mtx);
>
> *** DEADLOCK ***
>
> 3 locks held by ring0/123:
> #0: ffffff8087251170 (&gpu->lock){+.+.}-{3:3}, at: msm_job_run+0x64/0x150
> #1: ffffffd00b0e57e8 (dma_fence_map){++++}-{0:0}, at: msm_job_run+0x68/0x150
> #2: ffffff8087251208 (&gpu->active_lock){+.+.}-{3:3}, at: msm_gpu_submit+0xec/0x178
>
> stack backtrace:
> CPU: 6 PID: 123 Comm: ring0 Not tainted 6.2.0-rc8-debug+ #559
> Hardware name: Google Lazor (rev1 - 2) with LTE (DT)
> Call trace:
> dump_backtrace.part.0+0xb4/0xf8
> show_stack+0x20/0x38
> dump_stack_lvl+0x9c/0xd0
> dump_stack+0x18/0x34
> print_circular_bug+0x1b4/0x1f0
> check_noncircular+0x78/0xac
> __lock_acquire+0xe00/0x1060
> lock_acquire+0x1e0/0x2f8
> __mutex_lock+0xcc/0x3c8
> mutex_lock_nested+0x30/0x44
> dev_pm_qos_update_request+0x38/0x68
> msm_devfreq_boost+0x40/0x70
> msm_devfreq_active+0xc0/0xf0
> msm_gpu_submit+0x10c/0x178
> msm_job_run+0x78/0x150
> drm_sched_main+0x290/0x370
> kthread+0xf0/0x100
> ret_from_fork+0x10/0x20
>
> The issue is that dev_pm_qos_mtx is held in the runpm suspend/resume (or
> freq change) path, but it is also held across allocations that could
> recurse into shrinker.
>
> Solve this by changing dev_pm_qos_constraints_allocate() into a function
> that can be called unconditionally before the device qos object is
> needed and before aquiring dev_pm_qos_mtx. This way the allocations can
> be done without holding the mutex. In the case that we raced with
> another thread to allocate the qos object, detect this *after* acquiring
> the dev_pm_qos_mtx and simply free the redundant allocations.
Honestly, I don't like this approach.
In particular, dropping a lock just in order to grab it again right
away is really confusing (and I'm not even sure it is correct ATM).
Let me think about how to possibly address the issue at hand in a different way.
> Signed-off-by: Rob Clark <[email protected]>
> ---
> drivers/base/power/qos.c | 60 +++++++++++++++++++++++++++-------------
> 1 file changed, 41 insertions(+), 19 deletions(-)
>
> diff --git a/drivers/base/power/qos.c b/drivers/base/power/qos.c
> index 8e93167f1783..f3e0c6b65635 100644
> --- a/drivers/base/power/qos.c
> +++ b/drivers/base/power/qos.c
> @@ -185,18 +185,24 @@ static int apply_constraint(struct dev_pm_qos_request *req,
> }
>
> /*
> - * dev_pm_qos_constraints_allocate
> + * dev_pm_qos_constraints_ensure_allocated
> * @dev: device to allocate data for
> *
> - * Called at the first call to add_request, for constraint data allocation
> - * Must be called with the dev_pm_qos_mtx mutex held
> + * Called to ensure that devices qos is allocated, before acquiring
> + * dev_pm_qos_mtx.
> */
> -static int dev_pm_qos_constraints_allocate(struct device *dev)
> +static int dev_pm_qos_constraints_ensure_allocated(struct device *dev)
> {
> struct dev_pm_qos *qos;
> struct pm_qos_constraints *c;
> struct blocking_notifier_head *n;
>
> + if (!dev)
> + return -ENODEV;
> +
> + if (!IS_ERR_OR_NULL(dev->power.qos))
> + return 0;
> +
> qos = kzalloc(sizeof(*qos), GFP_KERNEL);
> if (!qos)
> return -ENOMEM;
> @@ -227,10 +233,26 @@ static int dev_pm_qos_constraints_allocate(struct device *dev)
>
> INIT_LIST_HEAD(&qos->flags.list);
>
> + mutex_lock(&dev_pm_qos_mtx);
> +
> + if (!IS_ERR_OR_NULL(dev->power.qos)) {
> + /*
> + * We have raced with another task to create the qos.
> + * No biggie, just free the resources we've allocated
> + * outside of dev_pm_qos_mtx and move on with life.
> + */
> + kfree(n);
> + kfree(qos);
> + goto unlock;
> + }
> +
> spin_lock_irq(&dev->power.lock);
> dev->power.qos = qos;
> spin_unlock_irq(&dev->power.lock);
>
> +unlock:
> + mutex_unlock(&dev_pm_qos_mtx);
> +
> return 0;
> }
>
> @@ -331,17 +353,15 @@ static int __dev_pm_qos_add_request(struct device *dev,
> {
> int ret = 0;
>
> - if (!dev || !req || dev_pm_qos_invalid_req_type(dev, type))
> + if (!req || dev_pm_qos_invalid_req_type(dev, type))
> return -EINVAL;
>
> if (WARN(dev_pm_qos_request_active(req),
> "%s() called for already added request\n", __func__))
> return -EINVAL;
>
> - if (IS_ERR(dev->power.qos))
> + if (IS_ERR_OR_NULL(dev->power.qos))
> ret = -ENODEV;
> - else if (!dev->power.qos)
> - ret = dev_pm_qos_constraints_allocate(dev);
>
> trace_dev_pm_qos_add_request(dev_name(dev), type, value);
> if (ret)
> @@ -390,6 +410,10 @@ int dev_pm_qos_add_request(struct device *dev, struct dev_pm_qos_request *req,
> {
> int ret;
>
> + ret = dev_pm_qos_constraints_ensure_allocated(dev);
> + if (ret)
> + return ret;
> +
> mutex_lock(&dev_pm_qos_mtx);
> ret = __dev_pm_qos_add_request(dev, req, type, value);
> mutex_unlock(&dev_pm_qos_mtx);
> @@ -537,15 +561,11 @@ int dev_pm_qos_add_notifier(struct device *dev, struct notifier_block *notifier,
> {
> int ret = 0;
>
> - mutex_lock(&dev_pm_qos_mtx);
> -
> - if (IS_ERR(dev->power.qos))
> - ret = -ENODEV;
> - else if (!dev->power.qos)
> - ret = dev_pm_qos_constraints_allocate(dev);
> -
> + ret = dev_pm_qos_constraints_ensure_allocated(dev);
> if (ret)
> - goto unlock;
> + return ret;
> +
> + mutex_lock(&dev_pm_qos_mtx);
>
> switch (type) {
> case DEV_PM_QOS_RESUME_LATENCY:
> @@ -565,7 +585,6 @@ int dev_pm_qos_add_notifier(struct device *dev, struct notifier_block *notifier,
> ret = -EINVAL;
> }
>
> -unlock:
> mutex_unlock(&dev_pm_qos_mtx);
> return ret;
> }
> @@ -905,10 +924,13 @@ int dev_pm_qos_update_user_latency_tolerance(struct device *dev, s32 val)
> {
> int ret;
>
> + ret = dev_pm_qos_constraints_ensure_allocated(dev);
> + if (ret)
> + return ret;
> +
> mutex_lock(&dev_pm_qos_mtx);
>
> - if (IS_ERR_OR_NULL(dev->power.qos)
> - || !dev->power.qos->latency_tolerance_req) {
> + if (!dev->power.qos->latency_tolerance_req) {
> struct dev_pm_qos_request *req;
>
> if (val < 0) {
> --
On Mon, Mar 27, 2023 at 10:53 AM Rafael J. Wysocki <[email protected]> wrote:
>
> On Mon, Mar 20, 2023 at 3:45 PM Rob Clark <[email protected]> wrote:
> >
> > From: Rob Clark <[email protected]>
> >
> > In the process of adding lockdep annotation for drm GPU scheduler's
> > job_run() to detect potential deadlock against shrinker/reclaim, I hit
> > this lockdep splat:
> >
> > ======================================================
> > WARNING: possible circular locking dependency detected
> > 6.2.0-rc8-debug+ #558 Tainted: G W
> > ------------------------------------------------------
> > ring0/125 is trying to acquire lock:
> > ffffffd6d6ce0f28 (dev_pm_qos_mtx){+.+.}-{3:3}, at: dev_pm_qos_update_request+0x38/0x68
> >
> > but task is already holding lock:
> > ffffff8087239208 (&gpu->active_lock){+.+.}-{3:3}, at: msm_gpu_submit+0xec/0x178
> >
> > which lock already depends on the new lock.
> >
> > the existing dependency chain (in reverse order) is:
> >
> > -> #4 (&gpu->active_lock){+.+.}-{3:3}:
> > __mutex_lock+0xcc/0x3c8
> > mutex_lock_nested+0x30/0x44
> > msm_gpu_submit+0xec/0x178
> > msm_job_run+0x78/0x150
> > drm_sched_main+0x290/0x370
> > kthread+0xf0/0x100
> > ret_from_fork+0x10/0x20
> >
> > -> #3 (dma_fence_map){++++}-{0:0}:
> > __dma_fence_might_wait+0x74/0xc0
> > dma_resv_lockdep+0x1f4/0x2f4
> > do_one_initcall+0x104/0x2bc
> > kernel_init_freeable+0x344/0x34c
> > kernel_init+0x30/0x134
> > ret_from_fork+0x10/0x20
> >
> > -> #2 (mmu_notifier_invalidate_range_start){+.+.}-{0:0}:
> > fs_reclaim_acquire+0x80/0xa8
> > slab_pre_alloc_hook.constprop.0+0x40/0x25c
> > __kmem_cache_alloc_node+0x60/0x1cc
> > __kmalloc+0xd8/0x100
> > topology_parse_cpu_capacity+0x8c/0x178
> > get_cpu_for_node+0x88/0xc4
> > parse_cluster+0x1b0/0x28c
> > parse_cluster+0x8c/0x28c
> > init_cpu_topology+0x168/0x188
> > smp_prepare_cpus+0x24/0xf8
> > kernel_init_freeable+0x18c/0x34c
> > kernel_init+0x30/0x134
> > ret_from_fork+0x10/0x20
> >
> > -> #1 (fs_reclaim){+.+.}-{0:0}:
> > __fs_reclaim_acquire+0x3c/0x48
> > fs_reclaim_acquire+0x54/0xa8
> > slab_pre_alloc_hook.constprop.0+0x40/0x25c
> > __kmem_cache_alloc_node+0x60/0x1cc
> > kmalloc_trace+0x50/0xa8
> > dev_pm_qos_constraints_allocate+0x38/0x100
> > __dev_pm_qos_add_request+0xb0/0x1e8
> > dev_pm_qos_add_request+0x58/0x80
> > dev_pm_qos_expose_latency_limit+0x60/0x13c
> > register_cpu+0x12c/0x130
> > topology_init+0xac/0xbc
> > do_one_initcall+0x104/0x2bc
> > kernel_init_freeable+0x344/0x34c
> > kernel_init+0x30/0x134
> > ret_from_fork+0x10/0x20
> >
> > -> #0 (dev_pm_qos_mtx){+.+.}-{3:3}:
> > __lock_acquire+0xe00/0x1060
> > lock_acquire+0x1e0/0x2f8
> > __mutex_lock+0xcc/0x3c8
> > mutex_lock_nested+0x30/0x44
> > dev_pm_qos_update_request+0x38/0x68
> > msm_devfreq_boost+0x40/0x70
> > msm_devfreq_active+0xc0/0xf0
> > msm_gpu_submit+0x10c/0x178
> > msm_job_run+0x78/0x150
> > drm_sched_main+0x290/0x370
> > kthread+0xf0/0x100
> > ret_from_fork+0x10/0x20
> >
> > other info that might help us debug this:
> >
> > Chain exists of:
> > dev_pm_qos_mtx --> dma_fence_map --> &gpu->active_lock
> >
> > Possible unsafe locking scenario:
> >
> > CPU0 CPU1
> > ---- ----
> > lock(&gpu->active_lock);
> > lock(dma_fence_map);
> > lock(&gpu->active_lock);
> > lock(dev_pm_qos_mtx);
> >
> > *** DEADLOCK ***
> >
> > 3 locks held by ring0/123:
> > #0: ffffff8087251170 (&gpu->lock){+.+.}-{3:3}, at: msm_job_run+0x64/0x150
> > #1: ffffffd00b0e57e8 (dma_fence_map){++++}-{0:0}, at: msm_job_run+0x68/0x150
> > #2: ffffff8087251208 (&gpu->active_lock){+.+.}-{3:3}, at: msm_gpu_submit+0xec/0x178
> >
> > stack backtrace:
> > CPU: 6 PID: 123 Comm: ring0 Not tainted 6.2.0-rc8-debug+ #559
> > Hardware name: Google Lazor (rev1 - 2) with LTE (DT)
> > Call trace:
> > dump_backtrace.part.0+0xb4/0xf8
> > show_stack+0x20/0x38
> > dump_stack_lvl+0x9c/0xd0
> > dump_stack+0x18/0x34
> > print_circular_bug+0x1b4/0x1f0
> > check_noncircular+0x78/0xac
> > __lock_acquire+0xe00/0x1060
> > lock_acquire+0x1e0/0x2f8
> > __mutex_lock+0xcc/0x3c8
> > mutex_lock_nested+0x30/0x44
> > dev_pm_qos_update_request+0x38/0x68
> > msm_devfreq_boost+0x40/0x70
> > msm_devfreq_active+0xc0/0xf0
> > msm_gpu_submit+0x10c/0x178
> > msm_job_run+0x78/0x150
> > drm_sched_main+0x290/0x370
> > kthread+0xf0/0x100
> > ret_from_fork+0x10/0x20
> >
> > The issue is that dev_pm_qos_mtx is held in the runpm suspend/resume (or
> > freq change) path, but it is also held across allocations that could
> > recurse into shrinker.
> >
> > Solve this by changing dev_pm_qos_constraints_allocate() into a function
> > that can be called unconditionally before the device qos object is
> > needed and before aquiring dev_pm_qos_mtx. This way the allocations can
> > be done without holding the mutex. In the case that we raced with
> > another thread to allocate the qos object, detect this *after* acquiring
> > the dev_pm_qos_mtx and simply free the redundant allocations.
>
> Honestly, I don't like this approach.
>
> In particular, dropping a lock just in order to grab it again right
> away is really confusing (and I'm not even sure it is correct ATM).
This patch isn't actually doing that. (And you are right, if it were,
that would be a thing to be suspicious of)
It is just moving the allocations ahead of the locking.
> Let me think about how to possibly address the issue at hand in a different way.
Per device locking would make this easier. But I suppose that gets
into needing ww_mutex when you have things like device_link?
BR,
-R
> > Signed-off-by: Rob Clark <[email protected]>
> > ---
> > drivers/base/power/qos.c | 60 +++++++++++++++++++++++++++-------------
> > 1 file changed, 41 insertions(+), 19 deletions(-)
> >
> > diff --git a/drivers/base/power/qos.c b/drivers/base/power/qos.c
> > index 8e93167f1783..f3e0c6b65635 100644
> > --- a/drivers/base/power/qos.c
> > +++ b/drivers/base/power/qos.c
> > @@ -185,18 +185,24 @@ static int apply_constraint(struct dev_pm_qos_request *req,
> > }
> >
> > /*
> > - * dev_pm_qos_constraints_allocate
> > + * dev_pm_qos_constraints_ensure_allocated
> > * @dev: device to allocate data for
> > *
> > - * Called at the first call to add_request, for constraint data allocation
> > - * Must be called with the dev_pm_qos_mtx mutex held
> > + * Called to ensure that devices qos is allocated, before acquiring
> > + * dev_pm_qos_mtx.
> > */
> > -static int dev_pm_qos_constraints_allocate(struct device *dev)
> > +static int dev_pm_qos_constraints_ensure_allocated(struct device *dev)
> > {
> > struct dev_pm_qos *qos;
> > struct pm_qos_constraints *c;
> > struct blocking_notifier_head *n;
> >
> > + if (!dev)
> > + return -ENODEV;
> > +
> > + if (!IS_ERR_OR_NULL(dev->power.qos))
> > + return 0;
> > +
> > qos = kzalloc(sizeof(*qos), GFP_KERNEL);
> > if (!qos)
> > return -ENOMEM;
> > @@ -227,10 +233,26 @@ static int dev_pm_qos_constraints_allocate(struct device *dev)
> >
> > INIT_LIST_HEAD(&qos->flags.list);
> >
> > + mutex_lock(&dev_pm_qos_mtx);
> > +
> > + if (!IS_ERR_OR_NULL(dev->power.qos)) {
> > + /*
> > + * We have raced with another task to create the qos.
> > + * No biggie, just free the resources we've allocated
> > + * outside of dev_pm_qos_mtx and move on with life.
> > + */
> > + kfree(n);
> > + kfree(qos);
> > + goto unlock;
> > + }
> > +
> > spin_lock_irq(&dev->power.lock);
> > dev->power.qos = qos;
> > spin_unlock_irq(&dev->power.lock);
> >
> > +unlock:
> > + mutex_unlock(&dev_pm_qos_mtx);
> > +
> > return 0;
> > }
> >
> > @@ -331,17 +353,15 @@ static int __dev_pm_qos_add_request(struct device *dev,
> > {
> > int ret = 0;
> >
> > - if (!dev || !req || dev_pm_qos_invalid_req_type(dev, type))
> > + if (!req || dev_pm_qos_invalid_req_type(dev, type))
> > return -EINVAL;
> >
> > if (WARN(dev_pm_qos_request_active(req),
> > "%s() called for already added request\n", __func__))
> > return -EINVAL;
> >
> > - if (IS_ERR(dev->power.qos))
> > + if (IS_ERR_OR_NULL(dev->power.qos))
> > ret = -ENODEV;
> > - else if (!dev->power.qos)
> > - ret = dev_pm_qos_constraints_allocate(dev);
> >
> > trace_dev_pm_qos_add_request(dev_name(dev), type, value);
> > if (ret)
> > @@ -390,6 +410,10 @@ int dev_pm_qos_add_request(struct device *dev, struct dev_pm_qos_request *req,
> > {
> > int ret;
> >
> > + ret = dev_pm_qos_constraints_ensure_allocated(dev);
> > + if (ret)
> > + return ret;
> > +
> > mutex_lock(&dev_pm_qos_mtx);
> > ret = __dev_pm_qos_add_request(dev, req, type, value);
> > mutex_unlock(&dev_pm_qos_mtx);
> > @@ -537,15 +561,11 @@ int dev_pm_qos_add_notifier(struct device *dev, struct notifier_block *notifier,
> > {
> > int ret = 0;
> >
> > - mutex_lock(&dev_pm_qos_mtx);
> > -
> > - if (IS_ERR(dev->power.qos))
> > - ret = -ENODEV;
> > - else if (!dev->power.qos)
> > - ret = dev_pm_qos_constraints_allocate(dev);
> > -
> > + ret = dev_pm_qos_constraints_ensure_allocated(dev);
> > if (ret)
> > - goto unlock;
> > + return ret;
> > +
> > + mutex_lock(&dev_pm_qos_mtx);
> >
> > switch (type) {
> > case DEV_PM_QOS_RESUME_LATENCY:
> > @@ -565,7 +585,6 @@ int dev_pm_qos_add_notifier(struct device *dev, struct notifier_block *notifier,
> > ret = -EINVAL;
> > }
> >
> > -unlock:
> > mutex_unlock(&dev_pm_qos_mtx);
> > return ret;
> > }
> > @@ -905,10 +924,13 @@ int dev_pm_qos_update_user_latency_tolerance(struct device *dev, s32 val)
> > {
> > int ret;
> >
> > + ret = dev_pm_qos_constraints_ensure_allocated(dev);
> > + if (ret)
> > + return ret;
> > +
> > mutex_lock(&dev_pm_qos_mtx);
> >
> > - if (IS_ERR_OR_NULL(dev->power.qos)
> > - || !dev->power.qos->latency_tolerance_req) {
> > + if (!dev->power.qos->latency_tolerance_req) {
> > struct dev_pm_qos_request *req;
> >
> > if (val < 0) {
> > --
On Mon, 20 Mar 2023 07:43:22 -0700, Rob Clark wrote:
> From: Rob Clark <[email protected]>
>
> Inspired by https://lore.kernel.org/dri-devel/[email protected]/
> it seemed like a good idea to get rid of memory allocation in job_run()
> fence signaling path, and use lockdep annotations to yell at us about
> anything that could deadlock against shrinker/reclaim. Anything that
> can trigger reclaim, or block on any other thread that has triggered
> reclaim, can block the GPU shrinker from releasing memory if it is
> waiting the job to complete, causing deadlock.
>
> [...]
Applied, thanks!
[20/23] soc: qcom: smd-rpm: Use GFP_ATOMIC in write path
commit: 5808c532ca0a983d643319caca44f2bcb148298f
Best regards,
--
Bjorn Andersson <[email protected]>