2023-05-24 18:53:00

by Stephan Gerhold

[permalink] [raw]
Subject: [PATCH] opp: Fix use-after-free in lazy_opp_tables after probe deferral

When dev_pm_opp_of_find_icc_paths() in _allocate_opp_table() returns
-EPROBE_DEFER, the opp_table is freed again, to wait until all the
interconnect paths are available.

However, if the OPP table is using required-opps then it may already
have been added to the global lazy_opp_tables list. The error path
does not remove the opp_table from the list again.

This can cause crashes later when the provider of the required-opps
is added, since we will iterate over OPP tables that have already been
freed. E.g.:

Unable to handle kernel NULL pointer dereference when read
CPU: 0 PID: 7 Comm: kworker/0:0 Not tainted 6.4.0-rc3
PC is at _of_add_opp_table_v2 (include/linux/of.h:949
drivers/opp/of.c:98 drivers/opp/of.c:344 drivers/opp/of.c:404
drivers/opp/of.c:1032) -> lazy_link_required_opp_table()

Fix this by removing the opp_table from the list before freeing it.

Cc: [email protected]
Fixes: 7eba0c7641b0 ("opp: Allow lazy-linking of required-opps")
Signed-off-by: Stephan Gerhold <[email protected]>
---
This fixes the crash I ran into after adding an OPP table with
both "required-opps" and interconnect paths (opp-peak-kBps).

By the way, the "lazy_opp_tables" does not seem to be protected by any
locks(?) so I could imagine that theoretically there could be a race
condition while adding/removing OPP tables there. This is unrelated
to the crash I saw, though.
---
drivers/opp/core.c | 1 +
1 file changed, 1 insertion(+)

diff --git a/drivers/opp/core.c b/drivers/opp/core.c
index 85cbc8de407c..6a3a320be7df 100644
--- a/drivers/opp/core.c
+++ b/drivers/opp/core.c
@@ -1358,6 +1358,7 @@ static struct opp_table *_allocate_opp_table(struct device *dev, int index)
return opp_table;

remove_opp_dev:
+ list_del(&opp_table->lazy);
_remove_opp_dev(opp_dev, opp_table);
err:
kfree(opp_table);

---
base-commit: 9e28f7a74581204807f20ae46568939038e327aa
change-id: 20230524-opp-lazy-uaf-60a004b385ec

Best regards,
--
Stephan Gerhold
Kernkonzept GmbH at Dresden, Germany, HRB 31129, CEO Dr.-Ing. Michael Hohmuth



2023-05-29 05:49:10

by Viresh Kumar

[permalink] [raw]
Subject: Re: [PATCH] opp: Fix use-after-free in lazy_opp_tables after probe deferral

On 24-05-23, 19:56, Stephan Gerhold wrote:
> When dev_pm_opp_of_find_icc_paths() in _allocate_opp_table() returns
> -EPROBE_DEFER, the opp_table is freed again, to wait until all the
> interconnect paths are available.
>
> However, if the OPP table is using required-opps then it may already
> have been added to the global lazy_opp_tables list. The error path
> does not remove the opp_table from the list again.
>
> This can cause crashes later when the provider of the required-opps
> is added, since we will iterate over OPP tables that have already been
> freed. E.g.:
>
> Unable to handle kernel NULL pointer dereference when read
> CPU: 0 PID: 7 Comm: kworker/0:0 Not tainted 6.4.0-rc3
> PC is at _of_add_opp_table_v2 (include/linux/of.h:949
> drivers/opp/of.c:98 drivers/opp/of.c:344 drivers/opp/of.c:404
> drivers/opp/of.c:1032) -> lazy_link_required_opp_table()
>
> Fix this by removing the opp_table from the list before freeing it.

I think you need this instead:

diff --git a/drivers/opp/core.c b/drivers/opp/core.c
index 954c94865cf5..b5973fefdfd8 100644
--- a/drivers/opp/core.c
+++ b/drivers/opp/core.c
@@ -1358,7 +1358,10 @@ static struct opp_table *_allocate_opp_table(struct device *dev, int index)
return opp_table;

remove_opp_dev:
+ _of_clear_opp_table(opp_table);
_remove_opp_dev(opp_dev, opp_table);
+ mutex_destroy(&opp_table->genpd_virt_dev_lock);
+ mutex_destroy(&opp_table->lock);
err:
kfree(opp_table);
return ERR_PTR(ret);

> Cc: [email protected]
> Fixes: 7eba0c7641b0 ("opp: Allow lazy-linking of required-opps")
> Signed-off-by: Stephan Gerhold <[email protected]>
> ---
> This fixes the crash I ran into after adding an OPP table with
> both "required-opps" and interconnect paths (opp-peak-kBps).
>
> By the way, the "lazy_opp_tables" does not seem to be protected by any
> locks(?)

It is always accessed with opp_table_lock held I believe.

--
viresh

2023-05-30 09:12:59

by Stephan Gerhold

[permalink] [raw]
Subject: Re: [PATCH] opp: Fix use-after-free in lazy_opp_tables after probe deferral

On Mon, May 29, 2023 at 11:01:48AM +0530, Viresh Kumar wrote:
> On 24-05-23, 19:56, Stephan Gerhold wrote:
> > When dev_pm_opp_of_find_icc_paths() in _allocate_opp_table() returns
> > -EPROBE_DEFER, the opp_table is freed again, to wait until all the
> > interconnect paths are available.
> >
> > However, if the OPP table is using required-opps then it may already
> > have been added to the global lazy_opp_tables list. The error path
> > does not remove the opp_table from the list again.
> >
> > This can cause crashes later when the provider of the required-opps
> > is added, since we will iterate over OPP tables that have already been
> > freed. E.g.:
> >
> > Unable to handle kernel NULL pointer dereference when read
> > CPU: 0 PID: 7 Comm: kworker/0:0 Not tainted 6.4.0-rc3
> > PC is at _of_add_opp_table_v2 (include/linux/of.h:949
> > drivers/opp/of.c:98 drivers/opp/of.c:344 drivers/opp/of.c:404
> > drivers/opp/of.c:1032) -> lazy_link_required_opp_table()
> >
> > Fix this by removing the opp_table from the list before freeing it.
>
> I think you need this instead:
>
> diff --git a/drivers/opp/core.c b/drivers/opp/core.c
> index 954c94865cf5..b5973fefdfd8 100644
> --- a/drivers/opp/core.c
> +++ b/drivers/opp/core.c
> @@ -1358,7 +1358,10 @@ static struct opp_table *_allocate_opp_table(struct device *dev, int index)
> return opp_table;
>
> remove_opp_dev:
> + _of_clear_opp_table(opp_table);
> _remove_opp_dev(opp_dev, opp_table);
> + mutex_destroy(&opp_table->genpd_virt_dev_lock);
> + mutex_destroy(&opp_table->lock);
> err:
> kfree(opp_table);
> return ERR_PTR(ret);
>

Thanks, this seems to fix the crash as well. Are you going to handle it
or should I send a v2 with this diff?

> > Cc: [email protected]
> > Fixes: 7eba0c7641b0 ("opp: Allow lazy-linking of required-opps")
> > Signed-off-by: Stephan Gerhold <[email protected]>
> > ---
> > This fixes the crash I ran into after adding an OPP table with
> > both "required-opps" and interconnect paths (opp-peak-kBps).
> >
> > By the way, the "lazy_opp_tables" does not seem to be protected by any
> > locks(?)
>
> It is always accessed with opp_table_lock held I believe.
>

During _allocate_opp_table() it's accessed without the opp_table_lock,
because of

/* Drop the lock to reduce the size of critical section */
mutex_unlock(&opp_table_lock);

if (opp_table) {
/* ... */
mutex_lock(&opp_table_lock);
} else {
opp_table = _allocate_opp_table(dev, index);

mutex_lock(&opp_table_lock);
/* ... */
}

This doesn't seem to cause any problems in my case though so it's
unrelated to the crash I observed.

Thanks,
Stephan
--
Kernkonzept GmbH at Dresden, Germany, HRB 31129, CEO Dr.-Ing. Michael Hohmuth

2023-05-30 09:33:09

by Viresh Kumar

[permalink] [raw]
Subject: Re: [PATCH] opp: Fix use-after-free in lazy_opp_tables after probe deferral

On 30-05-23, 10:31, Stephan Gerhold wrote:
> Thanks, this seems to fix the crash as well. Are you going to handle it
> or should I send a v2 with this diff?

Please send a V2 :)

> During _allocate_opp_table() it's accessed without the opp_table_lock,
> because of
>
> /* Drop the lock to reduce the size of critical section */
> mutex_unlock(&opp_table_lock);
>
> if (opp_table) {
> /* ... */
> mutex_lock(&opp_table_lock);
> } else {
> opp_table = _allocate_opp_table(dev, index);
>
> mutex_lock(&opp_table_lock);
> /* ... */
> }
>
> This doesn't seem to cause any problems in my case though so it's
> unrelated to the crash I observed.

Hmm, right. Maybe we need a lock for that list, want to take that up ?

--
viresh

2023-05-30 16:58:56

by Stephan Gerhold

[permalink] [raw]
Subject: Re: [PATCH] opp: Fix use-after-free in lazy_opp_tables after probe deferral

On Tue, May 30, 2023 at 02:43:30PM +0530, Viresh Kumar wrote:
> On 30-05-23, 10:31, Stephan Gerhold wrote:
> > Thanks, this seems to fix the crash as well. Are you going to handle it
> > or should I send a v2 with this diff?
>
> Please send a V2 :)
>

Done!

> > During _allocate_opp_table() it's accessed without the opp_table_lock,
> > because of
> >
> > /* Drop the lock to reduce the size of critical section */
> > mutex_unlock(&opp_table_lock);
> >
> > if (opp_table) {
> > /* ... */
> > mutex_lock(&opp_table_lock);
> > } else {
> > opp_table = _allocate_opp_table(dev, index);
> >
> > mutex_lock(&opp_table_lock);
> > /* ... */
> > }
> >
> > This doesn't seem to cause any problems in my case though so it's
> > unrelated to the crash I observed.
>
> Hmm, right. Maybe we need a lock for that list, want to take that up ?
>

Yeah, a lock would probably be good to be safe. I would appreciate if
you or someone else could create a patch for this though, since I'm not
too familiar with the overall OPP implementation. I would be happy to
test that it works properly for my apparently quite special use case
(I have several OPP tables with interconnects and required-opps).

Thanks!
Stephan
--
Kernkonzept GmbH at Dresden, Germany, HRB 31129, CEO Dr.-Ing. Michael Hohmuth