2007-02-18 21:44:01

by Oleg Nesterov

[permalink] [raw]
Subject: [PATCH 1/3] net/bridge/br_if.c: don't use _WORK_NAR

Afaics, noautorel work_struct buys nothing for "struct net_bridge_port".

If del_nbp()->cancel_delayed_work(&p->carrier_check) fails, port_carrier_check
may be called later anyway. So the reading of *work in port_carrier_check() is
equally unsafe with or without this patch.

Signed-off-by: Oleg Nesterov <[email protected]>

--- WQ/net/bridge/br_if.c~1_bridge 2007-02-18 22:56:49.000000000 +0300
+++ WQ/net/bridge/br_if.c 2007-02-18 23:06:15.000000000 +0300
@@ -85,7 +85,6 @@ static void port_carrier_check(struct wo

dev = container_of(work, struct net_bridge_port,
carrier_check.work)->dev;
- work_release(work);

rtnl_lock();
p = dev->br_port;
@@ -282,7 +281,7 @@ static struct net_bridge_port *new_nbp(s
p->port_no = index;
br_init_port(p);
p->state = BR_STATE_DISABLED;
- INIT_DELAYED_WORK_NAR(&p->carrier_check, port_carrier_check);
+ INIT_DELAYED_WORK(&p->carrier_check, port_carrier_check);
br_stp_port_timer_init(p);

kobject_init(&p->kobj);


2007-02-19 10:57:00

by Jarek Poplawski

[permalink] [raw]
Subject: Re: [PATCH 1/3] net/bridge/br_if.c: don't use _WORK_NAR

On Mon, Feb 19, 2007 at 12:43:59AM +0300, Oleg Nesterov wrote:
> Afaics, noautorel work_struct buys nothing for "struct net_bridge_port".
>
> If del_nbp()->cancel_delayed_work(&p->carrier_check) fails, port_carrier_check
> may be called later anyway. So the reading of *work in port_carrier_check() is
> equally unsafe with or without this patch.

I think this _WORK_NAR is to give some additional
control, but also is more logical: it lets to decide
when the work_struct is really release-able (and it's
definitely not before work function is called, as
without noautorel).

So, even if this functionality isn't used now, I can't
see what changing this could buy.

Regards,
Jarek P.

2007-02-19 11:27:59

by David Howells

[permalink] [raw]
Subject: Re: [PATCH 1/3] net/bridge/br_if.c: don't use _WORK_NAR

Oleg Nesterov <[email protected]> wrote:

> Afaics, noautorel work_struct buys nothing for "struct net_bridge_port".

You may be right.

> If del_nbp()->cancel_delayed_work(&p->carrier_check) fails, port_carrier_check
> may be called later anyway.

Called by what? Something outside of br_if.c?

> So the reading of *work in port_carrier_check() is equally unsafe with or
> without this patch.

Hmmm... cancel_delayed_work() in del_nbp() probably ought to be followed by a
flush_scheduled_work().

David

2007-02-19 12:00:00

by Oleg Nesterov

[permalink] [raw]
Subject: Re: [PATCH 1/3] net/bridge/br_if.c: don't use _WORK_NAR

On 02/19, David Howells wrote:
>
> Oleg Nesterov <[email protected]> wrote:
>
> > Afaics, noautorel work_struct buys nothing for "struct net_bridge_port".
>
> You may be right.
>
> > If del_nbp()->cancel_delayed_work(&p->carrier_check) fails, port_carrier_check
> > may be called later anyway.
>
> Called by what? Something outside of br_if.c?

No. if cancel_delayed_work() fails, the work may sit pending in cwq->worklist,
or it may be running right now, waiting for rtnl_mutex.

> > So the reading of *work in port_carrier_check() is equally unsafe with or
> > without this patch.
>
> Hmmm... cancel_delayed_work() in del_nbp() probably ought to be followed by a
> flush_scheduled_work().

Yes, but this deadlocks: we hold rtnl_mutex, and work->func() takes it too.

I think the fix should be so that port_carrier_check() does get/put on
"struct net_bridge_port" (container), but not on "struct net_device", and

del_nbp(struct net_bridge_port *p)

if (cancel_delayed_work(&p->carrier_check))
- dev_put(p->dev);
+ kobject_put(&p->kobj);

Oleg.

2007-02-19 12:03:53

by Oleg Nesterov

[permalink] [raw]
Subject: Re: [PATCH 1/3] net/bridge/br_if.c: don't use _WORK_NAR

On 02/19, Jarek Poplawski wrote:
>
> On Mon, Feb 19, 2007 at 12:43:59AM +0300, Oleg Nesterov wrote:
> > Afaics, noautorel work_struct buys nothing for "struct net_bridge_port".
> >
> > If del_nbp()->cancel_delayed_work(&p->carrier_check) fails, port_carrier_check
> > may be called later anyway. So the reading of *work in port_carrier_check() is
> > equally unsafe with or without this patch.
>
> I think this _WORK_NAR is to give some additional
> control, but also is more logical: it lets to decide
> when the work_struct is really release-able

Sadly, it doesn't help here.

(and it's
> definitely not before work function is called, as
> without noautorel).

kfree() doesn't check WORK_STRUCT_PENDING, it makes no
difference if it is set or not when work->func() runs.

> So, even if this functionality isn't used now, I can't
> see what changing this could buy.

We are going to kill _NAR stuff.

Oleg.

2007-02-19 13:15:25

by David Howells

[permalink] [raw]
Subject: Re: [PATCH 1/3] net/bridge/br_if.c: don't use _WORK_NAR

Oleg Nesterov <[email protected]> wrote:

> > Called by what? Something outside of br_if.c?
>
> No. if cancel_delayed_work() fails, the work may sit pending in cwq->worklist,
> or it may be running right now, waiting for rtnl_mutex.

OIC. I understood "called" to mean "scheduled", but that's not what you meant.

> > Hmmm... cancel_delayed_work() in del_nbp() probably ought to be followed by
> > a flush_scheduled_work().
>
> Yes, but this deadlocks: we hold rtnl_mutex, and work->func() takes it too.

Oh, yuck!

Hmmm... You've got a work_struct (well, a delayed_work actually) - can you
just punt the destruction of the object over to keventd to perform, I wonder?

The big problem with that that I see is that the workqueue facility has no
guards in place against a work_struct's handler function running on several
CPUs at once in response to the same work_struct.

> I think the fix should be so that port_carrier_check() does get/put on
> "struct net_bridge_port" (container), but not on "struct net_device", and

I'm not sure how this helps. You still have to get rid of the net_device at
some point.

David

2007-02-19 13:23:41

by Jarek Poplawski

[permalink] [raw]
Subject: Re: [PATCH 1/3] net/bridge/br_if.c: don't use _WORK_NAR

On Mon, Feb 19, 2007 at 03:03:53PM +0300, Oleg Nesterov wrote:
> On 02/19, Jarek Poplawski wrote:
...
> kfree() doesn't check WORK_STRUCT_PENDING, it makes no
> difference if it is set or not when work->func() runs.

It looks like it's to be checked before kfree.

> > So, even if this functionality isn't used now, I can't
> > see what changing this could buy.
>
> We are going to kill _NAR stuff.

If you're sure nobody uses this in any way then it
seems the right decision.

Jarek P.

2007-02-19 14:53:27

by Oleg Nesterov

[permalink] [raw]
Subject: Re: [PATCH 1/3] net/bridge/br_if.c: don't use _WORK_NAR

On 02/19, David Howells wrote:
>
> Hmmm... You've got a work_struct (well, a delayed_work actually) - can you
> just punt the destruction of the object over to keventd to perform, I wonder?

Yes, this is close (I think) to what I suggested, see below,

> The big problem with that that I see is that the workqueue facility has no
> guards in place against a work_struct's handler function running on several
> CPUs at once in response to the same work_struct.

Yes. And for this problem WORK_STRUCT_NOAUTOREL does help, but not so much.
It can prevent re-scheduling of the same work, but only if work->func() did
not do work_release() yet.

> > I think the fix should be so that port_carrier_check() does get/put on
> > "struct net_bridge_port" (container), but not on "struct net_device", and
>
> I'm not sure how this helps. You still have to get rid of the net_device at
> some point.

Yes, destroy_nbp() does dev_put(dev). del_nbp() sets dev->br_port = NULL,
port_carrier_check() goes to "done" in that case. So everething looks safe
to me (but again, I do not even know what the "bridge" is :), so we should
only take care about container, nothing more.

I'll try to make a patch for illustration on evening.

Oleg.

2007-02-19 15:01:56

by Oleg Nesterov

[permalink] [raw]
Subject: Re: [PATCH 1/3] net/bridge/br_if.c: don't use _WORK_NAR

On 02/19, Jarek Poplawski wrote:
>
> On Mon, Feb 19, 2007 at 03:03:53PM +0300, Oleg Nesterov wrote:
> > On 02/19, Jarek Poplawski wrote:
> ...
> > kfree() doesn't check WORK_STRUCT_PENDING, it makes no
> > difference if it is set or not when work->func() runs.
>
> It looks like it's to be checked before kfree.

Here,
br_add_if:

schedule_delayed_work(&p->carrier_check, BR_PORT_DEBOUNCE);

schedule_delayed_work() fails if this bit is set. So the only difference with
this patch is:

before:
schedule_delayed_work() fails unless port_carrier_check() passed
work_release() (before rtnl_lock())

after:
schedule_delayed_work() fails unless run_workqueue() cleared this
bit (before calling port_carrier_check())

> > We are going to kill _NAR stuff.
>
> If you're sure nobody uses this in any way then it
> seems the right decision.

Yes, this series converts all users.

Oleg.

2007-02-19 15:16:04

by David Howells

[permalink] [raw]
Subject: Re: [PATCH 1/3] net/bridge/br_if.c: don't use _WORK_NAR

Oleg Nesterov <[email protected]> wrote:

> Yes, destroy_nbp() does dev_put(dev). del_nbp() sets dev->br_port = NULL,
> port_carrier_check() goes to "done" in that case. So everething looks safe
> to me (but again, I do not even know what the "bridge" is :), so we should
> only take care about container, nothing more.

Sounds like a plan.

> I'll try to make a patch for illustration on evening.

:-)

David

2007-02-19 22:11:33

by Oleg Nesterov

[permalink] [raw]
Subject: PATCH? net/bridge/br_if.c: fix use after free in port_carrier_check()

On 02/19, Oleg Nesterov wrote:
>
> I think the fix should be so that port_carrier_check() does get/put on
> "struct net_bridge_port" (container), but not on "struct net_device", and
>
> del_nbp(struct net_bridge_port *p)
>
> if (cancel_delayed_work(&p->carrier_check))
> - dev_put(p->dev);
> + kobject_put(&p->kobj);

Perhaps something like the patch below? (on top of this patch).
We should do something, the stable tree has the same bug.

Oleg.

--- WQ/net/bridge/br_if.c~5_bridge_uaf 2007-02-18 23:06:15.000000000 +0300
+++ WQ/net/bridge/br_if.c 2007-02-20 00:59:54.000000000 +0300
@@ -83,14 +83,14 @@ static void port_carrier_check(struct wo
struct net_device *dev;
struct net_bridge *br;

- dev = container_of(work, struct net_bridge_port,
- carrier_check.work)->dev;
+ p = container_of(work, struct net_bridge_port, carrier_check.work);

rtnl_lock();
- p = dev->br_port;
- if (!p)
- goto done;
br = p->br;
+ dev = p->dev;
+
+ if (!dev->br_port)
+ goto done;

if (netif_carrier_ok(dev))
p->path_cost = port_cost(dev);
@@ -107,14 +107,16 @@ static void port_carrier_check(struct wo
spin_unlock_bh(&br->lock);
}
done:
- dev_put(dev);
rtnl_unlock();
+ kobject_put(&p->kobj);
}

static void release_nbp(struct kobject *kobj)
{
struct net_bridge_port *p
= container_of(kobj, struct net_bridge_port, kobj);
+
+ dev_put(p->dev);
kfree(p);
}

@@ -127,12 +129,6 @@ static struct kobj_type brport_ktype = {

static void destroy_nbp(struct net_bridge_port *p)
{
- struct net_device *dev = p->dev;
-
- p->br = NULL;
- p->dev = NULL;
- dev_put(dev);
-
kobject_put(&p->kobj);
}

@@ -162,7 +158,7 @@ static void del_nbp(struct net_bridge_po
dev_set_promiscuity(dev, -1);

if (cancel_delayed_work(&p->carrier_check))
- dev_put(dev);
+ kobject_put(&p->kobj);

spin_lock_bh(&br->lock);
br_stp_disable_port(p);
@@ -446,7 +442,7 @@ int br_add_if(struct net_bridge *br, str
br_stp_recalculate_bridge_id(br);
br_features_recompute(br);
if (schedule_delayed_work(&p->carrier_check, BR_PORT_DEBOUNCE))
- dev_hold(dev);
+ kobject_get(&p->kobj);

spin_unlock_bh(&br->lock);


2007-02-20 10:45:25

by David Howells

[permalink] [raw]
Subject: Re: PATCH? net/bridge/br_if.c: fix use after free in port_carrier_check()


Oleg Nesterov <[email protected]> wrote:

> static void release_nbp(struct kobject *kobj)
> {
> struct net_bridge_port *p
> = container_of(kobj, struct net_bridge_port, kobj);
> +
> + dev_put(p->dev);

Does this need to be done with the mutex held? And does anything actually pay
attention to the refcount on dev? I assume not...

Should you clear p->dev->br_port before calling dev_put()?

Looks reasonable. I like it.

Acked-By: David Howells <[email protected]>

2007-02-20 13:22:15

by Jarek Poplawski

[permalink] [raw]
Subject: Re: [PATCH 1/3] net/bridge/br_if.c: don't use _WORK_NAR

On Mon, Feb 19, 2007 at 06:04:45PM +0300, Oleg Nesterov wrote:
> On 02/19, Jarek Poplawski wrote:
> >
> > On Mon, Feb 19, 2007 at 03:03:53PM +0300, Oleg Nesterov wrote:
> > > On 02/19, Jarek Poplawski wrote:
> > ...
> > > kfree() doesn't check WORK_STRUCT_PENDING, it makes no
> > > difference if it is set or not when work->func() runs.
> >
> > It looks like it's to be checked before kfree.
>
> Here,
> br_add_if:
...

I meant "it's to be checked", if it's used by a program.
The name: work_release seems to tell the work function
could signal, when it doesn't need a structure no more.
But br_if.c doesn't seem to use this infomation now,
so there should be no difference after changing to
"without _NAR". This change will limit some potential
changes in the future, but if it's not used by anybody
than the simpler api is a gain.

Jarek P.

2007-02-20 14:25:20

by Oleg Nesterov

[permalink] [raw]
Subject: Re: [PATCH 1/3] net/bridge/br_if.c: don't use _WORK_NAR

On 02/20, Jarek Poplawski wrote:
>
> On Mon, Feb 19, 2007 at 06:04:45PM +0300, Oleg Nesterov wrote:
> > On 02/19, Jarek Poplawski wrote:
> > >
> > > It looks like it's to be checked before kfree.
> >
> > Here,
> > br_add_if:
> ...
>
> I meant "it's to be checked",

Jarek, I was too quick and misread your message, sorry!

> if it's used by a program.
> The name: work_release seems to tell the work function
> could signal, when it doesn't need a structure no more.
> But br_if.c doesn't seem to use this infomation now,
> so there should be no difference after changing to
> "without _NAR". This change will limit some potential
> changes in the future, but if it's not used by anybody
> than the simpler api is a gain.

Agreed.

Oleg.

2007-02-20 14:34:16

by Oleg Nesterov

[permalink] [raw]
Subject: Re: PATCH? net/bridge/br_if.c: fix use after free in port_carrier_check()

On 02/20, David Howells wrote:
>
> Oleg Nesterov <[email protected]> wrote:
>
> > static void release_nbp(struct kobject *kobj)
> > {
> > struct net_bridge_port *p
> > = container_of(kobj, struct net_bridge_port, kobj);
> > +
> > + dev_put(p->dev);
>
> Does this need to be done with the mutex held?

I think no. At least the current code does dev_put() without mutex held.

> And does anything actually pay
> attention to the refcount on dev? I assume not...

I guess net/core/dev.c:netdev_wait_allrefs(), but not sure.

> Should you clear p->dev->br_port before calling dev_put()?

Looks like it is protected by RCU... Anyway the current code does the same.

> Looks reasonable. I like it.
>
> Acked-By: David Howells <[email protected]>

Thanks! I'll re-send with a proper changelog later today.

Oleg.