2021-04-21 13:08:42

by Greg KH

[permalink] [raw]
Subject: [PATCH 037/190] Revert "RDMA/core: Fix several reference count leaks."

This reverts commit 0b8e125e213204508e1b3c4bdfe69713280b7abd.

Commits from @umn.edu addresses have been found to be submitted in "bad
faith" to try to test the kernel community's ability to review "known
malicious" changes. The result of these submissions can be found in a
paper published at the 42nd IEEE Symposium on Security and Privacy
entitled, "Open Source Insecurity: Stealthily Introducing
Vulnerabilities via Hypocrite Commits" written by Qiushi Wu (University
of Minnesota) and Kangjie Lu (University of Minnesota).

Because of this, all submissions from this group must be reverted from
the kernel tree and will need to be re-reviewed again to determine if
they actually are a valid fix. Until that work is complete, remove this
change to ensure that no problems are being introduced into the
codebase.

Cc: https
Cc: Qiushi Wu <[email protected]>
Cc: Jason Gunthorpe <[email protected]>
Cc: Jason Gunthorpe <[email protected]>
Signed-off-by: Greg Kroah-Hartman <[email protected]>
---
drivers/infiniband/core/sysfs.c | 10 +++++-----
1 file changed, 5 insertions(+), 5 deletions(-)

diff --git a/drivers/infiniband/core/sysfs.c b/drivers/infiniband/core/sysfs.c
index b8abb30f80df..1ff580362a7c 100644
--- a/drivers/infiniband/core/sysfs.c
+++ b/drivers/infiniband/core/sysfs.c
@@ -1076,7 +1076,8 @@ static int add_port(struct ib_core_device *coredev, int port_num)
coredev->ports_kobj,
"%d", port_num);
if (ret) {
- goto err_put;
+ kfree(p);
+ return ret;
}

p->gid_attr_group = kzalloc(sizeof(*p->gid_attr_group), GFP_KERNEL);
@@ -1089,7 +1090,8 @@ static int add_port(struct ib_core_device *coredev, int port_num)
ret = kobject_init_and_add(&p->gid_attr_group->kobj, &gid_attr_type,
&p->kobj, "gid_attrs");
if (ret) {
- goto err_put_gid_attrs;
+ kfree(p->gid_attr_group);
+ goto err_put;
}

if (device->ops.process_mad && is_full_dev) {
@@ -1452,10 +1454,8 @@ int ib_port_register_module_stat(struct ib_device *device, u8 port_num,

ret = kobject_init_and_add(kobj, ktype, &port->kobj, "%s",
name);
- if (ret) {
- kobject_put(kobj);
+ if (ret)
return ret;
- }
}

return 0;
--
2.31.1


2021-04-21 19:05:28

by Jason Gunthorpe

[permalink] [raw]
Subject: Re: [PATCH 037/190] Revert "RDMA/core: Fix several reference count leaks."

On Wed, Apr 21, 2021 at 02:58:32PM +0200, Greg Kroah-Hartman wrote:
> This reverts commit 0b8e125e213204508e1b3c4bdfe69713280b7abd.
>
> Commits from @umn.edu addresses have been found to be submitted in "bad
> faith" to try to test the kernel community's ability to review "known
> malicious" changes. The result of these submissions can be found in a
> paper published at the 42nd IEEE Symposium on Security and Privacy
> entitled, "Open Source Insecurity: Stealthily Introducing
> Vulnerabilities via Hypocrite Commits" written by Qiushi Wu (University
> of Minnesota) and Kangjie Lu (University of Minnesota).
>
> Because of this, all submissions from this group must be reverted from
> the kernel tree and will need to be re-reviewed again to determine if
> they actually are a valid fix. Until that work is complete, remove this
> change to ensure that no problems are being introduced into the
> codebase.
>
> Cc: https
> Cc: Qiushi Wu <[email protected]>
> Cc: Jason Gunthorpe <[email protected]>
> Cc: Jason Gunthorpe <[email protected]>
> Signed-off-by: Greg Kroah-Hartman <[email protected]>
> ---
> drivers/infiniband/core/sysfs.c | 10 +++++-----
> 1 file changed, 5 insertions(+), 5 deletions(-)

The original commit is certainly moving things closer to being correct
but I'm concerned the entire area is still not bug free, it will take
me some time to check the whole code. Reverting seems reasonable for
now

Jason

2021-04-27 14:29:51

by Greg KH

[permalink] [raw]
Subject: Re: [PATCH 037/190] Revert "RDMA/core: Fix several reference count leaks."

On Wed, Apr 21, 2021 at 11:14:44AM -0300, Jason Gunthorpe wrote:
> On Wed, Apr 21, 2021 at 02:58:32PM +0200, Greg Kroah-Hartman wrote:
> > This reverts commit 0b8e125e213204508e1b3c4bdfe69713280b7abd.
> >
> > Commits from @umn.edu addresses have been found to be submitted in "bad
> > faith" to try to test the kernel community's ability to review "known
> > malicious" changes. The result of these submissions can be found in a
> > paper published at the 42nd IEEE Symposium on Security and Privacy
> > entitled, "Open Source Insecurity: Stealthily Introducing
> > Vulnerabilities via Hypocrite Commits" written by Qiushi Wu (University
> > of Minnesota) and Kangjie Lu (University of Minnesota).
> >
> > Because of this, all submissions from this group must be reverted from
> > the kernel tree and will need to be re-reviewed again to determine if
> > they actually are a valid fix. Until that work is complete, remove this
> > change to ensure that no problems are being introduced into the
> > codebase.
> >
> > Cc: https
> > Cc: Qiushi Wu <[email protected]>
> > Cc: Jason Gunthorpe <[email protected]>
> > Cc: Jason Gunthorpe <[email protected]>
> > Signed-off-by: Greg Kroah-Hartman <[email protected]>
> > ---
> > drivers/infiniband/core/sysfs.c | 10 +++++-----
> > 1 file changed, 5 insertions(+), 5 deletions(-)
>
> The original commit is certainly moving things closer to being correct
> but I'm concerned the entire area is still not bug free, it will take
> me some time to check the whole code. Reverting seems reasonable for
> now

I just re-reviewed it, and it looks like the original commit is ok, so I
will drop this.

But ugh, this is horrid code, you should NOT be dealing with "raw"
kobjects here at all, just to try to make a subdirectory tree. That
causes a total mess as this function shows and probably should be
cleaned up sometime as obviously userspace tools are not properly seeing
the objects you are creating here (hint, you can not put a kobject below
a 'struct device' in the device tree and have udev and others see it
properly...)

But that's a topic for a later time, dropping this now.

greg k-h

2021-04-27 16:20:55

by Jason Gunthorpe

[permalink] [raw]
Subject: Re: [PATCH 037/190] Revert "RDMA/core: Fix several reference count leaks."

On Tue, Apr 27, 2021 at 04:27:57PM +0200, Greg Kroah-Hartman wrote:
> On Wed, Apr 21, 2021 at 11:14:44AM -0300, Jason Gunthorpe wrote:
> > On Wed, Apr 21, 2021 at 02:58:32PM +0200, Greg Kroah-Hartman wrote:
> > > This reverts commit 0b8e125e213204508e1b3c4bdfe69713280b7abd.
> > >
> > > Commits from @umn.edu addresses have been found to be submitted in "bad
> > > faith" to try to test the kernel community's ability to review "known
> > > malicious" changes. The result of these submissions can be found in a
> > > paper published at the 42nd IEEE Symposium on Security and Privacy
> > > entitled, "Open Source Insecurity: Stealthily Introducing
> > > Vulnerabilities via Hypocrite Commits" written by Qiushi Wu (University
> > > of Minnesota) and Kangjie Lu (University of Minnesota).
> > >
> > > Because of this, all submissions from this group must be reverted from
> > > the kernel tree and will need to be re-reviewed again to determine if
> > > they actually are a valid fix. Until that work is complete, remove this
> > > change to ensure that no problems are being introduced into the
> > > codebase.
> > >
> > > Cc: https
> > > Cc: Qiushi Wu <[email protected]>
> > > Cc: Jason Gunthorpe <[email protected]>
> > > Cc: Jason Gunthorpe <[email protected]>
> > > Signed-off-by: Greg Kroah-Hartman <[email protected]>
> > > drivers/infiniband/core/sysfs.c | 10 +++++-----
> > > 1 file changed, 5 insertions(+), 5 deletions(-)
> >
> > The original commit is certainly moving things closer to being correct
> > but I'm concerned the entire area is still not bug free, it will take
> > me some time to check the whole code. Reverting seems reasonable for
> > now
>
> I just re-reviewed it, and it looks like the original commit is ok, so I
> will drop this.

This was what I thought too when I originally applied it..

> But ugh, this is horrid code, you should NOT be dealing with "raw"
> kobjects here at all, just to try to make a subdirectory tree. That
> causes a total mess as this function shows and probably should be
> cleaned up sometime as obviously userspace tools are not properly seeing
> the objects you are creating here (hint, you can not put a kobject below
> a 'struct device' in the device tree and have udev and others see it
> properly...)

We've talked about this specifically before:

http://lore.kernel.org/r/[email protected]

I still don't understand what you mean by "udev sees it properly", as
above, all the tests I thought of look OK.

It is uABI at this point that real userspace software relies on, so we
can't change the sysfs paths. If there is a more right way to code it
then let me know I can have a go - Kees also recently found some bad
stuff here I hope to get to soon.

Jason

2021-04-28 14:20:40

by Greg KH

[permalink] [raw]
Subject: Re: [PATCH 037/190] Revert "RDMA/core: Fix several reference count leaks."

On Tue, Apr 27, 2021 at 01:12:35PM -0300, Jason Gunthorpe wrote:
> On Tue, Apr 27, 2021 at 04:27:57PM +0200, Greg Kroah-Hartman wrote:
> > On Wed, Apr 21, 2021 at 11:14:44AM -0300, Jason Gunthorpe wrote:
> > > On Wed, Apr 21, 2021 at 02:58:32PM +0200, Greg Kroah-Hartman wrote:
> > > > This reverts commit 0b8e125e213204508e1b3c4bdfe69713280b7abd.
> > > >
> > > > Commits from @umn.edu addresses have been found to be submitted in "bad
> > > > faith" to try to test the kernel community's ability to review "known
> > > > malicious" changes. The result of these submissions can be found in a
> > > > paper published at the 42nd IEEE Symposium on Security and Privacy
> > > > entitled, "Open Source Insecurity: Stealthily Introducing
> > > > Vulnerabilities via Hypocrite Commits" written by Qiushi Wu (University
> > > > of Minnesota) and Kangjie Lu (University of Minnesota).
> > > >
> > > > Because of this, all submissions from this group must be reverted from
> > > > the kernel tree and will need to be re-reviewed again to determine if
> > > > they actually are a valid fix. Until that work is complete, remove this
> > > > change to ensure that no problems are being introduced into the
> > > > codebase.
> > > >
> > > > Cc: https
> > > > Cc: Qiushi Wu <[email protected]>
> > > > Cc: Jason Gunthorpe <[email protected]>
> > > > Cc: Jason Gunthorpe <[email protected]>
> > > > Signed-off-by: Greg Kroah-Hartman <[email protected]>
> > > > drivers/infiniband/core/sysfs.c | 10 +++++-----
> > > > 1 file changed, 5 insertions(+), 5 deletions(-)
> > >
> > > The original commit is certainly moving things closer to being correct
> > > but I'm concerned the entire area is still not bug free, it will take
> > > me some time to check the whole code. Reverting seems reasonable for
> > > now
> >
> > I just re-reviewed it, and it looks like the original commit is ok, so I
> > will drop this.
>
> This was what I thought too when I originally applied it..
>
> > But ugh, this is horrid code, you should NOT be dealing with "raw"
> > kobjects here at all, just to try to make a subdirectory tree. That
> > causes a total mess as this function shows and probably should be
> > cleaned up sometime as obviously userspace tools are not properly seeing
> > the objects you are creating here (hint, you can not put a kobject below
> > a 'struct device' in the device tree and have udev and others see it
> > properly...)
>
> We've talked about this specifically before:
>
> http://lore.kernel.org/r/[email protected]
>
> I still don't understand what you mean by "udev sees it properly", as
> above, all the tests I thought of look OK.

Can you query the udev database to see the attribute values?

> It is uABI at this point that real userspace software relies on, so we
> can't change the sysfs paths. If there is a more right way to code it
> then let me know I can have a go - Kees also recently found some bad
> stuff here I hope to get to soon.

It's on my long "list of things to get to eventually"...

As you say, it's uABI for now, so odds are nothing can be changed. It's
just no fun for when other subsystems want to do this same thing, they
point at this code and say "see, they did it!" :)

thanks,

greg k-h

2021-04-28 14:52:40

by Jason Gunthorpe

[permalink] [raw]
Subject: Re: [PATCH 037/190] Revert "RDMA/core: Fix several reference count leaks."

On Wed, Apr 28, 2021 at 02:23:40PM +0200, Greg Kroah-Hartman wrote:

> > We've talked about this specifically before:
> >
> > http://lore.kernel.org/r/[email protected]
> >
> > I still don't understand what you mean by "udev sees it properly", as
> > above, all the tests I thought of look OK.
>
> Can you query the udev database to see the attribute values?

It appears so unless I misunderstand your ask:

$ udevadm info -a /sys/class/infiniband/ibp0s9
ATTR{ports/1/cm_rx_duplicates/dreq}=="0"

> As you say, it's uABI for now, so odds are nothing can be changed. It's
> just no fun for when other subsystems want to do this same thing, they
> point at this code and say "see, they did it!" :)

Are you sure we shouldn't just formally support this?

What is the exact technical blocker?

Jason

2021-04-29 13:41:03

by Greg KH

[permalink] [raw]
Subject: Re: [PATCH 037/190] Revert "RDMA/core: Fix several reference count leaks."

On Wed, Apr 28, 2021 at 10:00:44AM -0300, Jason Gunthorpe wrote:
> On Wed, Apr 28, 2021 at 02:23:40PM +0200, Greg Kroah-Hartman wrote:
>
> > > We've talked about this specifically before:
> > >
> > > http://lore.kernel.org/r/[email protected]
> > >
> > > I still don't understand what you mean by "udev sees it properly", as
> > > above, all the tests I thought of look OK.
> >
> > Can you query the udev database to see the attribute values?
>
> It appears so unless I misunderstand your ask:
>
> $ udevadm info -a /sys/class/infiniband/ibp0s9
> ATTR{ports/1/cm_rx_duplicates/dreq}=="0"

That works? Nice, I didn't think it did.

But what about the uevent that fired for "1", isn't there attibutes
assigned to it that udev ignores?

> > As you say, it's uABI for now, so odds are nothing can be changed. It's
> > just no fun for when other subsystems want to do this same thing, they
> > point at this code and say "see, they did it!" :)
>
> Are you sure we shouldn't just formally support this?
>
> What is the exact technical blocker?

Placing a raw kobject below a struct device breaks the "device tree"
model. You now have devices with an arbritrary number of levels deep
set of attributes, making it impossible to determine all attributes for
a device in a simple way.

thanks,

greg k-h

2021-05-03 18:33:34

by Jason Gunthorpe

[permalink] [raw]
Subject: Re: [PATCH 037/190] Revert "RDMA/core: Fix several reference count leaks."

On Thu, Apr 29, 2021 at 03:38:41PM +0200, Greg Kroah-Hartman wrote:
> On Wed, Apr 28, 2021 at 10:00:44AM -0300, Jason Gunthorpe wrote:
> > On Wed, Apr 28, 2021 at 02:23:40PM +0200, Greg Kroah-Hartman wrote:
> >
> > > > We've talked about this specifically before:
> > > >
> > > > http://lore.kernel.org/r/[email protected]
> > > >
> > > > I still don't understand what you mean by "udev sees it properly", as
> > > > above, all the tests I thought of look OK.
> > >
> > > Can you query the udev database to see the attribute values?
> >
> > It appears so unless I misunderstand your ask:
> >
> > $ udevadm info -a /sys/class/infiniband/ibp0s9
> > ATTR{ports/1/cm_rx_duplicates/dreq}=="0"
>
> That works? Nice, I didn't think it did.
>
> But what about the uevent that fired for "1", isn't there attibutes
> assigned to it that udev ignores?

I'm not completely familiar with uevents, but:

$ find /sys/class/infiniband/ibp0s9/ -name "uevent"
/sys/class/infiniband/ibp0s9/uevent

$ udevadm monitor & modprobe mlx5_ib
KERNEL[169.337295] add /bus/auxiliary/drivers/mlx5_ib.multiport (drivers)
UDEV [169.354621] add /bus/auxiliary/drivers/mlx5_ib.multiport (drivers)
KERNEL[169.393088] add /devices/pci0000:00/0000:00:09.0/infiniband_verbs/uverbs0 (infiniband_verbs)
KERNEL[169.393516] add /devices/pci0000:00/0000:00:09.0/infiniband_mad/umad0 (infiniband_mad)
KERNEL[169.394040] add /devices/pci0000:00/0000:00:09.0/infiniband_mad/issm0 (infiniband_mad)
UDEV [169.395189] add /devices/pci0000:00/0000:00:09.0/infiniband_verbs/uverbs0 (infiniband_verbs)
UDEV [169.397812] add /devices/pci0000:00/0000:00:09.0/infiniband_mad/issm0 (infiniband_mad)
KERNEL[169.407727] add /devices/pci0000:00/0000:00:09.0/net/ib0 (net)
KERNEL[169.407851] add /devices/pci0000:00/0000:00:09.0/net/ib0/queues/rx-0 (queues)
KERNEL[169.408113] add /devices/pci0000:00/0000:00:09.0/net/ib0/queues/tx-0 (queues)
KERNEL[169.409059] add /devices/pci0000:00/0000:00:09.0/infiniband/mlx5_0 (infiniband)
KERNEL[169.411483] bind /devices/pci0000:00/0000:00:09.0/mlx5_core.rdma.0 (auxiliary)
KERNEL[169.411836] add /bus/auxiliary/drivers/mlx5_ib.rdma (drivers)
KERNEL[169.411973] add /module/mlx5_ib (module)
UDEV [169.420570] bind /devices/pci0000:00/0000:00:09.0/mlx5_core.rdma.0 (auxiliary)
UDEV [169.421365] add /bus/auxiliary/drivers/mlx5_ib.rdma (drivers)
UDEV [169.447853] add /module/mlx5_ib (module)
KERNEL[169.482293] move /devices/pci0000:00/0000:00:09.0/infiniband/ibp0s9 (infiniband)
UDEV [169.486395] add /devices/pci0000:00/0000:00:09.0/infiniband/mlx5_0 (infiniband)
UDEV [169.495193] move /devices/pci0000:00/0000:00:09.0/infiniband/ibp0s9 (infiniband)
UDEV [169.698592] add /devices/pci0000:00/0000:00:09.0/net/ib0 (net)
UDEV [169.700436] add /devices/pci0000:00/0000:00:09.0/net/ib0/queues/rx-0 (queues)
UDEV [169.700712] add /devices/pci0000:00/0000:00:09.0/net/ib0/queues/tx-0 (queues)
UDEV [170.042132] add /devices/pci0000:00/0000:00:09.0/infiniband_mad/umad0 (infiniband_mad)

I don't see any uevents related to the nested attributes. Same on
removal.

As far I can tell udev is working perfectly fine with 4 levels of
attributes..

> Placing a raw kobject below a struct device breaks the "device tree"
> model. You now have devices with an arbritrary number of levels deep
> set of attributes, making it impossible to determine all attributes for
> a device in a simple way.

Since udev is able to determine the attributes, does this mean at some
point it abandoned the simple way and is now doing something complex?

Jason

2021-05-05 22:56:52

by Jason Gunthorpe

[permalink] [raw]
Subject: Re: [PATCH 037/190] Revert "RDMA/core: Fix several reference count leaks."

On Mon, May 03, 2021 at 03:30:51PM -0300, Jason Gunthorpe wrote:
> On Thu, Apr 29, 2021 at 03:38:41PM +0200, Greg Kroah-Hartman wrote:
> > On Wed, Apr 28, 2021 at 10:00:44AM -0300, Jason Gunthorpe wrote:
> > > On Wed, Apr 28, 2021 at 02:23:40PM +0200, Greg Kroah-Hartman wrote:
> > >
> > > > > We've talked about this specifically before:
> > > > >
> > > > > http://lore.kernel.org/r/[email protected]
> > > > >
> > > > > I still don't understand what you mean by "udev sees it properly", as
> > > > > above, all the tests I thought of look OK.
> > > >
> > > > Can you query the udev database to see the attribute values?
> > >
> > > It appears so unless I misunderstand your ask:
> > >
> > > $ udevadm info -a /sys/class/infiniband/ibp0s9
> > > ATTR{ports/1/cm_rx_duplicates/dreq}=="0"
> >
> > That works? Nice, I didn't think it did.
> >
> > But what about the uevent that fired for "1", isn't there attibutes
> > assigned to it that udev ignores?
>
> I'm not completely familiar with uevents, but:
>
> $ find /sys/class/infiniband/ibp0s9/ -name "uevent"
> /sys/class/infiniband/ibp0s9/uevent
>
> $ udevadm monitor & modprobe mlx5_ib
> KERNEL[169.337295] add /bus/auxiliary/drivers/mlx5_ib.multiport (drivers)
> UDEV [169.354621] add /bus/auxiliary/drivers/mlx5_ib.multiport (drivers)
> KERNEL[169.393088] add /devices/pci0000:00/0000:00:09.0/infiniband_verbs/uverbs0 (infiniband_verbs)
> KERNEL[169.393516] add /devices/pci0000:00/0000:00:09.0/infiniband_mad/umad0 (infiniband_mad)
> KERNEL[169.394040] add /devices/pci0000:00/0000:00:09.0/infiniband_mad/issm0 (infiniband_mad)
> UDEV [169.395189] add /devices/pci0000:00/0000:00:09.0/infiniband_verbs/uverbs0 (infiniband_verbs)
> UDEV [169.397812] add /devices/pci0000:00/0000:00:09.0/infiniband_mad/issm0 (infiniband_mad)
> KERNEL[169.407727] add /devices/pci0000:00/0000:00:09.0/net/ib0 (net)
> KERNEL[169.407851] add /devices/pci0000:00/0000:00:09.0/net/ib0/queues/rx-0 (queues)
> KERNEL[169.408113] add /devices/pci0000:00/0000:00:09.0/net/ib0/queues/tx-0 (queues)
> KERNEL[169.409059] add /devices/pci0000:00/0000:00:09.0/infiniband/mlx5_0 (infiniband)
> KERNEL[169.411483] bind /devices/pci0000:00/0000:00:09.0/mlx5_core.rdma.0 (auxiliary)
> KERNEL[169.411836] add /bus/auxiliary/drivers/mlx5_ib.rdma (drivers)
> KERNEL[169.411973] add /module/mlx5_ib (module)
> UDEV [169.420570] bind /devices/pci0000:00/0000:00:09.0/mlx5_core.rdma.0 (auxiliary)
> UDEV [169.421365] add /bus/auxiliary/drivers/mlx5_ib.rdma (drivers)
> UDEV [169.447853] add /module/mlx5_ib (module)
> KERNEL[169.482293] move /devices/pci0000:00/0000:00:09.0/infiniband/ibp0s9 (infiniband)
> UDEV [169.486395] add /devices/pci0000:00/0000:00:09.0/infiniband/mlx5_0 (infiniband)
> UDEV [169.495193] move /devices/pci0000:00/0000:00:09.0/infiniband/ibp0s9 (infiniband)
> UDEV [169.698592] add /devices/pci0000:00/0000:00:09.0/net/ib0 (net)
> UDEV [169.700436] add /devices/pci0000:00/0000:00:09.0/net/ib0/queues/rx-0 (queues)
> UDEV [169.700712] add /devices/pci0000:00/0000:00:09.0/net/ib0/queues/tx-0 (queues)
> UDEV [170.042132] add /devices/pci0000:00/0000:00:09.0/infiniband_mad/umad0 (infiniband_mad)
>
> I don't see any uevents related to the nested attributes. Same on
> removal.

With some debugging, the uevent situation is like this..

When '1' is created as a kobj the code does call

kobject_uevent(&port->kobj, KOBJ_ADD);

However DEBUG_KOBJECT reveals:

kobject: '1' (00000000d2367083): kobject_uevent_env: filter function caused the event to drop!

Which happens because
top_kobj == mlx5 (ie the struct device)
top_kobj->kset->uevent_ops == device_uevent_ops
get_ktype(kobj "1") == &port_type

Thus calling
dev_uevent_filter(mlx5 kset, kobj "1") == 0

As get_ktype(kobj "1") != &device_ktype

Which I read to mean these nested attributes under a struct device
won't generate a uevent.

The uevent for the struct device is supressed until all the child
kobjects are created and this explains how udev sees the child kobj's
and doesn't see extra uevents to confuse it.

Jason