2018-02-11 15:26:02

by Mathieu Malaterre

[permalink] [raw]
Subject: [kmemleak] unreferenced object 0xcd9c1a80 (size 192):

Alexei,

Could you please comment on why I am seeing those memleaks being
reported on my ppc32 system ? Should they be marked as false positive
?

System is Mac Mini G4, git/master (4.15.0+), ppc.

Thanks for your time

$ dmesg
...
[ 1281.504173] kmemleak: 36 new suspected memory leaks (see
/sys/kernel/debug/kmemleak)

Where:

# cat /sys/kernel/debug/kmemleak
unreferenced object 0xdee25000 (size 192):
comm "systemd", pid 1, jiffies 4294894348 (age 1438.580s)
hex dump (first 32 bytes):
c0 56 2f 88 00 00 00 00 00 00 00 0b 00 00 00 0c .V/.............
00 00 00 08 00 00 00 01 00 00 00 01 00 00 00 01 ................
backtrace:
[<6c69baf5>] trie_alloc+0xb0/0x150
[<fa093284>] SyS_bpf+0x288/0x1458
[<82182f53>] ret_from_syscall+0x0/0x38
unreferenced object 0xdee25900 (size 192):
comm "systemd", pid 1, jiffies 4294894540 (age 1437.812s)
hex dump (first 32 bytes):
c0 56 2f 88 00 00 00 00 00 00 00 0b 00 00 00 08 .V/.............
00 00 00 08 00 00 00 01 00 00 00 01 00 00 00 01 ................
backtrace:
[<6c69baf5>] trie_alloc+0xb0/0x150
[<fa093284>] SyS_bpf+0x288/0x1458
[<82182f53>] ret_from_syscall+0x0/0x38
...


2018-02-11 16:55:51

by Alexei Starovoitov

[permalink] [raw]
Subject: Re: [kmemleak] unreferenced object 0xcd9c1a80 (size 192):

On Sun, Feb 11, 2018 at 7:24 AM, Mathieu Malaterre <[email protected]> wrote:
> Alexei,
>
> Could you please comment on why I am seeing those memleaks being
> reported on my ppc32 system ? Should they be marked as false positive
> ?
>
> System is Mac Mini G4, git/master (4.15.0+), ppc.
>
> Thanks for your time
>
> $ dmesg
> ...
> [ 1281.504173] kmemleak: 36 new suspected memory leaks (see
> /sys/kernel/debug/kmemleak)
>
> Where:
>
> # cat /sys/kernel/debug/kmemleak
> unreferenced object 0xdee25000 (size 192):
> comm "systemd", pid 1, jiffies 4294894348 (age 1438.580s)
> hex dump (first 32 bytes):
> c0 56 2f 88 00 00 00 00 00 00 00 0b 00 00 00 0c .V/.............
> 00 00 00 08 00 00 00 01 00 00 00 01 00 00 00 01 ................
> backtrace:
> [<6c69baf5>] trie_alloc+0xb0/0x150
> [<fa093284>] SyS_bpf+0x288/0x1458
> [<82182f53>] ret_from_syscall+0x0/0x38
> unreferenced object 0xdee25900 (size 192):
> comm "systemd", pid 1, jiffies 4294894540 (age 1437.812s)
> hex dump (first 32 bytes):
> c0 56 2f 88 00 00 00 00 00 00 00 0b 00 00 00 08 .V/.............
> 00 00 00 08 00 00 00 01 00 00 00 01 00 00 00 01 ................
> backtrace:
> [<6c69baf5>] trie_alloc+0xb0/0x150
> [<fa093284>] SyS_bpf+0x288/0x1458
> [<82182f53>] ret_from_syscall+0x0/0x38

hmm. looks real. Is there a reproducer?
Yonghong, lpm map not cleaning after itself?

2018-02-11 19:19:48

by Mathieu Malaterre

[permalink] [raw]
Subject: Re: [kmemleak] unreferenced object 0xcd9c1a80 (size 192):

Hi,

On Sun, Feb 11, 2018 at 5:54 PM, Alexei Starovoitov
<[email protected]> wrote:
> On Sun, Feb 11, 2018 at 7:24 AM, Mathieu Malaterre <[email protected]> wrote:
>> Alexei,
>>
>> Could you please comment on why I am seeing those memleaks being
>> reported on my ppc32 system ? Should they be marked as false positive
>> ?
>>
>> System is Mac Mini G4, git/master (4.15.0+), ppc.
>>
>> Thanks for your time
>>
>> $ dmesg
>> ...
>> [ 1281.504173] kmemleak: 36 new suspected memory leaks (see
>> /sys/kernel/debug/kmemleak)
>>
>> Where:
>>
>> # cat /sys/kernel/debug/kmemleak
>> unreferenced object 0xdee25000 (size 192):
>> comm "systemd", pid 1, jiffies 4294894348 (age 1438.580s)
>> hex dump (first 32 bytes):
>> c0 56 2f 88 00 00 00 00 00 00 00 0b 00 00 00 0c .V/.............
>> 00 00 00 08 00 00 00 01 00 00 00 01 00 00 00 01 ................
>> backtrace:
>> [<6c69baf5>] trie_alloc+0xb0/0x150
>> [<fa093284>] SyS_bpf+0x288/0x1458
>> [<82182f53>] ret_from_syscall+0x0/0x38
>> unreferenced object 0xdee25900 (size 192):
>> comm "systemd", pid 1, jiffies 4294894540 (age 1437.812s)
>> hex dump (first 32 bytes):
>> c0 56 2f 88 00 00 00 00 00 00 00 0b 00 00 00 08 .V/.............
>> 00 00 00 08 00 00 00 01 00 00 00 01 00 00 00 01 ................
>> backtrace:
>> [<6c69baf5>] trie_alloc+0xb0/0x150
>> [<fa093284>] SyS_bpf+0x288/0x1458
>> [<82182f53>] ret_from_syscall+0x0/0x38
>
> hmm. looks real. Is there a reproducer?
> Yonghong, lpm map not cleaning after itself?

Not really. I simply boot up my machine and wait for the first kmemleak scan.

2018-02-12 07:13:47

by Yonghong Song

[permalink] [raw]
Subject: Re: [kmemleak] unreferenced object 0xcd9c1a80 (size 192):



On 2/11/18 11:18 AM, Mathieu Malaterre wrote:
> Hi,
>
> On Sun, Feb 11, 2018 at 5:54 PM, Alexei Starovoitov
> <[email protected]> wrote:
>> On Sun, Feb 11, 2018 at 7:24 AM, Mathieu Malaterre <[email protected]> wrote:
>>> Alexei,
>>>
>>> Could you please comment on why I am seeing those memleaks being
>>> reported on my ppc32 system ? Should they be marked as false positive
>>> ?
>>>
>>> System is Mac Mini G4, git/master (4.15.0+), ppc.
>>>
>>> Thanks for your time
>>>
>>> $ dmesg
>>> ...
>>> [ 1281.504173] kmemleak: 36 new suspected memory leaks (see
>>> /sys/kernel/debug/kmemleak)
>>>
>>> Where:
>>>
>>> # cat /sys/kernel/debug/kmemleak
>>> unreferenced object 0xdee25000 (size 192):
>>> comm "systemd", pid 1, jiffies 4294894348 (age 1438.580s)
>>> hex dump (first 32 bytes):
>>> c0 56 2f 88 00 00 00 00 00 00 00 0b 00 00 00 0c .V/.............
>>> 00 00 00 08 00 00 00 01 00 00 00 01 00 00 00 01 ................
>>> backtrace:
>>> [<6c69baf5>] trie_alloc+0xb0/0x150
>>> [<fa093284>] SyS_bpf+0x288/0x1458
>>> [<82182f53>] ret_from_syscall+0x0/0x38
>>> unreferenced object 0xdee25900 (size 192):
>>> comm "systemd", pid 1, jiffies 4294894540 (age 1437.812s)
>>> hex dump (first 32 bytes):
>>> c0 56 2f 88 00 00 00 00 00 00 00 0b 00 00 00 08 .V/.............
>>> 00 00 00 08 00 00 00 01 00 00 00 01 00 00 00 01 ................
>>> backtrace:
>>> [<6c69baf5>] trie_alloc+0xb0/0x150
>>> [<fa093284>] SyS_bpf+0x288/0x1458
>>> [<82182f53>] ret_from_syscall+0x0/0x38
>>
>> hmm. looks real. Is there a reproducer?
>> Yonghong, lpm map not cleaning after itself?
>
> Not really. I simply boot up my machine and wait for the first kmemleak scan.

I am not able to reproduce the issue. Tried with latest net-next on FC26
with kmemleak on. I only got this one after bootup,
'cat /sys/kernel/debug/kmemleak' or
'echo scan > /sys/kernel/debug/kmemleak
cat /sys/kernel/debug/kmemleak':

unreferenced object 0xffff99701a7386e0 (size 32):
comm "mount", pid 1856, jiffies 4294669263 (age 98.440s)
hex dump (first 32 bytes):
00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 ................
00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 ................
backtrace:
[<000000004668ec00>] security_sb_parse_opts_str+0x36/0x50
[<00000000a9807d2b>] parse_security_options+0x3d/0x60
[<00000000cc1e1d58>] btrfs_mount_root+0x139/0x720
[<00000000bdc4f1a3>] mount_fs+0x30/0x150
[<00000000f189f1bd>] vfs_kern_mount.part.26+0x54/0x100
[<0000000093ae5db7>] btrfs_mount+0x184/0x914
[<00000000bdc4f1a3>] mount_fs+0x30/0x150
[<00000000f189f1bd>] vfs_kern_mount.part.26+0x54/0x100
[<000000003b67b9fc>] do_mount+0x5b9/0xc70
[<00000000de4073a0>] SyS_mount+0x80/0xd0
[<00000000fc5a968a>] do_syscall_64+0x5d/0x110
[<000000003d61f5fc>] entry_SYSCALL_64_after_hwframe+0x21/0x86
[<00000000458a6ffa>] 0xffffffffffffffff

Not sure whether the above is a true issue or not.

However, by inspecting the code, I do find the trie_free in lpm_trie.c
may have missed freeing the trie memory.

The change likes below should work:
-bash-4.2$ git diff

diff --git a/kernel/bpf/lpm_trie.c b/kernel/bpf/lpm_trie.c
index 7b469d1..cecb259 100644
--- a/kernel/bpf/lpm_trie.c
+++ b/kernel/bpf/lpm_trie.c
@@ -589,6 +589,7 @@ static void trie_free(struct bpf_map *map)

unlock:
raw_spin_unlock(&trie->lock);
+ kfree(trie);
}

static int trie_get_next_key(struct bpf_map *map, void *_key, void
*_next_key)
-bash-4.2$

Will propose a formal patch for this soon.


>

2018-02-12 08:31:45

by Daniel Borkmann

[permalink] [raw]
Subject: Re: [kmemleak] unreferenced object 0xcd9c1a80 (size 192):

On 02/12/2018 06:47 AM, Yonghong Song wrote:
> On 2/11/18 11:18 AM, Mathieu Malaterre wrote:
>> On Sun, Feb 11, 2018 at 5:54 PM, Alexei Starovoitov
>> <[email protected]> wrote:
>>> On Sun, Feb 11, 2018 at 7:24 AM, Mathieu Malaterre <[email protected]> wrote:
>>>> Alexei,
>>>>
>>>> Could you please comment on why I am seeing those memleaks being
>>>> reported on my ppc32 system ? Should they be marked as false positive
>>>> ?
>>>>
>>>> System is Mac Mini G4, git/master (4.15.0+), ppc.
>>>>
>>>> Thanks for your time
>>>>
>>>> $ dmesg
>>>> ...
>>>> [ 1281.504173] kmemleak: 36 new suspected memory leaks (see
>>>> /sys/kernel/debug/kmemleak)
>>>>
>>>> Where:
>>>>
>>>> # cat /sys/kernel/debug/kmemleak
>>>> unreferenced object 0xdee25000 (size 192):
>>>>    comm "systemd", pid 1, jiffies 4294894348 (age 1438.580s)
>>>>    hex dump (first 32 bytes):
>>>>      c0 56 2f 88 00 00 00 00 00 00 00 0b 00 00 00 0c  .V/.............
>>>>      00 00 00 08 00 00 00 01 00 00 00 01 00 00 00 01  ................
>>>>    backtrace:
>>>>      [<6c69baf5>] trie_alloc+0xb0/0x150
>>>>      [<fa093284>] SyS_bpf+0x288/0x1458
>>>>      [<82182f53>] ret_from_syscall+0x0/0x38
>>>> unreferenced object 0xdee25900 (size 192):
>>>>    comm "systemd", pid 1, jiffies 4294894540 (age 1437.812s)
>>>>    hex dump (first 32 bytes):
>>>>      c0 56 2f 88 00 00 00 00 00 00 00 0b 00 00 00 08  .V/.............
>>>>      00 00 00 08 00 00 00 01 00 00 00 01 00 00 00 01  ................
>>>>    backtrace:
>>>>      [<6c69baf5>] trie_alloc+0xb0/0x150
>>>>      [<fa093284>] SyS_bpf+0x288/0x1458
>>>>      [<82182f53>] ret_from_syscall+0x0/0x38
>>>
>>> hmm. looks real. Is there a reproducer?
>>> Yonghong, lpm map not cleaning after itself?
>>
>> Not really. I simply boot up my machine and wait for the first kmemleak scan.
>
> I am not able to reproduce the issue. Tried with latest net-next on FC26 with kmemleak on. I only got this one after bootup,
> 'cat /sys/kernel/debug/kmemleak' or
> 'echo scan > /sys/kernel/debug/kmemleak
>  cat /sys/kernel/debug/kmemleak':
>
> unreferenced object 0xffff99701a7386e0 (size 32):
>   comm "mount", pid 1856, jiffies 4294669263 (age 98.440s)
>   hex dump (first 32 bytes):
>     00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00  ................
>     00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00  ................
>   backtrace:
>     [<000000004668ec00>] security_sb_parse_opts_str+0x36/0x50
>     [<00000000a9807d2b>] parse_security_options+0x3d/0x60
>     [<00000000cc1e1d58>] btrfs_mount_root+0x139/0x720
>     [<00000000bdc4f1a3>] mount_fs+0x30/0x150
>     [<00000000f189f1bd>] vfs_kern_mount.part.26+0x54/0x100
>     [<0000000093ae5db7>] btrfs_mount+0x184/0x914
>     [<00000000bdc4f1a3>] mount_fs+0x30/0x150
>     [<00000000f189f1bd>] vfs_kern_mount.part.26+0x54/0x100
>     [<000000003b67b9fc>] do_mount+0x5b9/0xc70
>     [<00000000de4073a0>] SyS_mount+0x80/0xd0
>     [<00000000fc5a968a>] do_syscall_64+0x5d/0x110
>     [<000000003d61f5fc>] entry_SYSCALL_64_after_hwframe+0x21/0x86
>     [<00000000458a6ffa>] 0xffffffffffffffff
>
> Not sure whether the above is a true issue or not.
>
> However, by inspecting the code, I do find the trie_free in lpm_trie.c
> may have missed freeing the trie memory.
>
> The change likes below should work:
> -bash-4.2$ git diff
> diff --git a/kernel/bpf/lpm_trie.c b/kernel/bpf/lpm_trie.c
> index 7b469d1..cecb259 100644
> --- a/kernel/bpf/lpm_trie.c
> +++ b/kernel/bpf/lpm_trie.c
> @@ -589,6 +589,7 @@ static void trie_free(struct bpf_map *map)
>
>  unlock:
>         raw_spin_unlock(&trie->lock);
> +       kfree(trie);
>  }
>
>  static int trie_get_next_key(struct bpf_map *map, void *_key, void *_next_key)
> -bash-4.2$
>
> Will propose a formal patch for this soon.

Agree, good catch, and I also think that this is the issue, since this
is what kmemleak reports in terms of size (192):

struct lpm_trie {
struct bpf_map map; /* 0 128 */
/* --- cacheline 2 boundary (128 bytes) --- */
struct lpm_trie_node * root; /* 128 8 */
size_t n_entries; /* 136 8 */
size_t max_prefixlen; /* 144 8 */
size_t data_size; /* 152 8 */
raw_spinlock_t lock; /* 160 4 */

/* size: 192, cachelines: 3, members: 6 */
/* padding: 28 */
};

2018-02-12 16:57:18

by Alexei Starovoitov

[permalink] [raw]
Subject: Re: [kmemleak] unreferenced object 0xcd9c1a80 (size 192):

On Mon, Feb 12, 2018 at 09:28:33AM +0100, Daniel Borkmann wrote:
> On 02/12/2018 06:47 AM, Yonghong Song wrote:
> > On 2/11/18 11:18 AM, Mathieu Malaterre wrote:
> >> On Sun, Feb 11, 2018 at 5:54 PM, Alexei Starovoitov
> >> <[email protected]> wrote:
> >>> On Sun, Feb 11, 2018 at 7:24 AM, Mathieu Malaterre <[email protected]> wrote:
> >>>> Alexei,
> >>>>
> >>>> Could you please comment on why I am seeing those memleaks being
> >>>> reported on my ppc32 system ? Should they be marked as false positive
> >>>> ?
> >>>>
> >>>> System is Mac Mini G4, git/master (4.15.0+), ppc.
> >>>>
> >>>> Thanks for your time
> >>>>
> >>>> $ dmesg
> >>>> ...
> >>>> [ 1281.504173] kmemleak: 36 new suspected memory leaks (see
> >>>> /sys/kernel/debug/kmemleak)
> >>>>
> >>>> Where:
> >>>>
> >>>> # cat /sys/kernel/debug/kmemleak
> >>>> unreferenced object 0xdee25000 (size 192):
> >>>> ?? comm "systemd", pid 1, jiffies 4294894348 (age 1438.580s)
> >>>> ?? hex dump (first 32 bytes):
> >>>> ???? c0 56 2f 88 00 00 00 00 00 00 00 0b 00 00 00 0c? .V/.............
> >>>> ???? 00 00 00 08 00 00 00 01 00 00 00 01 00 00 00 01? ................
> >>>> ?? backtrace:
> >>>> ???? [<6c69baf5>] trie_alloc+0xb0/0x150
> >>>> ???? [<fa093284>] SyS_bpf+0x288/0x1458
> >>>> ???? [<82182f53>] ret_from_syscall+0x0/0x38
> >>>> unreferenced object 0xdee25900 (size 192):
> >>>> ?? comm "systemd", pid 1, jiffies 4294894540 (age 1437.812s)
> >>>> ?? hex dump (first 32 bytes):
> >>>> ???? c0 56 2f 88 00 00 00 00 00 00 00 0b 00 00 00 08? .V/.............
> >>>> ???? 00 00 00 08 00 00 00 01 00 00 00 01 00 00 00 01? ................
> >>>> ?? backtrace:
> >>>> ???? [<6c69baf5>] trie_alloc+0xb0/0x150
> >>>> ???? [<fa093284>] SyS_bpf+0x288/0x1458
> >>>> ???? [<82182f53>] ret_from_syscall+0x0/0x38
> >>>
> >>> hmm. looks real. Is there a reproducer?
> >>> Yonghong, lpm map not cleaning after itself?
> >>
> >> Not really. I simply boot up my machine and wait for the first kmemleak scan.
> >
> > I am not able to reproduce the issue. Tried with latest net-next on FC26 with kmemleak on. I only got this one after bootup,
> > 'cat /sys/kernel/debug/kmemleak' or
> > 'echo scan > /sys/kernel/debug/kmemleak
> > ?cat /sys/kernel/debug/kmemleak':
> >
> > unreferenced object 0xffff99701a7386e0 (size 32):
> > ? comm "mount", pid 1856, jiffies 4294669263 (age 98.440s)
> > ? hex dump (first 32 bytes):
> > ??? 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00? ................
> > ??? 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00? ................
> > ? backtrace:
> > ??? [<000000004668ec00>] security_sb_parse_opts_str+0x36/0x50
> > ??? [<00000000a9807d2b>] parse_security_options+0x3d/0x60
> > ??? [<00000000cc1e1d58>] btrfs_mount_root+0x139/0x720
> > ??? [<00000000bdc4f1a3>] mount_fs+0x30/0x150
> > ??? [<00000000f189f1bd>] vfs_kern_mount.part.26+0x54/0x100
> > ??? [<0000000093ae5db7>] btrfs_mount+0x184/0x914
> > ??? [<00000000bdc4f1a3>] mount_fs+0x30/0x150
> > ??? [<00000000f189f1bd>] vfs_kern_mount.part.26+0x54/0x100
> > ??? [<000000003b67b9fc>] do_mount+0x5b9/0xc70
> > ??? [<00000000de4073a0>] SyS_mount+0x80/0xd0
> > ??? [<00000000fc5a968a>] do_syscall_64+0x5d/0x110
> > ??? [<000000003d61f5fc>] entry_SYSCALL_64_after_hwframe+0x21/0x86
> > ??? [<00000000458a6ffa>] 0xffffffffffffffff
> >
> > Not sure whether the above is a true issue or not.
> >
> > However, by inspecting the code, I do find the trie_free in lpm_trie.c
> > may have missed freeing the trie memory.
> >
> > The change likes below should work:
> > -bash-4.2$ git diff
> > diff --git a/kernel/bpf/lpm_trie.c b/kernel/bpf/lpm_trie.c
> > index 7b469d1..cecb259 100644
> > --- a/kernel/bpf/lpm_trie.c
> > +++ b/kernel/bpf/lpm_trie.c
> > @@ -589,6 +589,7 @@ static void trie_free(struct bpf_map *map)
> >
> > ?unlock:
> > ??????? raw_spin_unlock(&trie->lock);
> > +?????? kfree(trie);

also looks like trie_free() is missing
synchronize_rcu() + rcu_barrier()
it doesn't wait for parallel lookup/update/delete to complete
before freeing the elements.


2018-02-12 17:04:13

by Yonghong Song

[permalink] [raw]
Subject: Re: [kmemleak] unreferenced object 0xcd9c1a80 (size 192):



On 2/12/18 7:55 AM, Alexei Starovoitov wrote:
> On Mon, Feb 12, 2018 at 09:28:33AM +0100, Daniel Borkmann wrote:
>> On 02/12/2018 06:47 AM, Yonghong Song wrote:
>>> On 2/11/18 11:18 AM, Mathieu Malaterre wrote:
>>>> On Sun, Feb 11, 2018 at 5:54 PM, Alexei Starovoitov
>>>> <[email protected]> wrote:
>>>>> On Sun, Feb 11, 2018 at 7:24 AM, Mathieu Malaterre <[email protected]> wrote:
>>>>>> Alexei,
>>>>>>
>>>>>> Could you please comment on why I am seeing those memleaks being
>>>>>> reported on my ppc32 system ? Should they be marked as false positive
>>>>>> ?
>>>>>>
>>>>>> System is Mac Mini G4, git/master (4.15.0+), ppc.
>>>>>>
>>>>>> Thanks for your time
>>>>>>
>>>>>> $ dmesg
>>>>>> ...
>>>>>> [ 1281.504173] kmemleak: 36 new suspected memory leaks (see
>>>>>> /sys/kernel/debug/kmemleak)
>>>>>>
>>>>>> Where:
>>>>>>
>>>>>> # cat /sys/kernel/debug/kmemleak
>>>>>> unreferenced object 0xdee25000 (size 192):
>>>>>>    comm "systemd", pid 1, jiffies 4294894348 (age 1438.580s)
>>>>>>    hex dump (first 32 bytes):
>>>>>>      c0 56 2f 88 00 00 00 00 00 00 00 0b 00 00 00 0c  .V/.............
>>>>>>      00 00 00 08 00 00 00 01 00 00 00 01 00 00 00 01  ................
>>>>>>    backtrace:
>>>>>>      [<6c69baf5>] trie_alloc+0xb0/0x150
>>>>>>      [<fa093284>] SyS_bpf+0x288/0x1458
>>>>>>      [<82182f53>] ret_from_syscall+0x0/0x38
>>>>>> unreferenced object 0xdee25900 (size 192):
>>>>>>    comm "systemd", pid 1, jiffies 4294894540 (age 1437.812s)
>>>>>>    hex dump (first 32 bytes):
>>>>>>      c0 56 2f 88 00 00 00 00 00 00 00 0b 00 00 00 08  .V/.............
>>>>>>      00 00 00 08 00 00 00 01 00 00 00 01 00 00 00 01  ................
>>>>>>    backtrace:
>>>>>>      [<6c69baf5>] trie_alloc+0xb0/0x150
>>>>>>      [<fa093284>] SyS_bpf+0x288/0x1458
>>>>>>      [<82182f53>] ret_from_syscall+0x0/0x38
>>>>>
>>>>> hmm. looks real. Is there a reproducer?
>>>>> Yonghong, lpm map not cleaning after itself?
>>>>
>>>> Not really. I simply boot up my machine and wait for the first kmemleak scan.
>>>
>>> I am not able to reproduce the issue. Tried with latest net-next on FC26 with kmemleak on. I only got this one after bootup,
>>> 'cat /sys/kernel/debug/kmemleak' or
>>> 'echo scan > /sys/kernel/debug/kmemleak
>>>  cat /sys/kernel/debug/kmemleak':
>>>
>>> unreferenced object 0xffff99701a7386e0 (size 32):
>>>   comm "mount", pid 1856, jiffies 4294669263 (age 98.440s)
>>>   hex dump (first 32 bytes):
>>>     00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00  ................
>>>     00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00  ................
>>>   backtrace:
>>>     [<000000004668ec00>] security_sb_parse_opts_str+0x36/0x50
>>>     [<00000000a9807d2b>] parse_security_options+0x3d/0x60
>>>     [<00000000cc1e1d58>] btrfs_mount_root+0x139/0x720
>>>     [<00000000bdc4f1a3>] mount_fs+0x30/0x150
>>>     [<00000000f189f1bd>] vfs_kern_mount.part.26+0x54/0x100
>>>     [<0000000093ae5db7>] btrfs_mount+0x184/0x914
>>>     [<00000000bdc4f1a3>] mount_fs+0x30/0x150
>>>     [<00000000f189f1bd>] vfs_kern_mount.part.26+0x54/0x100
>>>     [<000000003b67b9fc>] do_mount+0x5b9/0xc70
>>>     [<00000000de4073a0>] SyS_mount+0x80/0xd0
>>>     [<00000000fc5a968a>] do_syscall_64+0x5d/0x110
>>>     [<000000003d61f5fc>] entry_SYSCALL_64_after_hwframe+0x21/0x86
>>>     [<00000000458a6ffa>] 0xffffffffffffffff
>>>
>>> Not sure whether the above is a true issue or not.
>>>
>>> However, by inspecting the code, I do find the trie_free in lpm_trie.c
>>> may have missed freeing the trie memory.
>>>
>>> The change likes below should work:
>>> -bash-4.2$ git diff
>>> diff --git a/kernel/bpf/lpm_trie.c b/kernel/bpf/lpm_trie.c
>>> index 7b469d1..cecb259 100644
>>> --- a/kernel/bpf/lpm_trie.c
>>> +++ b/kernel/bpf/lpm_trie.c
>>> @@ -589,6 +589,7 @@ static void trie_free(struct bpf_map *map)
>>>
>>>  unlock:
>>>         raw_spin_unlock(&trie->lock);
>>> +       kfree(trie);
>
> also looks like trie_free() is missing
> synchronize_rcu() + rcu_barrier()
> it doesn't wait for parallel lookup/update/delete to complete
> before freeing the elements.

Thanks, Alexei. I will address this in the patch as well.

2018-02-12 19:27:38

by Mathieu Malaterre

[permalink] [raw]
Subject: Re: [kmemleak] unreferenced object 0xcd9c1a80 (size 192):

On Mon, Feb 12, 2018 at 6:47 AM, Yonghong Song <[email protected]> wrote:
>
>
> On 2/11/18 11:18 AM, Mathieu Malaterre wrote:
>>
>> Hi,
>>
>> On Sun, Feb 11, 2018 at 5:54 PM, Alexei Starovoitov
>> <[email protected]> wrote:
>>>
>>> On Sun, Feb 11, 2018 at 7:24 AM, Mathieu Malaterre <[email protected]>
>>> wrote:
>>>>
>>>> Alexei,
>>>>
>>>> Could you please comment on why I am seeing those memleaks being
>>>> reported on my ppc32 system ? Should they be marked as false positive
>>>> ?
>>>>
>>>> System is Mac Mini G4, git/master (4.15.0+), ppc.
>>>>
>>>> Thanks for your time
>>>>
>>>> $ dmesg
>>>> ...
>>>> [ 1281.504173] kmemleak: 36 new suspected memory leaks (see
>>>> /sys/kernel/debug/kmemleak)
>>>>
>>>> Where:
>>>>
>>>> # cat /sys/kernel/debug/kmemleak
>>>> unreferenced object 0xdee25000 (size 192):
>>>> comm "systemd", pid 1, jiffies 4294894348 (age 1438.580s)
>>>> hex dump (first 32 bytes):
>>>> c0 56 2f 88 00 00 00 00 00 00 00 0b 00 00 00 0c .V/.............
>>>> 00 00 00 08 00 00 00 01 00 00 00 01 00 00 00 01 ................
>>>> backtrace:
>>>> [<6c69baf5>] trie_alloc+0xb0/0x150
>>>> [<fa093284>] SyS_bpf+0x288/0x1458
>>>> [<82182f53>] ret_from_syscall+0x0/0x38
>>>> unreferenced object 0xdee25900 (size 192):
>>>> comm "systemd", pid 1, jiffies 4294894540 (age 1437.812s)
>>>> hex dump (first 32 bytes):
>>>> c0 56 2f 88 00 00 00 00 00 00 00 0b 00 00 00 08 .V/.............
>>>> 00 00 00 08 00 00 00 01 00 00 00 01 00 00 00 01 ................
>>>> backtrace:
>>>> [<6c69baf5>] trie_alloc+0xb0/0x150
>>>> [<fa093284>] SyS_bpf+0x288/0x1458
>>>> [<82182f53>] ret_from_syscall+0x0/0x38
>>>
>>>
>>> hmm. looks real. Is there a reproducer?
>>> Yonghong, lpm map not cleaning after itself?
>>
>>
>> Not really. I simply boot up my machine and wait for the first kmemleak
>> scan.
>
>
> I am not able to reproduce the issue. Tried with latest net-next on FC26
> with kmemleak on. I only got this one after bootup,
> 'cat /sys/kernel/debug/kmemleak' or
> 'echo scan > /sys/kernel/debug/kmemleak
> cat /sys/kernel/debug/kmemleak':
>
> unreferenced object 0xffff99701a7386e0 (size 32):
> comm "mount", pid 1856, jiffies 4294669263 (age 98.440s)
> hex dump (first 32 bytes):
> 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 ................
> 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 ................
> backtrace:
> [<000000004668ec00>] security_sb_parse_opts_str+0x36/0x50
> [<00000000a9807d2b>] parse_security_options+0x3d/0x60
> [<00000000cc1e1d58>] btrfs_mount_root+0x139/0x720
> [<00000000bdc4f1a3>] mount_fs+0x30/0x150
> [<00000000f189f1bd>] vfs_kern_mount.part.26+0x54/0x100
> [<0000000093ae5db7>] btrfs_mount+0x184/0x914
> [<00000000bdc4f1a3>] mount_fs+0x30/0x150
> [<00000000f189f1bd>] vfs_kern_mount.part.26+0x54/0x100
> [<000000003b67b9fc>] do_mount+0x5b9/0xc70
> [<00000000de4073a0>] SyS_mount+0x80/0xd0
> [<00000000fc5a968a>] do_syscall_64+0x5d/0x110
> [<000000003d61f5fc>] entry_SYSCALL_64_after_hwframe+0x21/0x86
> [<00000000458a6ffa>] 0xffffffffffffffff
>
> Not sure whether the above is a true issue or not.
>
> However, by inspecting the code, I do find the trie_free in lpm_trie.c
> may have missed freeing the trie memory.
>
> The change likes below should work:
> -bash-4.2$ git diff
> diff --git a/kernel/bpf/lpm_trie.c b/kernel/bpf/lpm_trie.c
> index 7b469d1..cecb259 100644
> --- a/kernel/bpf/lpm_trie.c
> +++ b/kernel/bpf/lpm_trie.c
> @@ -589,6 +589,7 @@ static void trie_free(struct bpf_map *map)
>
> unlock:
> raw_spin_unlock(&trie->lock);
> + kfree(trie);
> }
>
> static int trie_get_next_key(struct bpf_map *map, void *_key, void
> *_next_key)
> -bash-4.2$

With this single patch added, system has been running for a couple of
hours, no memleak reported. So:

Tested-by: Mathieu Malaterre <[email protected]>

> Will propose a formal patch for this soon.

Will retest if needed.

Thanks !
-M