-tip testing found the rather scary looking slab corruption:
[ 35.419875] CPU0 attaching sched-domain:
[ 35.420101] domain 0: span 0-1 level CPU
[ 35.425883] groups: 0 1
[ 35.428527] CPU1 attaching sched-domain:
[ 35.432010] domain 0: span 0-1 level CPU
[ 35.437729] groups: 1 0
[ 37.380005] eth0: no IPv6 routers present
[ 44.478286] =============================================================================
[ 44.482064] BUG key_jar: Poison overwritten
[ 44.482064] -----------------------------------------------------------------------------
[ 44.482064]
[ 44.482064] INFO: 0xf5f320c0-0xf5f320c0. First byte 0x6a instead of 0x6b
[ 44.482064] INFO: Allocated in key_alloc+0xe7/0x30e age=291 cpu=1 pid=2815
[ 44.482064] INFO: Freed in key_cleanup+0xd8/0xdd age=292 cpu=1 pid=2520
[ 44.482064] INFO: Slab 0xc1f9cfb8 objects=21 used=2 fp=0xf5f320c0 flags=0x400000c3
[ 44.482064] INFO: Object 0xf5f320c0 @offset=192 fp=0xf5f32240
[ 44.482064]
[ 44.482064] Bytes b4 0xf5f320b0: 7c 05 ff ff 5a 5a 5a 5a 5a 5a 5a 5a 5a 5a 5a 5a |.��ZZZZZZZZZZZZ
[ 44.482064] Object 0xf5f320c0: 6a 6b 6b 6b 6b 6b 6b 6b 6b 6b 6b 6b 6b 6b 6b 6b jkkkkkkkkkkkkkkk
[ 44.482064] Object 0xf5f320d0: 6b 6b 6b 6b 6b 6b 6b 6b 6b 6b 6b 6b 6b 6b 6b 6b kkkkkkkkkkkkkkkk
[ 44.482064] Object 0xf5f320e0: 6b 6b 6b 6b 6b 6b 6b 6b 6b 6b 6b 6b 6b 6b 6b 6b kkkkkkkkkkkkkkkk
[ 44.482064] Object 0xf5f320f0: 6b 6b 6b 6b 6b 6b 6b 6b 6b 6b 6b 6b 6b 6b 6b 6b kkkkkkkkkkkkkkkk
[ 44.482064] Object 0xf5f32100: 6b 6b 6b 6b 6b 6b 6b 6b 6b 6b 6b 6b 6b 6b 6b 6b kkkkkkkkkkkkkkkk
[ 44.482064] Object 0xf5f32110: 6b 6b 6b 6b 6b 6b 6b 6b 6b 6b 6b 6b 6b 6b 6b 6b kkkkkkkkkkkkkkkk
[ 44.482064] Object 0xf5f32120: 6b 6b 6b 6b 6b 6b 6b 6b 6b 6b 6b 6b 6b 6b 6b 6b kkkkkkkkkkkkkkkk
[ 44.482064] Object 0xf5f32130: 6b 6b 6b 6b 6b 6b 6b 6b 6b 6b 6b 6b 6b 6b 6b 6b kkkkkkkkkkkkkkkk
[ 44.482064] Object 0xf5f32140: 6b 6b 6b 6b 6b 6b 6b 6b 6b 6b 6b a5 kkkkkkkkkkk�
[ 44.482064] Redzone 0xf5f3214c: bb bb bb bb ����
[ 44.482064] Padding 0xf5f32174: 5a 5a 5a 5a 5a 5a 5a 5a 5a 5a 5a 5a ZZZZZZZZZZZZ
[ 44.482064] Pid: 2832, comm: sudo Not tainted 2.6.29-rc1-tip-01097-gf5d0b1b-dirty #16445
[ 44.482064] Call Trace:
[ 44.482064] [<c018e25c>] print_trailer+0xcd/0xd5
[ 44.482064] [<c018e2dc>] check_bytes_and_report+0x78/0x94
[ 44.482064] [<c018e55c>] check_object+0xa9/0x191
[ 44.482064] [<c018eff3>] __slab_alloc+0x365/0x42c
[ 44.482064] [<c0144dda>] ? trace_hardirqs_off+0xb/0xd
[ 44.482064] [<c018f369>] kmem_cache_alloc+0x64/0xc1
[ 44.482064] [<c02d040e>] ? key_alloc+0xe7/0x30e
[ 44.482064] [<c02d040e>] ? key_alloc+0xe7/0x30e
[ 44.482064] [<c02d040e>] key_alloc+0xe7/0x30e
[ 44.482064] [<c02d1429>] keyring_alloc+0x24/0x58
[ 44.482064] [<c02d2781>] install_session_keyring_to_cred+0x43/0x92
[ 44.482064] [<c02d2d61>] lookup_user_key+0xe0/0x30b
[ 44.482064] [<c02d17ab>] keyctl_get_keyring_ID+0x12/0x2e
[ 44.482064] [<c02d22c2>] sys_keyctl+0x36/0xe3
[ 44.482064] [<c0102d55>] sysenter_do_call+0x12/0x35
[ 44.482064] FIX key_jar: Restoring 0xf5f320c0-0xf5f320c0=0x6b
[ 44.482064]
[ 44.482064] FIX key_jar: Marking all objects used
Config attached. The system seemed to stay intact after this incident.
Ingo
On Thu, Jan 15, 2009 at 7:16 PM, Ingo Molnar <[email protected]> wrote:>> -tip testing found the rather scary looking slab corruption:>> [ 35.419875] CPU0 attaching sched-domain:> [ 35.420101] domain 0: span 0-1 level CPU> [ 35.425883] groups: 0 1> [ 35.428527] CPU1 attaching sched-domain:> [ 35.432010] domain 0: span 0-1 level CPU> [ 35.437729] groups: 1 0> [ 37.380005] eth0: no IPv6 routers present> [ 44.478286] =============================================================================> [ 44.482064] BUG key_jar: Poison overwritten> [ 44.482064] -----------------------------------------------------------------------------> [ 44.482064]> [ 44.482064] INFO: 0xf5f320c0-0xf5f320c0. First byte 0x6a instead of 0x6b> [ 44.482064] INFO: Allocated in key_alloc+0xe7/0x30e age=291 cpu=1 pid=2815> [ 44.482064] INFO: Freed in key_cleanup+0xd8/0xdd age=292 cpu=1 pid=2520> [ 44.482064] INFO: Slab 0xc1f9cfb8 objects=21 used=2 fp=0xf5f320c0 flags=0x400000c3> [ 44.482064] INFO: Object 0xf5f320c0 @offset=192 fp=0xf5f32240> [ 44.482064]> [ 44.482064] Bytes b4 0xf5f320b0: 7c 05 ff ff 5a 5a 5a 5a 5a 5a 5a 5a 5a 5a 5a 5a |.��ZZZZZZZZZZZZ> [ 44.482064] Object 0xf5f320c0: 6a 6b 6b 6b 6b 6b 6b 6b 6b 6b 6b 6b 6b 6b 6b 6b jkkkkkkkkkkkkkkk> [ 44.482064] Object 0xf5f320d0: 6b 6b 6b 6b 6b 6b 6b 6b 6b 6b 6b 6b 6b 6b 6b 6b kkkkkkkkkkkkkkkk> [ 44.482064] Object 0xf5f320e0: 6b 6b 6b 6b 6b 6b 6b 6b 6b 6b 6b 6b 6b 6b 6b 6b kkkkkkkkkkkkkkkk> [ 44.482064] Object 0xf5f320f0: 6b 6b 6b 6b 6b 6b 6b 6b 6b 6b 6b 6b 6b 6b 6b 6b kkkkkkkkkkkkkkkk> [ 44.482064] Object 0xf5f32100: 6b 6b 6b 6b 6b 6b 6b 6b 6b 6b 6b 6b 6b 6b 6b 6b kkkkkkkkkkkkkkkk> [ 44.482064] Object 0xf5f32110: 6b 6b 6b 6b 6b 6b 6b 6b 6b 6b 6b 6b 6b 6b 6b 6b kkkkkkkkkkkkkkkk> [ 44.482064] Object 0xf5f32120: 6b 6b 6b 6b 6b 6b 6b 6b 6b 6b 6b 6b 6b 6b 6b 6b kkkkkkkkkkkkkkkk> [ 44.482064] Object 0xf5f32130: 6b 6b 6b 6b 6b 6b 6b 6b 6b 6b 6b 6b 6b 6b 6b 6b kkkkkkkkkkkkkkkk> [ 44.482064] Object 0xf5f32140: 6b 6b 6b 6b 6b 6b 6b 6b 6b 6b 6b a5 kkkkkkkkkkk�> [ 44.482064] Redzone 0xf5f3214c: bb bb bb bb ����> [ 44.482064] Padding 0xf5f32174: 5a 5a 5a 5a 5a 5a 5a 5a 5a 5a 5a 5a ZZZZZZZZZZZZ> [ 44.482064] Pid: 2832, comm: sudo Not tainted 2.6.29-rc1-tip-01097-gf5d0b1b-dirty #16445> [ 44.482064] Call Trace:> [ 44.482064] [<c018e25c>] print_trailer+0xcd/0xd5> [ 44.482064] [<c018e2dc>] check_bytes_and_report+0x78/0x94> [ 44.482064] [<c018e55c>] check_object+0xa9/0x191> [ 44.482064] [<c018eff3>] __slab_alloc+0x365/0x42c> [ 44.482064] [<c0144dda>] ? trace_hardirqs_off+0xb/0xd> [ 44.482064] [<c018f369>] kmem_cache_alloc+0x64/0xc1> [ 44.482064] [<c02d040e>] ? key_alloc+0xe7/0x30e> [ 44.482064] [<c02d040e>] ? key_alloc+0xe7/0x30e> [ 44.482064] [<c02d040e>] key_alloc+0xe7/0x30e> [ 44.482064] [<c02d1429>] keyring_alloc+0x24/0x58> [ 44.482064] [<c02d2781>] install_session_keyring_to_cred+0x43/0x92> [ 44.482064] [<c02d2d61>] lookup_user_key+0xe0/0x30b> [ 44.482064] [<c02d17ab>] keyctl_get_keyring_ID+0x12/0x2e> [ 44.482064] [<c02d22c2>] sys_keyctl+0x36/0xe3> [ 44.482064] [<c0102d55>] sysenter_do_call+0x12/0x35> [ 44.482064] FIX key_jar: Restoring 0xf5f320c0-0xf5f320c0=0x6b> [ 44.482064]> [ 44.482064] FIX key_jar: Marking all objects used>> Config attached. The system seemed to stay intact after this incident.
Are you sure this is the right config?
# Security options## CONFIG_KEYS is not set
...sys_keyctl() returns -ENOSYS...!?
Vegard
-- "The animistic metaphor of the bug that maliciously sneaked in whilethe programmer was not looking is intellectually dishonest as itdisguises that the error is the programmer's own creation." -- E. W. Dijkstra, EWD1036????{.n?+???????+%?????ݶ??w??{.n?+????{??G?????{ay? ʇڙ?,j??f???h?????????z_??(?階?ݢj"???m??????G????????????&???~???iO???z??v?^?m???? ????????I?
* Vegard Nossum <[email protected]> wrote:
> On Thu, Jan 15, 2009 at 7:16 PM, Ingo Molnar <[email protected]> wrote:
> >
> > -tip testing found the rather scary looking slab corruption:
> >
> > [ 35.419875] CPU0 attaching sched-domain:
> > [ 35.420101] domain 0: span 0-1 level CPU
> > [ 35.425883] groups: 0 1
> > [ 35.428527] CPU1 attaching sched-domain:
> > [ 35.432010] domain 0: span 0-1 level CPU
> > [ 35.437729] groups: 1 0
> > [ 37.380005] eth0: no IPv6 routers present
> > [ 44.478286] =============================================================================
> > [ 44.482064] BUG key_jar: Poison overwritten
> > [ 44.482064] -----------------------------------------------------------------------------
> > [ 44.482064]
> > [ 44.482064] INFO: 0xf5f320c0-0xf5f320c0. First byte 0x6a instead of 0x6b
> > [ 44.482064] INFO: Allocated in key_alloc+0xe7/0x30e age=291 cpu=1 pid=2815
> > [ 44.482064] INFO: Freed in key_cleanup+0xd8/0xdd age=292 cpu=1 pid=2520
> > [ 44.482064] INFO: Slab 0xc1f9cfb8 objects=21 used=2 fp=0xf5f320c0 flags=0x400000c3
> > [ 44.482064] INFO: Object 0xf5f320c0 @offset=192 fp=0xf5f32240
> > [ 44.482064]
> > [ 44.482064] Bytes b4 0xf5f320b0: 7c 05 ff ff 5a 5a 5a 5a 5a 5a 5a 5a 5a 5a 5a 5a |.��ZZZZZZZZZZZZ
> > [ 44.482064] Object 0xf5f320c0: 6a 6b 6b 6b 6b 6b 6b 6b 6b 6b 6b 6b 6b 6b 6b 6b jkkkkkkkkkkkkkkk
> > [ 44.482064] Object 0xf5f320d0: 6b 6b 6b 6b 6b 6b 6b 6b 6b 6b 6b 6b 6b 6b 6b 6b kkkkkkkkkkkkkkkk
> > [ 44.482064] Object 0xf5f320e0: 6b 6b 6b 6b 6b 6b 6b 6b 6b 6b 6b 6b 6b 6b 6b 6b kkkkkkkkkkkkkkkk
> > [ 44.482064] Object 0xf5f320f0: 6b 6b 6b 6b 6b 6b 6b 6b 6b 6b 6b 6b 6b 6b 6b 6b kkkkkkkkkkkkkkkk
> > [ 44.482064] Object 0xf5f32100: 6b 6b 6b 6b 6b 6b 6b 6b 6b 6b 6b 6b 6b 6b 6b 6b kkkkkkkkkkkkkkkk
> > [ 44.482064] Object 0xf5f32110: 6b 6b 6b 6b 6b 6b 6b 6b 6b 6b 6b 6b 6b 6b 6b 6b kkkkkkkkkkkkkkkk
> > [ 44.482064] Object 0xf5f32120: 6b 6b 6b 6b 6b 6b 6b 6b 6b 6b 6b 6b 6b 6b 6b 6b kkkkkkkkkkkkkkkk
> > [ 44.482064] Object 0xf5f32130: 6b 6b 6b 6b 6b 6b 6b 6b 6b 6b 6b 6b 6b 6b 6b 6b kkkkkkkkkkkkkkkk
> > [ 44.482064] Object 0xf5f32140: 6b 6b 6b 6b 6b 6b 6b 6b 6b 6b 6b a5 kkkkkkkkkkk�
> > [ 44.482064] Redzone 0xf5f3214c: bb bb bb bb ����
> > [ 44.482064] Padding 0xf5f32174: 5a 5a 5a 5a 5a 5a 5a 5a 5a 5a 5a 5a ZZZZZZZZZZZZ
> > [ 44.482064] Pid: 2832, comm: sudo Not tainted 2.6.29-rc1-tip-01097-gf5d0b1b-dirty #16445
> > [ 44.482064] Call Trace:
> > [ 44.482064] [<c018e25c>] print_trailer+0xcd/0xd5
> > [ 44.482064] [<c018e2dc>] check_bytes_and_report+0x78/0x94
> > [ 44.482064] [<c018e55c>] check_object+0xa9/0x191
> > [ 44.482064] [<c018eff3>] __slab_alloc+0x365/0x42c
> > [ 44.482064] [<c0144dda>] ? trace_hardirqs_off+0xb/0xd
> > [ 44.482064] [<c018f369>] kmem_cache_alloc+0x64/0xc1
> > [ 44.482064] [<c02d040e>] ? key_alloc+0xe7/0x30e
> > [ 44.482064] [<c02d040e>] ? key_alloc+0xe7/0x30e
> > [ 44.482064] [<c02d040e>] key_alloc+0xe7/0x30e
> > [ 44.482064] [<c02d1429>] keyring_alloc+0x24/0x58
> > [ 44.482064] [<c02d2781>] install_session_keyring_to_cred+0x43/0x92
> > [ 44.482064] [<c02d2d61>] lookup_user_key+0xe0/0x30b
> > [ 44.482064] [<c02d17ab>] keyctl_get_keyring_ID+0x12/0x2e
> > [ 44.482064] [<c02d22c2>] sys_keyctl+0x36/0xe3
> > [ 44.482064] [<c0102d55>] sysenter_do_call+0x12/0x35
> > [ 44.482064] FIX key_jar: Restoring 0xf5f320c0-0xf5f320c0=0x6b
> > [ 44.482064]
> > [ 44.482064] FIX key_jar: Marking all objects used
> >
> > Config attached. The system seemed to stay intact after this incident.
>
>
> Are you sure this is the right config?
>
> # Security options
> #
> # CONFIG_KEYS is not set
>
> ...sys_keyctl() returns -ENOSYS...!?
no, sorry. The right 'bad' config attached.
Ingo
On Thu, Jan 15, 2009 at 10:01 PM, Ingo Molnar <[email protected]> wrote:>> * Vegard Nossum <[email protected]> wrote:>>> On Thu, Jan 15, 2009 at 7:16 PM, Ingo Molnar <[email protected]> wrote:>> >>> > -tip testing found the rather scary looking slab corruption:>> >>> > [ 35.419875] CPU0 attaching sched-domain:>> > [ 35.420101] domain 0: span 0-1 level CPU>> > [ 35.425883] groups: 0 1>> > [ 35.428527] CPU1 attaching sched-domain:>> > [ 35.432010] domain 0: span 0-1 level CPU>> > [ 35.437729] groups: 1 0>> > [ 37.380005] eth0: no IPv6 routers present>> > [ 44.478286] =============================================================================>> > [ 44.482064] BUG key_jar: Poison overwritten>> > [ 44.482064] ----------------------------------------------------------------------------->> > [ 44.482064]>> > [ 44.482064] INFO: 0xf5f320c0-0xf5f320c0. First byte 0x6a instead of 0x6b>> > [ 44.482064] INFO: Allocated in key_alloc+0xe7/0x30e age=291 cpu=1 pid=2815>> > [ 44.482064] INFO: Freed in key_cleanup+0xd8/0xdd age=292 cpu=1 pid=2520>> > [ 44.482064] INFO: Slab 0xc1f9cfb8 objects=21 used=2 fp=0xf5f320c0 flags=0x400000c3>> > [ 44.482064] INFO: Object 0xf5f320c0 @offset=192 fp=0xf5f32240>> > [ 44.482064]>> > [ 44.482064] Bytes b4 0xf5f320b0: 7c 05 ff ff 5a 5a 5a 5a 5a 5a 5a 5a 5a 5a 5a 5a |.��ZZZZZZZZZZZZ>> > [ 44.482064] Object 0xf5f320c0: 6a 6b 6b 6b 6b 6b 6b 6b 6b 6b 6b 6b 6b 6b 6b 6b jkkkkkkkkkkkkkkk>> > [ 44.482064] Object 0xf5f320d0: 6b 6b 6b 6b 6b 6b 6b 6b 6b 6b 6b 6b 6b 6b 6b 6b kkkkkkkkkkkkkkkk>> > [ 44.482064] Object 0xf5f320e0: 6b 6b 6b 6b 6b 6b 6b 6b 6b 6b 6b 6b 6b 6b 6b 6b kkkkkkkkkkkkkkkk>> > [ 44.482064] Object 0xf5f320f0: 6b 6b 6b 6b 6b 6b 6b 6b 6b 6b 6b 6b 6b 6b 6b 6b kkkkkkkkkkkkkkkk>> > [ 44.482064] Object 0xf5f32100: 6b 6b 6b 6b 6b 6b 6b 6b 6b 6b 6b 6b 6b 6b 6b 6b kkkkkkkkkkkkkkkk>> > [ 44.482064] Object 0xf5f32110: 6b 6b 6b 6b 6b 6b 6b 6b 6b 6b 6b 6b 6b 6b 6b 6b kkkkkkkkkkkkkkkk>> > [ 44.482064] Object 0xf5f32120: 6b 6b 6b 6b 6b 6b 6b 6b 6b 6b 6b 6b 6b 6b 6b 6b kkkkkkkkkkkkkkkk>> > [ 44.482064] Object 0xf5f32130: 6b 6b 6b 6b 6b 6b 6b 6b 6b 6b 6b 6b 6b 6b 6b 6b kkkkkkkkkkkkkkkk>> > [ 44.482064] Object 0xf5f32140: 6b 6b 6b 6b 6b 6b 6b 6b 6b 6b 6b a5 kkkkkkkkkkk�>> > [ 44.482064] Redzone 0xf5f3214c: bb bb bb bb ����>> > [ 44.482064] Padding 0xf5f32174: 5a 5a 5a 5a 5a 5a 5a 5a 5a 5a 5a 5a ZZZZZZZZZZZZ>> > [ 44.482064] Pid: 2832, comm: sudo Not tainted 2.6.29-rc1-tip-01097-gf5d0b1b-dirty #16445>> > [ 44.482064] Call Trace:>> > [ 44.482064] [<c018e25c>] print_trailer+0xcd/0xd5>> > [ 44.482064] [<c018e2dc>] check_bytes_and_report+0x78/0x94>> > [ 44.482064] [<c018e55c>] check_object+0xa9/0x191>> > [ 44.482064] [<c018eff3>] __slab_alloc+0x365/0x42c>> > [ 44.482064] [<c0144dda>] ? trace_hardirqs_off+0xb/0xd>> > [ 44.482064] [<c018f369>] kmem_cache_alloc+0x64/0xc1>> > [ 44.482064] [<c02d040e>] ? key_alloc+0xe7/0x30e>> > [ 44.482064] [<c02d040e>] ? key_alloc+0xe7/0x30e>> > [ 44.482064] [<c02d040e>] key_alloc+0xe7/0x30e>> > [ 44.482064] [<c02d1429>] keyring_alloc+0x24/0x58>> > [ 44.482064] [<c02d2781>] install_session_keyring_to_cred+0x43/0x92>> > [ 44.482064] [<c02d2d61>] lookup_user_key+0xe0/0x30b>> > [ 44.482064] [<c02d17ab>] keyctl_get_keyring_ID+0x12/0x2e>> > [ 44.482064] [<c02d22c2>] sys_keyctl+0x36/0xe3>> > [ 44.482064] [<c0102d55>] sysenter_do_call+0x12/0x35>> > [ 44.482064] FIX key_jar: Restoring 0xf5f320c0-0xf5f320c0=0x6b>> > [ 44.482064]>> > [ 44.482064] FIX key_jar: Marking all objects used>> >>> > Config attached. The system seemed to stay intact after this incident.>>>>>> Are you sure this is the right config?>>>> # Security options>> #>> # CONFIG_KEYS is not set>>>> ...sys_keyctl() returns -ENOSYS...!?>> no, sorry. The right 'bad' config attached.
Are you sure, is this expected?
CC kernel/printk.occ1: warnings being treated as errorsarch/x86/kernel/cpu/intel_cacheinfo.c: In function 'show_cache_disable':arch/x86/kernel/cpu/intel_cacheinfo.c:710: error: unused variable 'mask'arch/x86/kernel/cpu/intel_cacheinfo.c: In function 'store_cache_disable':arch/x86/kernel/cpu/intel_cacheinfo.c:745: error: unused variable 'mask'make[2]: *** [arch/x86/kernel/cpu/intel_cacheinfo.o] Error 1make[1]: *** [arch/x86/kernel/cpu] Error 2make: *** [arch/x86/kernel] Error 2make: *** Waiting for unfinished jobs....
Will restart make with CONFIG_ALLOW_WARNINGS=y...
Vegard
-- "The animistic metaphor of the bug that maliciously sneaked in whilethe programmer was not looking is intellectually dishonest as itdisguises that the error is the programmer's own creation." -- E. W. Dijkstra, EWD1036????{.n?+???????+%?????ݶ??w??{.n?+????{??G?????{ay? ʇڙ?,j??f???h?????????z_??(?階?ݢj"???m??????G????????????&???~???iO???z??v?^?m???? ????????I?
* Vegard Nossum <[email protected]> wrote:
> On Thu, Jan 15, 2009 at 10:01 PM, Ingo Molnar <[email protected]> wrote:
> >
> > * Vegard Nossum <[email protected]> wrote:
> >
> >> On Thu, Jan 15, 2009 at 7:16 PM, Ingo Molnar <[email protected]> wrote:
> >> >
> >> > -tip testing found the rather scary looking slab corruption:
> >> >
> >> > [ 35.419875] CPU0 attaching sched-domain:
> >> > [ 35.420101] domain 0: span 0-1 level CPU
> >> > [ 35.425883] groups: 0 1
> >> > [ 35.428527] CPU1 attaching sched-domain:
> >> > [ 35.432010] domain 0: span 0-1 level CPU
> >> > [ 35.437729] groups: 1 0
> >> > [ 37.380005] eth0: no IPv6 routers present
> >> > [ 44.478286] =============================================================================
> >> > [ 44.482064] BUG key_jar: Poison overwritten
> >> > [ 44.482064] -----------------------------------------------------------------------------
> >> > [ 44.482064]
> >> > [ 44.482064] INFO: 0xf5f320c0-0xf5f320c0. First byte 0x6a instead of 0x6b
> >> > [ 44.482064] INFO: Allocated in key_alloc+0xe7/0x30e age=291 cpu=1 pid=2815
> >> > [ 44.482064] INFO: Freed in key_cleanup+0xd8/0xdd age=292 cpu=1 pid=2520
> >> > [ 44.482064] INFO: Slab 0xc1f9cfb8 objects=21 used=2 fp=0xf5f320c0 flags=0x400000c3
> >> > [ 44.482064] INFO: Object 0xf5f320c0 @offset=192 fp=0xf5f32240
> >> > [ 44.482064]
> >> > [ 44.482064] Bytes b4 0xf5f320b0: 7c 05 ff ff 5a 5a 5a 5a 5a 5a 5a 5a 5a 5a 5a 5a |.��ZZZZZZZZZZZZ
> >> > [ 44.482064] Object 0xf5f320c0: 6a 6b 6b 6b 6b 6b 6b 6b 6b 6b 6b 6b 6b 6b 6b 6b jkkkkkkkkkkkkkkk
> >> > [ 44.482064] Object 0xf5f320d0: 6b 6b 6b 6b 6b 6b 6b 6b 6b 6b 6b 6b 6b 6b 6b 6b kkkkkkkkkkkkkkkk
> >> > [ 44.482064] Object 0xf5f320e0: 6b 6b 6b 6b 6b 6b 6b 6b 6b 6b 6b 6b 6b 6b 6b 6b kkkkkkkkkkkkkkkk
> >> > [ 44.482064] Object 0xf5f320f0: 6b 6b 6b 6b 6b 6b 6b 6b 6b 6b 6b 6b 6b 6b 6b 6b kkkkkkkkkkkkkkkk
> >> > [ 44.482064] Object 0xf5f32100: 6b 6b 6b 6b 6b 6b 6b 6b 6b 6b 6b 6b 6b 6b 6b 6b kkkkkkkkkkkkkkkk
> >> > [ 44.482064] Object 0xf5f32110: 6b 6b 6b 6b 6b 6b 6b 6b 6b 6b 6b 6b 6b 6b 6b 6b kkkkkkkkkkkkkkkk
> >> > [ 44.482064] Object 0xf5f32120: 6b 6b 6b 6b 6b 6b 6b 6b 6b 6b 6b 6b 6b 6b 6b 6b kkkkkkkkkkkkkkkk
> >> > [ 44.482064] Object 0xf5f32130: 6b 6b 6b 6b 6b 6b 6b 6b 6b 6b 6b 6b 6b 6b 6b 6b kkkkkkkkkkkkkkkk
> >> > [ 44.482064] Object 0xf5f32140: 6b 6b 6b 6b 6b 6b 6b 6b 6b 6b 6b a5 kkkkkkkkkkk�
> >> > [ 44.482064] Redzone 0xf5f3214c: bb bb bb bb ����
> >> > [ 44.482064] Padding 0xf5f32174: 5a 5a 5a 5a 5a 5a 5a 5a 5a 5a 5a 5a ZZZZZZZZZZZZ
> >> > [ 44.482064] Pid: 2832, comm: sudo Not tainted 2.6.29-rc1-tip-01097-gf5d0b1b-dirty #16445
> >> > [ 44.482064] Call Trace:
> >> > [ 44.482064] [<c018e25c>] print_trailer+0xcd/0xd5
> >> > [ 44.482064] [<c018e2dc>] check_bytes_and_report+0x78/0x94
> >> > [ 44.482064] [<c018e55c>] check_object+0xa9/0x191
> >> > [ 44.482064] [<c018eff3>] __slab_alloc+0x365/0x42c
> >> > [ 44.482064] [<c0144dda>] ? trace_hardirqs_off+0xb/0xd
> >> > [ 44.482064] [<c018f369>] kmem_cache_alloc+0x64/0xc1
> >> > [ 44.482064] [<c02d040e>] ? key_alloc+0xe7/0x30e
> >> > [ 44.482064] [<c02d040e>] ? key_alloc+0xe7/0x30e
> >> > [ 44.482064] [<c02d040e>] key_alloc+0xe7/0x30e
> >> > [ 44.482064] [<c02d1429>] keyring_alloc+0x24/0x58
> >> > [ 44.482064] [<c02d2781>] install_session_keyring_to_cred+0x43/0x92
> >> > [ 44.482064] [<c02d2d61>] lookup_user_key+0xe0/0x30b
> >> > [ 44.482064] [<c02d17ab>] keyctl_get_keyring_ID+0x12/0x2e
> >> > [ 44.482064] [<c02d22c2>] sys_keyctl+0x36/0xe3
> >> > [ 44.482064] [<c0102d55>] sysenter_do_call+0x12/0x35
> >> > [ 44.482064] FIX key_jar: Restoring 0xf5f320c0-0xf5f320c0=0x6b
> >> > [ 44.482064]
> >> > [ 44.482064] FIX key_jar: Marking all objects used
> >> >
> >> > Config attached. The system seemed to stay intact after this incident.
> >>
> >>
> >> Are you sure this is the right config?
> >>
> >> # Security options
> >> #
> >> # CONFIG_KEYS is not set
> >>
> >> ...sys_keyctl() returns -ENOSYS...!?
> >
> > no, sorry. The right 'bad' config attached.
>
> Are you sure, is this expected?
>
> CC kernel/printk.o
> cc1: warnings being treated as errors
> arch/x86/kernel/cpu/intel_cacheinfo.c: In function 'show_cache_disable':
> arch/x86/kernel/cpu/intel_cacheinfo.c:710: error: unused variable 'mask'
> arch/x86/kernel/cpu/intel_cacheinfo.c: In function 'store_cache_disable':
> arch/x86/kernel/cpu/intel_cacheinfo.c:745: error: unused variable 'mask'
> make[2]: *** [arch/x86/kernel/cpu/intel_cacheinfo.o] Error 1
> make[1]: *** [arch/x86/kernel/cpu] Error 2
> make: *** [arch/x86/kernel] Error 2
> make: *** Waiting for unfinished jobs....
>
> Will restart make with CONFIG_ALLOW_WARNINGS=y...
yes, i had that off via a local patch.
Ingo
Ingo Molnar <[email protected]> wrote:
> -tip testing found the rather scary looking slab corruption:
Unfortunately, I'm not in a position to look into this much right now:-(.
I'll see if I can find a test machine next week, but if not, it'll be Feb
before I can do anything about it.
That said, what sort of testing were you doing?
David
* David Howells <[email protected]> wrote:
> Ingo Molnar <[email protected]> wrote:
>
> > -tip testing found the rather scary looking slab corruption:
>
> Unfortunately, I'm not in a position to look into this much right
> now:-(. I'll see if I can find a test machine next week, but if not,
> it'll be Feb before I can do anything about it.
>
> That said, what sort of testing were you doing?
It was the regular -tip test cycle/iteration: booting the kernel and
building the next new (random) kernel on that system, then booting that
(random) kernel ... etc. The warning occured some time after bootup, so it
probably triggered during the building of the next kernel. (one kernel
build takes about 1-2 minutes to finish)
The problem was not reproducible - i tried the same config once more and
it didnt produce the bug. (this system has no known hardware flukes)
Ingo
Ingo Molnar <[email protected]> wrote:
> The problem was not reproducible - i tried the same config once more and
> it didnt produce the bug. (this system has no known hardware flukes)
Having thought about it some more, we can't necessarily pin the blame on the
key management code. I think it more likely due to the previous owner of the
page as it's the guard poisoning that got corrupted, not the allocated key
struct. The problem is we don't know who that was. I wonder if it's
practical to store a recent history of page allocs and releases in a circular
buffer.
David
On Sat, Jan 17, 2009 at 9:33 PM, David Howells <[email protected]> wrote:> Ingo Molnar <[email protected]> wrote:>>> The problem was not reproducible - i tried the same config once more and>> it didnt produce the bug. (this system has no known hardware flukes)>> Having thought about it some more, we can't necessarily pin the blame on the> key management code. I think it more likely due to the previous owner of the> page as it's the guard poisoning that got corrupted, not the allocated key> struct. The problem is we don't know who that was. I wonder if it's> practical to store a recent history of page allocs and releases in a circular> buffer.
This is what I think:
[ 44.482064] INFO: 0xf5f320c0-0xf5f320c0. First byte 0x6a instead of 0x6b...[ 44.482064] Object 0xf5f320c0: 6a 6b 6b 6b 6b 6b 6b 6b 6b 6b 6b6b 6b 6b 6b 6b jkkkkkkkkkkkkkkk
Only the first byte has been changed. It changed from 0x6b to 0x6a.Smells like a decrement. And yes, this particular cache allocates'struct key's, which has an 'atomic_t usage' as its first member. So arefcounting bug, most likely (key_put() was called too many times). Ora missing key_get() somewhere?
It also seems noteworthy that no other data has been changed since thelast key_put() (i.e. since the refcount hit zero).
[ 44.482064] Bytes b4 0xf5f320b0: 7c 05 ff ff 5a 5a 5a 5a 5a 5a 5a5a 5a 5a 5a 5a |.��ZZZZZZZZZZZZ
...I guess "bytes b4" means "bytes before"?
Vegard
-- "The animistic metaphor of the bug that maliciously sneaked in whilethe programmer was not looking is intellectually dishonest as itdisguises that the error is the programmer's own creation." -- E. W. Dijkstra, EWD1036????{.n?+???????+%?????ݶ??w??{.n?+????{??G?????{ay? ʇڙ?,j??f???h?????????z_??(?階?ݢj"???m??????G????????????&???~???iO???z??v?^?m???? ????????I?