2008-03-17 23:30:44

by Jan Engelhardt

[permalink] [raw]
Subject: vfree with spin_lock_bh

Hi,


while transforming some code with big allocations (like 120 KB) from
kmalloc to vmalloc — virtual contiguity is sufficient — I hit a
BUG_ON in mm/vmalloc.c a number of times:

void vfree(const void *addr)
{
BUG_ON(in_interrupt());
__vunmap(addr, 1);
}

First I was thinking “how could iptables -F run in interrupt context?”,
but apparently, it does seem to make a difference:

...
spin_lock_bh(&a_local_spinlock);
list_del_rcu(&node->list);
printk(KERN_INFO "Interrupt? %lu\n", in_interrupt());
/* vfree not worky here */
spin_unlock_bh(&a_local_spinlock);
printk(KERN_INFO "Interrupt? %lu\n", in_interrupt());
/* now possible */
vfree(node);
...

and this gives (x86_32)

Interrupt? 256
Interrupt? 0

So this may be a "property" of spinlocks, but it is a bit strange to me.
Why should not I be able to call vfree() when I am, in fact, in
user context (but with a bh spinlock held...).

Do I perhaps need a non-bh spinlock? There's RCU going on on that
linked list so I am not sure whether I could just call the normal
spin_lock() function.

Looking at the code of _spin_lock_bh in kernel/spinlock.c reveals that
it is actually disabling preempt instead of being in an interrupt.
Making an uneducated guess, would

BUG_ON(in_interrupt() != 0 && in_interrupt() != 256)

in vfree() be safe?


thanks,
Jan


2008-03-18 11:24:16

by Johannes Weiner

[permalink] [raw]
Subject: Re: vfree with spin_lock_bh

Hi Jan,

Jan Engelhardt <[email protected]> writes:

> while transforming some code with big allocations (like 120 KB) from
> kmalloc to vmalloc — virtual contiguity is sufficient — I hit a
> BUG_ON in mm/vmalloc.c a number of times:
>
> void vfree(const void *addr)
> {
> BUG_ON(in_interrupt());
> __vunmap(addr, 1);
> }
>
> First I was thinking “how could iptables -F run in interrupt context?”,
> but apparently, it does seem to make a difference:
>
> ...
> spin_lock_bh(&a_local_spinlock);
> list_del_rcu(&node->list);
> printk(KERN_INFO "Interrupt? %lu\n", in_interrupt());
> /* vfree not worky here */
> spin_unlock_bh(&a_local_spinlock);
> printk(KERN_INFO "Interrupt? %lu\n", in_interrupt());
> /* now possible */
> vfree(node);
> ...
>
> and this gives (x86_32)
>
> Interrupt? 256
> Interrupt? 0
>
> So this may be a "property" of spinlocks, but it is a bit strange to me.
> Why should not I be able to call vfree() when I am, in fact, in
> user context (but with a bh spinlock held...).

in_interrupt() checks for both, hard- and softirqs. Since
spin_lock_bh() disables softirq's you have to be as fast as possible to
avoid softirq latency.

> Do I perhaps need a non-bh spinlock? There's RCU going on on that
> linked list so I am not sure whether I could just call the normal
> spin_lock() function.

Perhaps a call_rcu() which vfree()s the node? But I am just guessing
wildly here.

> Looking at the code of _spin_lock_bh in kernel/spinlock.c reveals that
> it is actually disabling preempt instead of being in an interrupt.
> Making an uneducated guess, would
>
> BUG_ON(in_interrupt() != 0 && in_interrupt() != 256)

That's basically a lacky in_irq(). The 256 you see here is
1<<SOFTIRQ_SHIFT but softirq disabling can be nested and you can not
check for 256 directly.

> in vfree() be safe?

Can not judge that.

Hannes