Avoid calling kfree() from under pidmap spinlock and reduces code size
by 16 bytes on gcc 4.4.1 on Core 2.
André Goddard Rosa (2):
pid: tighten pidmap spinlock critical section by removing kfree()
pid: reduce code size by using a pointer to iterate over array
kernel/pid.c | 12 ++++++------
1 files changed, 6 insertions(+), 6 deletions(-)
Avoid calling kfree() under pidmap spinlock, calling it afterwards.
Normally kfree() is very fast, but sometimes it can be slow, so avoid
calling it under the spinlock if we can.
Signed-off-by: André Goddard Rosa <[email protected]>
cc: Pekka Enberg <[email protected]>
---
kernel/pid.c | 7 ++++---
1 files changed, 4 insertions(+), 3 deletions(-)
diff --git a/kernel/pid.c b/kernel/pid.c
index d3f722d..55fd590 100644
--- a/kernel/pid.c
+++ b/kernel/pid.c
@@ -141,11 +141,12 @@ static int alloc_pidmap(struct pid_namespace *pid_ns)
* installing it:
*/
spin_lock_irq(&pidmap_lock);
- if (map->page)
- kfree(page);
- else
+ if (!map->page) {
map->page = page;
+ page = NULL;
+ }
spin_unlock_irq(&pidmap_lock);
+ kfree(page);
if (unlikely(!map->page))
break;
}
--
1.6.5.3.298.g39add
It decreases code size by 16 bytes on my gcc 4.4.1 on Core 2:
text data bss dec hex filename
4314 2216 8 6538 198a kernel/pid.o-BEFORE
4298 2216 8 6522 197a kernel/pid.o-AFTER
Signed-off-by: André Goddard Rosa <[email protected]>
---
kernel/pid.c | 5 ++---
1 files changed, 2 insertions(+), 3 deletions(-)
diff --git a/kernel/pid.c b/kernel/pid.c
index 55fd590..2e17c9c 100644
--- a/kernel/pid.c
+++ b/kernel/pid.c
@@ -269,12 +269,11 @@ struct pid *alloc_pid(struct pid_namespace *ns)
for (type = 0; type < PIDTYPE_MAX; ++type)
INIT_HLIST_HEAD(&pid->tasks[type]);
+ upid = pid->numbers + ns->level;
spin_lock_irq(&pidmap_lock);
- for (i = ns->level; i >= 0; i--) {
- upid = &pid->numbers[i];
+ for ( ; upid >= pid->numbers; --upid)
hlist_add_head_rcu(&upid->pid_chain,
&pid_hash[pid_hashfn(upid->nr, upid->ns)]);
- }
spin_unlock_irq(&pidmap_lock);
out:
--
1.6.5.3.298.g39add
(Adding some CC's.)
On Sat, Nov 21, 2009 at 2:16 PM, Andr? Goddard Rosa
<[email protected]> wrote:
> Avoid calling kfree() under pidmap spinlock, calling it afterwards.
>
> Normally kfree() is very fast, but sometimes it can be slow, so avoid
> calling it under the spinlock if we can.
>
> Signed-off-by: Andr? Goddard Rosa <[email protected]>
> cc: Pekka Enberg <[email protected]>
> ---
> ?kernel/pid.c | ? ?7 ++++---
> ?1 files changed, 4 insertions(+), 3 deletions(-)
>
> diff --git a/kernel/pid.c b/kernel/pid.c
> index d3f722d..55fd590 100644
> --- a/kernel/pid.c
> +++ b/kernel/pid.c
> @@ -141,11 +141,12 @@ static int alloc_pidmap(struct pid_namespace *pid_ns)
> ? ? ? ? ? ? ? ? ? ? ? ? * installing it:
> ? ? ? ? ? ? ? ? ? ? ? ? */
> ? ? ? ? ? ? ? ? ? ? ? ?spin_lock_irq(&pidmap_lock);
> - ? ? ? ? ? ? ? ? ? ? ? if (map->page)
> - ? ? ? ? ? ? ? ? ? ? ? ? ? ? ? kfree(page);
> - ? ? ? ? ? ? ? ? ? ? ? else
> + ? ? ? ? ? ? ? ? ? ? ? if (!map->page) {
> ? ? ? ? ? ? ? ? ? ? ? ? ? ? ? ?map->page = page;
> + ? ? ? ? ? ? ? ? ? ? ? ? ? ? ? page = NULL;
> + ? ? ? ? ? ? ? ? ? ? ? }
> ? ? ? ? ? ? ? ? ? ? ? ?spin_unlock_irq(&pidmap_lock);
> + ? ? ? ? ? ? ? ? ? ? ? kfree(page);
> ? ? ? ? ? ? ? ? ? ? ? ?if (unlikely(!map->page))
> ? ? ? ? ? ? ? ? ? ? ? ? ? ? ? ?break;
> ? ? ? ? ? ? ? ?}
> --
> 1.6.5.3.298.g39add
>
> --
> To unsubscribe from this list: send the line "unsubscribe linux-kernel" in
> the body of a message to [email protected]
> More majordomo info at ?http://vger.kernel.org/majordomo-info.html
> Please read the FAQ at ?http://www.tux.org/lkml/
>
On 11/23, Pekka Enberg wrote:
> (Adding some CC's.)
>
> On Sat, Nov 21, 2009 at 2:16 PM, André Goddard Rosa
> <[email protected]> wrote:
> > Avoid calling kfree() under pidmap spinlock, calling it afterwards.
> >
> > Normally kfree() is very fast, but sometimes it can be slow, so avoid
> > calling it under the spinlock if we can.
kfree() is called when we race with another process which also
finds map->page == NULL, allocs the new page and takes pidmap_lock
before us. This is extremely unlikely case, right?
> > @@ -141,11 +141,12 @@ static int alloc_pidmap(struct pid_namespace *pid_ns)
> > * installing it:
> > */
> > spin_lock_irq(&pidmap_lock);
> > - if (map->page)
> > - kfree(page);
> > - else
> > + if (!map->page) {
> > map->page = page;
> > + page = NULL;
> > + }
> > spin_unlock_irq(&pidmap_lock);
> > + kfree(page);
And this change pessimizes (a little bit) the likely case, when
the race doesn't happen. And imho this change doesn't make the
code more readable.
But this is subjective, and technically the patch is correct
afaics.
> > if (unlikely(!map->page))
> > �
Hmm. Off-topic, but why alloc_pidmap() does not do this right
after kzalloc() ?
Oleg.
Hi, Oleg!
On Mon, Nov 23, 2009 at 12:03 PM, Oleg Nesterov <[email protected]> wrote:
> On 11/23, Pekka Enberg wrote:
>> (Adding some CC's.)
>>
>> On Sat, Nov 21, 2009 at 2:16 PM, André Goddard Rosa
>> <[email protected]> wrote:
>> > Avoid calling kfree() under pidmap spinlock, calling it afterwards.
>> >
>> > Normally kfree() is very fast, but sometimes it can be slow, so avoid
>> > calling it under the spinlock if we can.
>
> kfree() is called when we race with another process which also
> finds map->page == NULL, allocs the new page and takes pidmap_lock
> before us. This is extremely unlikely case, right?
Right, somehow.
>> > @@ -141,11 +141,12 @@ static int alloc_pidmap(struct pid_namespace *pid_ns)
>> > * installing it:
>> > */
>> > spin_lock_irq(&pidmap_lock);
>> > - if (map->page)
>> > - kfree(page);
>> > - else
>> > + if (!map->page) {
>> > map->page = page;
>> > + page = NULL;
>> > + }
>> > spin_unlock_irq(&pidmap_lock);
>> > + kfree(page);
>
> And this change pessimizes (a little bit) the likely case, when
> the race doesn't happen. And imho this change doesn't make the
> code more readable.
>
> But this is subjective, and technically the patch is correct
> afaics.
It does not affect the likely case which happens when the pidmap is
already allocated.
In the unlikely case where the pidmap must be allocated, if we think
that we could have
let's say 8 processes contending for that spinlock, while one process
got it first and allocated
the page, having the kfree() out of the spinlock would make those
other 7 processes doing
useful work (performing the release of the page) before, because it
would avoid all of them
spinning around waiting until the all the others also free their
allocated pages.
>> > if (unlikely(!map->page))
>> > �
>
> Hmm. Off-topic, but why alloc_pidmap() does not do this right
> after kzalloc() ?
Hmm... I would say that it's an optimistic best effort. We avoid
failing right away
hoping that another process (racing) had success allocating the page.
That is unlikely! :)
Thank you,
André
On Thu, 3 Dec 2009 12:53:37 -0200
Andr__ Goddard Rosa <[email protected]> wrote:
> Avoid calling kfree() under pidmap spinlock, calling it afterwards.
>
> Normally kfree() is fast, but sometimes it can be slow, so avoid
> calling it under the spinlock if we can do it.
>
> Signed-off-by: Andr__ Goddard Rosa <[email protected]>
> cc: Pekka Enberg <[email protected]>
> ---
> kernel/pid.c | 7 ++++---
> 1 files changed, 4 insertions(+), 3 deletions(-)
>
> diff --git a/kernel/pid.c b/kernel/pid.c
> index d3f722d..55fd590 100644
> --- a/kernel/pid.c
> +++ b/kernel/pid.c
> @@ -141,11 +141,12 @@ static int alloc_pidmap(struct pid_namespace *pid_ns)
> * installing it:
> */
> spin_lock_irq(&pidmap_lock);
> - if (map->page)
> - kfree(page);
> - else
> + if (!map->page) {
> map->page = page;
> + page = NULL;
> + }
> spin_unlock_irq(&pidmap_lock);
> + kfree(page);
> if (unlikely(!map->page))
> break;
> }
um, OK, but the chances of that kfree() actually being executed are
very small.