2010-12-07 14:13:19

by Pavel Emelyanov

[permalink] [raw]
Subject: [PATCH] user_ns: Improve the user_ns on-the-slab packaging

Currently on 64-bit arch the user_namespace is 2096 and when
being kmalloc-ed it resides on a 4k slab wasting 2003 bytes.

If we allocate a separate cache for it and reduce the hash size
from 128 to 64 chains the packaging becomes *much* better - the
struct is 1072 bytes and the hole between is 98 bytes.

Signed-off-by: Pavel Emelyanov <[email protected]>

---

diff --git a/include/linux/user_namespace.h b/include/linux/user_namespace.h
index 8178156..faf4679 100644
--- a/include/linux/user_namespace.h
+++ b/include/linux/user_namespace.h
@@ -6,7 +6,7 @@
#include <linux/sched.h>
#include <linux/err.h>

-#define UIDHASH_BITS (CONFIG_BASE_SMALL ? 3 : 8)
+#define UIDHASH_BITS (CONFIG_BASE_SMALL ? 3 : 7)
#define UIDHASH_SZ (1 << UIDHASH_BITS)

struct user_namespace {
diff --git a/kernel/user_namespace.c b/kernel/user_namespace.c
index 2591583..8aafa80 100644
--- a/kernel/user_namespace.c
+++ b/kernel/user_namespace.c
@@ -12,6 +12,8 @@
#include <linux/highuid.h>
#include <linux/cred.h>

+static struct kmem_cache *user_ns_cachep __read_mostly;
+
/*
* Create a new user namespace, deriving the creator from the user in the
* passed credentials, and replacing that user with the new root user for the
@@ -26,7 +28,7 @@ int create_user_ns(struct cred *new)
struct user_struct *root_user;
int n;

- ns = kmalloc(sizeof(struct user_namespace), GFP_KERNEL);
+ ns = kmem_cache_alloc(user_ns_cachep, GFP_KERNEL);
if (!ns)
return -ENOMEM;

@@ -38,7 +40,7 @@ int create_user_ns(struct cred *new)
/* Alloc new root user. */
root_user = alloc_uid(ns, 0);
if (!root_user) {
- kfree(ns);
+ kmem_cache_free(user_ns_cachep, ns);
return -ENOMEM;
}

@@ -71,7 +73,7 @@ static void free_user_ns_work(struct work_struct *work)
struct user_namespace *ns =
container_of(work, struct user_namespace, destroyer);
free_uid(ns->creator);
- kfree(ns);
+ kmem_cache_free(user_ns_cachep, ns);
}

void free_user_ns(struct kref *kref)
@@ -126,3 +128,11 @@ gid_t user_ns_map_gid(struct user_namespace *to, const struct cred *cred, gid_t
/* No useful relationship so no mapping */
return overflowgid;
}
+
+static __init int user_namespaces_init(void)
+{
+ user_ns_cachep = KMEM_CACHE(user_namespace, SLAB_PANIC);
+ return 0;
+}
+
+__initcall(user_namespaces_init);


2010-12-07 14:28:13

by Serge Hallyn

[permalink] [raw]
Subject: Re: [PATCH] user_ns: Improve the user_ns on-the-slab packaging

Quoting Pavel Emelyanov ([email protected]):
> Currently on 64-bit arch the user_namespace is 2096 and when
> being kmalloc-ed it resides on a 4k slab wasting 2003 bytes.
>
> If we allocate a separate cache for it and reduce the hash size
> from 128 to 64 chains the packaging becomes *much* better - the

Hey Pavel,

I trust you've done some performance tests and found no
regressions with a few hundred users?

-serge

2010-12-07 14:33:18

by Pavel Emelyanov

[permalink] [raw]
Subject: Re: [PATCH] user_ns: Improve the user_ns on-the-slab packaging

On 12/07/2010 05:27 PM, Serge E. Hallyn wrote:
> Quoting Pavel Emelyanov ([email protected]):
>> Currently on 64-bit arch the user_namespace is 2096 and when
>> being kmalloc-ed it resides on a 4k slab wasting 2003 bytes.
>>
>> If we allocate a separate cache for it and reduce the hash size
>> from 128 to 64 chains the packaging becomes *much* better - the
>
> Hey Pavel,
>
> I trust you've done some performance tests and found no
> regressions with a few hundred users?

How many hundreds are you interested in? :) 128 users didn't
reveal any regressions.

> -serge
>

2010-12-07 14:50:53

by Serge Hallyn

[permalink] [raw]
Subject: Re: [PATCH] user_ns: Improve the user_ns on-the-slab packaging

Quoting Pavel Emelyanov ([email protected]):
> On 12/07/2010 05:27 PM, Serge E. Hallyn wrote:
> > Quoting Pavel Emelyanov ([email protected]):
> >> Currently on 64-bit arch the user_namespace is 2096 and when
> >> being kmalloc-ed it resides on a 4k slab wasting 2003 bytes.
> >>
> >> If we allocate a separate cache for it and reduce the hash size
> >> from 128 to 64 chains the packaging becomes *much* better - the
> >
> > Hey Pavel,
> >
> > I trust you've done some performance tests and found no
> > regressions with a few hundred users?
>
> How many hundreds are you interested in? :) 128 users didn't
> reveal any regressions.

I have no good guess, would have said 500, 128 sounds good :) So
long as actual benchmarks showed no regression within a 95%
confidence interval.

Thanks for the patch, the memory savings are impressive.

Acked-by: Serge E. Hallyn <[email protected]>

-serge

PS - I'm hoping to send out a version of the targeted capabilities
(based on userns) patchset later this week.

2010-12-07 22:13:20

by Andrew Morton

[permalink] [raw]
Subject: Re: [PATCH] user_ns: Improve the user_ns on-the-slab packaging

On Tue, 07 Dec 2010 17:12:33 +0300
Pavel Emelyanov <[email protected]> wrote:

> @@ -126,3 +128,11 @@ gid_t user_ns_map_gid(struct user_namespace *to, const struct cred *cred, gid_t
> /* No useful relationship so no mapping */
> return overflowgid;
> }
> +
> +static __init int user_namespaces_init(void)
> +{
> + user_ns_cachep = KMEM_CACHE(user_namespace, SLAB_PANIC);
> + return 0;
> +}
> +
> +__initcall(user_namespaces_init);

checkpatch (which you apparently didn't use) says

WARNING: please use device_initcall() instead of __initcall()
#81: FILE: kernel/user_namespace.c:138:
+__initcall(user_namespaces_init);

which was a somewhat random recommendation. I think I'll switch it to
plain old module_init().

Presumably user-namespaces don't get used prior to initcalls being run.