2017-03-28 13:14:45

by Zhou Chengming

[permalink] [raw]
Subject: [PATCH v2] livepatch: Reduce the time of finding module symbols

It's reported that the time of insmoding a klp.ko for one of our
out-tree modules is too long.

~ time sudo insmod klp.ko
real 0m23.799s
user 0m0.036s
sys 0m21.256s

Then we found the reason: our out-tree module used a lot of static local
variables, so klp.ko has a lot of relocation records which reference the
module. Then for each such entry klp_find_object_symbol() is called to
resolve it, but this function uses the interface kallsyms_on_each_symbol()
even for finding module symbols, so will waste a lot of time on walking
through vmlinux kallsyms table many times.

This patch changes it to use module_kallsyms_on_each_symbol() for modules
symbols. After we apply this patch, the sys time reduced dramatically.

~ time sudo insmod klp.ko
real 0m1.007s
user 0m0.032s
sys 0m0.924s

Signed-off-by: Zhou Chengming <[email protected]>
---
kernel/livepatch/core.c | 5 ++++-
1 file changed, 4 insertions(+), 1 deletion(-)

diff --git a/kernel/livepatch/core.c b/kernel/livepatch/core.c
index af46438..b4b8bb0 100644
--- a/kernel/livepatch/core.c
+++ b/kernel/livepatch/core.c
@@ -182,7 +182,10 @@ static int klp_find_object_symbol(const char *objname, const char *name,
};

mutex_lock(&module_mutex);
- kallsyms_on_each_symbol(klp_find_callback, &args);
+ if (objname)
+ module_kallsyms_on_each_symbol(klp_find_callback, &args);
+ else
+ kallsyms_on_each_symbol(klp_find_callback, &args);
mutex_unlock(&module_mutex);

/*
--
1.8.3.1


2017-03-29 14:59:12

by Josh Poimboeuf

[permalink] [raw]
Subject: Re: [PATCH v2] livepatch: Reduce the time of finding module symbols

On Tue, Mar 28, 2017 at 09:10:35PM +0800, Zhou Chengming wrote:
> It's reported that the time of insmoding a klp.ko for one of our
> out-tree modules is too long.
>
> ~ time sudo insmod klp.ko
> real 0m23.799s
> user 0m0.036s
> sys 0m21.256s
>
> Then we found the reason: our out-tree module used a lot of static local
> variables, so klp.ko has a lot of relocation records which reference the
> module. Then for each such entry klp_find_object_symbol() is called to
> resolve it, but this function uses the interface kallsyms_on_each_symbol()
> even for finding module symbols, so will waste a lot of time on walking
> through vmlinux kallsyms table many times.
>
> This patch changes it to use module_kallsyms_on_each_symbol() for modules
> symbols. After we apply this patch, the sys time reduced dramatically.
>
> ~ time sudo insmod klp.ko
> real 0m1.007s
> user 0m0.032s
> sys 0m0.924s
>
> Signed-off-by: Zhou Chengming <[email protected]>

It would be good to make kallsyms faster, but either way this still a
performance improvement for patching modules.

Acked-by: Josh Poimboeuf <[email protected]>

--
Josh

2017-03-29 19:40:49

by Jessica Yu

[permalink] [raw]
Subject: Re: [PATCH v2] livepatch: Reduce the time of finding module symbols

+++ Zhou Chengming [28/03/17 21:10 +0800]:
>It's reported that the time of insmoding a klp.ko for one of our
>out-tree modules is too long.
>
>~ time sudo insmod klp.ko
>real 0m23.799s
>user 0m0.036s
>sys 0m21.256s
>
>Then we found the reason: our out-tree module used a lot of static local
>variables, so klp.ko has a lot of relocation records which reference the
>module. Then for each such entry klp_find_object_symbol() is called to
>resolve it, but this function uses the interface kallsyms_on_each_symbol()
>even for finding module symbols, so will waste a lot of time on walking
>through vmlinux kallsyms table many times.
>
>This patch changes it to use module_kallsyms_on_each_symbol() for modules
>symbols. After we apply this patch, the sys time reduced dramatically.
>
>~ time sudo insmod klp.ko
>real 0m1.007s
>user 0m0.032s
>sys 0m0.924s
>
>Signed-off-by: Zhou Chengming <[email protected]>

Limiting the search space for symbol lookups is a valid improvement, so:

Acked-by: Jessica Yu <[email protected]>

>---
> kernel/livepatch/core.c | 5 ++++-
> 1 file changed, 4 insertions(+), 1 deletion(-)
>
>diff --git a/kernel/livepatch/core.c b/kernel/livepatch/core.c
>index af46438..b4b8bb0 100644
>--- a/kernel/livepatch/core.c
>+++ b/kernel/livepatch/core.c
>@@ -182,7 +182,10 @@ static int klp_find_object_symbol(const char *objname, const char *name,
> };
>
> mutex_lock(&module_mutex);
>- kallsyms_on_each_symbol(klp_find_callback, &args);
>+ if (objname)
>+ module_kallsyms_on_each_symbol(klp_find_callback, &args);
>+ else
>+ kallsyms_on_each_symbol(klp_find_callback, &args);
> mutex_unlock(&module_mutex);
>
> /*
>--
>1.8.3.1
>
>--
>To unsubscribe from this list: send the line "unsubscribe live-patching" in
>the body of a message to [email protected]
>More majordomo info at http://vger.kernel.org/majordomo-info.html

2017-03-30 08:02:00

by Miroslav Benes

[permalink] [raw]
Subject: Re: [PATCH v2] livepatch: Reduce the time of finding module symbols

On Tue, 28 Mar 2017, Zhou Chengming wrote:

> It's reported that the time of insmoding a klp.ko for one of our
> out-tree modules is too long.
>
> ~ time sudo insmod klp.ko
> real 0m23.799s
> user 0m0.036s
> sys 0m21.256s
>
> Then we found the reason: our out-tree module used a lot of static local
> variables, so klp.ko has a lot of relocation records which reference the
> module. Then for each such entry klp_find_object_symbol() is called to
> resolve it, but this function uses the interface kallsyms_on_each_symbol()
> even for finding module symbols, so will waste a lot of time on walking
> through vmlinux kallsyms table many times.
>
> This patch changes it to use module_kallsyms_on_each_symbol() for modules
> symbols. After we apply this patch, the sys time reduced dramatically.
>
> ~ time sudo insmod klp.ko
> real 0m1.007s
> user 0m0.032s
> sys 0m0.924s
>
> Signed-off-by: Zhou Chengming <[email protected]>

We are the only user of kallsyms_on_each_symbol() interface right now, so
it is not that bad to optimize here. Temporarily :)

Acked-by: Miroslav Benes <[email protected]>

Miroslav

2017-03-30 08:42:44

by Jiri Kosina

[permalink] [raw]
Subject: Re: [PATCH v2] livepatch: Reduce the time of finding module symbols

On Tue, 28 Mar 2017, Zhou Chengming wrote:

> It's reported that the time of insmoding a klp.ko for one of our
> out-tree modules is too long.

Applied to for-4.12/upstream. Thanks,

--
Jiri Kosina
SUSE Labs