Received-SPF: pass (google.com: domain of linux-kernel-owner@vger.kernel.org designates 2620:137:e000::1:20 as permitted sender) client-ip=2620:137:e000::1:20;
Date:   Mon, 9 Jan 2023 16:12:53 -0800
From:   Luis Chamberlain <mcgrof@kernel.org>
To:     Petr Mladek <pmladek@suse.com>
Cc:     Zhen Lei <thunder.leizhen@huawei.com>,
        linux-kernel@vger.kernel.org, linux-modules@vger.kernel.org,
        Anders Roxell <anders.roxell@linaro.org>
Subject: Re: [PATCH v2] kallsyms: Fix sleeping function called from invalid
 context when CONFIG_KALLSYMS_SELFTEST=y
Message-ID: <Y7ythXxlzsJkFQcd@bombadil.infradead.org>
References: <20221228014511.328-1-thunder.leizhen@huawei.com>
 <Y7wZSxw+Ys5MNf8g@alley>
MIME-Version: 1.0
Content-Type: text/plain; charset=us-ascii
Content-Disposition: inline
In-Reply-To: <Y7wZSxw+Ys5MNf8g@alley>
Sender: Luis Chamberlain <mcgrof@infradead.org>
Precedence: bulk

On Mon, Jan 09, 2023 at 02:40:27PM +0100, Petr Mladek wrote:
> Why are try hardly comparable?
> 
> 1. The speed depends on the number of loaded modules
>    and number of symbols. It highly depends on the configuration
>    that was used to build the kernel.
> 
> 2. The test runs only once. As a result it is hard to judge
>    how big is the noise.
> 
> 3. The noise might depend on the size and state of CPU caches.
> 
> 
> I personally vote for removing this selftest!

Even so, just as with testing a filesystem with different types of
configurations, at least testing a few configs helps and it's what
we do. Then, if anyone ever wanted to try to increase performance
on symbol lookup today they have no easy way to measure things. How
would they go about comparing things performance without this selftest?

This selftests helps generically with that *and* helps peg on to it any sanity
checks you may wish to add to those APIs which we just don't want to do
upstream.

That was the rationale behind it, just as with any other selftest.

However, if measuring the time is not possible that's separate topic.
But to say that measuring time on some config is not valuable I think
is not a fair statement. Yes, the noise things are good points, but
the test can be enhanced for that too. And any patch which anyone
in the future would try to propose for new enhancements would likely
be looking at obvious gains beyond any type of noise.

  Luis