Received-SPF: pass (google.com: domain of linux-kernel-owner@vger.kernel.org designates 23.128.96.18 as permitted sender) client-ip=23.128.96.18;
MIME-Version: 1.0
References: <20210419184455.2987243-1-lrizzo@google.com> <20210419191712.GB26214@worktop.programming.kicks-ass.net>
In-Reply-To: <20210419191712.GB26214@worktop.programming.kicks-ass.net>
From:   Luigi Rizzo <lrizzo@google.com>
Date:   Mon, 19 Apr 2021 23:07:08 +0200
Message-ID: <CAMOZA0JxwQFfA=acfr255xcR85ztv-URwaQQFHR6RrLk-cPbvg@mail.gmail.com>
Subject: Re: [PATCH] smp: add a best_effort version of smp_call_function_many()
To:     Peter Zijlstra <peterz@infradead.org>
Cc:     linux-kernel <linux-kernel@vger.kernel.org>, axboe@kernel.dk,
        paulmck@kernel.org
Content-Type: text/plain; charset="UTF-8"
Precedence: bulk

On Mon, Apr 19, 2021 at 9:17 PM Peter Zijlstra <peterz@infradead.org> wrote:
>
> On Mon, Apr 19, 2021 at 11:44:55AM -0700, Luigi Rizzo wrote:
> > Regardless of the 'wait' argument, smp_call_function_many() must spin
> > if any of the target CPUs have their csd busy waiting to be processed
> > for a previous call. This may cause high tail latencies e.g. when some
> > of the target CPUs are running functions that disable interrupts for a
> > long time; getrusage() is one possible culprit.
> >
> > Here we introduce a variant, __smp_call_function_many(), that adds
> > a third 'best_effort' mode to the two existing ones (nowait, wait).
> > In best effort mode, the call will skip CPUs whose csd is busy, and if
> > any CPU is skipped it returns -EBUSY and the set of busy in the mask.
> > This allows the caller to decide how to proceed, e.g. it might retry at
> > a later time, or use a private csd, etc..
> >
> > The new function is a compromise to avoid touching existing callers of
> > smp_call_function_many(). If the feature is considered interesting, we
> > could even replace the 'wait' argument with a ternary 'mode' in all
> > smp_call_function_*() and derived methods.
>
> I don't see a user of this...

This is actually something for which I was looking for feedback:

my use case is similar to a periodic garbage collect request:
the caller tells targets that it may be time to do some work,
but it does not matter if the request is dropped because the
caller knows who was busy and will reissue pending requests later.

I would expect something like the above could be useful e.g.
in various kinds of resource manager.

However, a grep for on_each_cpu_*() and smp_call_function_*()
mostly returns synchronous calls (wait=1).

Any possible candidates that people can think of ?

thanks
luigi