netdev@ - Original thread is at
https://lore.kernel.org/wireguard/[email protected]/
Hi Charles-François,
On Tue, Apr 05, 2022 at 10:21:29PM +0100, Charles-Francois Natali wrote:
> WireGuard currently uses round-robin to dispatch the handling of
> packets, handling them on all online CPUs, including isolated ones
> (isolcpus).
>
> This is unfortunate because it causes significant latency on isolated
> CPUs - see e.g. below over 240 usec:
>
> kworker/47:1-2373323 [047] 243644.756405: funcgraph_entry: |
> process_one_work() { kworker/47:1-2373323 [047] 243644.756406:
> funcgraph_entry: | wg_packet_decrypt_worker() { [...]
> kworker/47:1-2373323 [047] 243644.756647: funcgraph_exit: 0.591 us | }
> kworker/47:1-2373323 [047] 243644.756647: funcgraph_exit: ! 242.655 us
> | }
>
> Instead, restrict to non-isolated CPUs.
Huh, interesting... I haven't seen this feature before. What's the
intended use case? To never run _anything_ on those cores except
processes you choose? To run some things but not intensive things? Is it
sort of a RT-lite?
I took a look in padata/pcrypt and it doesn't look like they're
examining the housekeeping mask at all. Grepping for
housekeeping_cpumask doesn't appear to show many results in things like
workqueues, but rather in core scheduling stuff. So I'm not quite sure
what to make of this patch.
I suspect the thing to do might be to patch both wireguard and padata,
and send a patch series to me, the padata people, and
[email protected], and we can all hash this out together.
Regarding your patch, is there a way to make that a bit more succinct,
without introducing all of those helper functions? It seems awfully
verbose for something that seems like a matter of replacing the online
mask with the housekeeping mask.
Jason
On Fri, 22 Apr 2022 02:02:21 +0200
"Jason A. Donenfeld" <[email protected]> wrote:
> netdev@ - Original thread is at
> https://lore.kernel.org/wireguard/[email protected]/
>
> Hi Charles-François,
>
> On Tue, Apr 05, 2022 at 10:21:29PM +0100, Charles-Francois Natali wrote:
> > WireGuard currently uses round-robin to dispatch the handling of
> > packets, handling them on all online CPUs, including isolated ones
> > (isolcpus).
> >
> > This is unfortunate because it causes significant latency on isolated
> > CPUs - see e.g. below over 240 usec:
> >
> > kworker/47:1-2373323 [047] 243644.756405: funcgraph_entry: |
> > process_one_work() { kworker/47:1-2373323 [047] 243644.756406:
> > funcgraph_entry: | wg_packet_decrypt_worker() { [...]
> > kworker/47:1-2373323 [047] 243644.756647: funcgraph_exit: 0.591 us | }
> > kworker/47:1-2373323 [047] 243644.756647: funcgraph_exit: ! 242.655 us
> > | }
> >
> > Instead, restrict to non-isolated CPUs.
>
> Huh, interesting... I haven't seen this feature before. What's the
> intended use case? To never run _anything_ on those cores except
> processes you choose? To run some things but not intensive things? Is it
> sort of a RT-lite?
>
> I took a look in padata/pcrypt and it doesn't look like they're
> examining the housekeeping mask at all. Grepping for
> housekeeping_cpumask doesn't appear to show many results in things like
> workqueues, but rather in core scheduling stuff. So I'm not quite sure
> what to make of this patch.
>
> I suspect the thing to do might be to patch both wireguard and padata,
> and send a patch series to me, the padata people, and
> [email protected], and we can all hash this out together.
>
> Regarding your patch, is there a way to make that a bit more succinct,
> without introducing all of those helper functions? It seems awfully
> verbose for something that seems like a matter of replacing the online
> mask with the housekeeping mask.
>
> Jason
Applications like DPDK that do polling often use isolcpus or cgroups
to keep unwanted rabble off of their cpus. Having wireguard use those
cpus seems bad.
Hi,
On Fri, 22 Apr 2022 at 01:02, Jason A. Donenfeld <[email protected]> wrote:
>
> netdev@ - Original thread is at
> https://lore.kernel.org/wireguard/[email protected]/
>
> Hi Charles-François,
>
> On Tue, Apr 05, 2022 at 10:21:29PM +0100, Charles-Francois Natali wrote:
> > WireGuard currently uses round-robin to dispatch the handling of
> > packets, handling them on all online CPUs, including isolated ones
> > (isolcpus).
> >
> > This is unfortunate because it causes significant latency on isolated
> > CPUs - see e.g. below over 240 usec:
> >
> > kworker/47:1-2373323 [047] 243644.756405: funcgraph_entry: |
> > process_one_work() { kworker/47:1-2373323 [047] 243644.756406:
> > funcgraph_entry: | wg_packet_decrypt_worker() { [...]
> > kworker/47:1-2373323 [047] 243644.756647: funcgraph_exit: 0.591 us | }
> > kworker/47:1-2373323 [047] 243644.756647: funcgraph_exit: ! 242.655 us
> > | }
> >
> > Instead, restrict to non-isolated CPUs.
>
> Huh, interesting... I haven't seen this feature before. What's the
> intended use case? To never run _anything_ on those cores except
> processes you choose? To run some things but not intensive things? Is it
> sort of a RT-lite?
Yes, the idea is to not run anything on those cores: no user tasks, no unbound
workqueues, etc.
Typically one would also set IRQ affinity etc to avoid those cores, to avoid
(soft)IRQS which cause significant latency as well.
This series by Frederic Weisbecker is a good introduction:
https://www.suse.com/c/cpu-isolation-introduction-part-1/
The idea is to achieve low latency and jitter.
With a reasonably tuned kernel one can reach around 10usec latency - however
whenever we start using wireguard, we can see the bound workqueues used for
round-robin dispatch cause up to 1ms stalls, which is just not
acceptable for us.
Currently our only option is to either patch the wireguard code, or
stop using it,
which would be a shame :).
> I took a look in padata/pcrypt and it doesn't look like they're
> examining the housekeeping mask at all. Grepping for
> housekeeping_cpumask doesn't appear to show many results in things like
> workqueues, but rather in core scheduling stuff. So I'm not quite sure
> what to make of this patch.
Thanks, I didn't know about padata, but after skimming through the code it does
seem that it would suffer from the same issue.
> I suspect the thing to do might be to patch both wireguard and padata,
> and send a patch series to me, the padata people, and
> [email protected], and we can all hash this out together.
Sure, I'll try to have a look at the padata code and write something up.
> Regarding your patch, is there a way to make that a bit more succinct,
> without introducing all of those helper functions? It seems awfully
> verbose for something that seems like a matter of replacing the online
> mask with the housekeeping mask.
Indeed, I wasn't really happy about that.
The reason I've written those helper functions is that the housekeeping mask
includes possible CPUs (cpu_possible_mask), so unfortunately it's not just a
matter of e.g. replacing cpu_online_mask with
housekeeping_cpumask(HK_FLAG_DOMAIN), we have to perform an AND
whenever we compute the weight, find the next CPU in the mask etc.
And I'd rather have the operations and mask in a single location instead of
scattered throughout the code, to make it easier to understand and maintain.
Happy to change to something more inline though, or open to suggestions.
Cheers,
Charles
>
> Jason
Hi Charles,
On Fri, Apr 22, 2022 at 11:23:01PM +0100, Charles-François Natali wrote:
> > Regarding your patch, is there a way to make that a bit more succinct,
> > without introducing all of those helper functions? It seems awfully
> > verbose for something that seems like a matter of replacing the online
> > mask with the housekeeping mask.
>
> Indeed, I wasn't really happy about that.
> The reason I've written those helper functions is that the housekeeping mask
> includes possible CPUs (cpu_possible_mask), so unfortunately it's not just a
> matter of e.g. replacing cpu_online_mask with
> housekeeping_cpumask(HK_FLAG_DOMAIN), we have to perform an AND
> whenever we compute the weight, find the next CPU in the mask etc.
>
> And I'd rather have the operations and mask in a single location instead of
> scattered throughout the code, to make it easier to understand and maintain.
>
> Happy to change to something more inline though, or open to suggestions.
Probably more inlined, yea. A simpler version of your patch would
probably be something like this, right?
diff --git a/drivers/net/wireguard/queueing.h b/drivers/net/wireguard/queueing.h
index 583adb37ee1e..b3117cdd647d 100644
--- a/drivers/net/wireguard/queueing.h
+++ b/drivers/net/wireguard/queueing.h
@@ -112,6 +112,8 @@ static inline int wg_cpumask_choose_online(int *stored_cpu, unsigned int id)
cpu = cpumask_first(cpu_online_mask);
for (i = 0; i < cpu_index; ++i)
cpu = cpumask_next(cpu, cpu_online_mask);
+ while (!housekeeping_test_cpu(cpu, HK_???))
+ cpu = cpumask_next(cpu, cpu_online_mask);
*stored_cpu = cpu;
}
return cpu;
@@ -128,7 +130,7 @@ static inline int wg_cpumask_next_online(int *next)
{
int cpu = *next;
- while (unlikely(!cpumask_test_cpu(cpu, cpu_online_mask)))
+ while (unlikely(!cpumask_test_cpu(cpu, cpu_online_mask) && !housekeeping_test_cpu(cpu, HK_???)))
cpu = cpumask_next(cpu, cpu_online_mask) % nr_cpumask_bits;
*next = cpumask_next(cpu, cpu_online_mask) % nr_cpumask_bits;
return cpu;
However, from looking at kernel/sched/isolation.c a bit, I noticed that
indeed you're right that most of these functions (save one) are based on
cpu_possible_mask rather than cpu_online_mask. This is frustrating
because the code makes smart use of static branches to remain quick, but
ANDing housekeeping_cpumask() with cpu_online_mask would, in the fast
path, wind up ANDing cpu_online_mask with cpu_possible_mask, which is
silly and pointless. That makes me suspect that maybe the best approach
would be adding a relevant helper to kernel/sched/isolation.c, so that
the helper can then do the `if (static_branch_unlikely(&housekeeping_overridden))`
stuff internally.
Or maybe you'll do some measurements and decide that just [ab]using
housekeeping_test_cpu() like above is actually optimal? Not really sure
myself.
Anyway, I'll keep an eye out for your joint wireguard/padata series. Be
sure to CC the people who wrote the isolation & housekeeping code, as
they likely have opinions about this stuff (and certainly know more than
me about it).
Jason