This patch series addresses the following two problems:
1. A customer provided some evidence which indicates that
the idle tick was stopped; albeit, CPU-specific vmstat
counters still remained populated.
Thus one can only assume quiet_vmstat() was not
invoked on return to the idle loop. If I understand
correctly, I suspect this divergence might erroneously
prevent a reclaim attempt by kswapd. If the number of
zone specific free pages are below their per-cpu drift
value then zone_page_state_snapshot() is used to
compute a more accurate view of the aforementioned
statistic. Thus any task blocked on the NUMA node
specific pfmemalloc_wait queue will be unable to make
significant progress via direct reclaim unless it is
killed after being woken up by kswapd
(see throttle_direct_reclaim())
2. With a SCHED_FIFO task that busy loops on a given CPU,
and kworker for that CPU at SCHED_OTHER priority,
queuing work to sync per-vmstats will either cause that
work to never execute, or stalld (i.e. stall daemon)
boosts kworker priority which causes a latency
violation
By having vmstat_shepherd flush the per-CPU counters to the
global counters from remote CPUs.
This is done using cmpxchg to manipulate the counters,
both CPU locally (via the account functions),
and remotely (via cpu_vm_stats_fold).
Thanks to Aaron Tomlin for diagnosing issue 1 and writing
the initial patch series.
v2:
- actually use LOCK CMPXCHG on counter mod/inc/dec functions
(Christoph Lameter)
- use try_cmpxchg for cmpxchg loops
(Uros Bizjak / Matthew Wilcox)
arch/arm64/include/asm/percpu.h | 16 ++-
arch/loongarch/include/asm/percpu.h | 23 ++++
arch/s390/include/asm/percpu.h | 5 +
arch/x86/include/asm/percpu.h | 39 ++++----
include/asm-generic/percpu.h | 17 +++
include/linux/mmzone.h | 3
kernel/fork.c | 2
kernel/scs.c | 2
mm/vmstat.c | 424 ++++++++++++++++++++++++++++++++++++++++++++++++------------------------------------------
9 files changed, 308 insertions(+), 223 deletions(-)
On Thu, Feb 09, 2023 at 12:01:50PM -0300, Marcelo Tosatti wrote:
> This patch series addresses the following two problems:
>
> 1. A customer provided some evidence which indicates that
> the idle tick was stopped; albeit, CPU-specific vmstat
> counters still remained populated.
>
> Thus one can only assume quiet_vmstat() was not
> invoked on return to the idle loop. If I understand
> correctly, I suspect this divergence might erroneously
> prevent a reclaim attempt by kswapd. If the number of
> zone specific free pages are below their per-cpu drift
> value then zone_page_state_snapshot() is used to
> compute a more accurate view of the aforementioned
> statistic. Thus any task blocked on the NUMA node
> specific pfmemalloc_wait queue will be unable to make
> significant progress via direct reclaim unless it is
> killed after being woken up by kswapd
> (see throttle_direct_reclaim())
>
> 2. With a SCHED_FIFO task that busy loops on a given CPU,
> and kworker for that CPU at SCHED_OTHER priority,
> queuing work to sync per-vmstats will either cause that
> work to never execute, or stalld (i.e. stall daemon)
> boosts kworker priority which causes a latency
> violation
>
> By having vmstat_shepherd flush the per-CPU counters to the
> global counters from remote CPUs.
>
> This is done using cmpxchg to manipulate the counters,
> both CPU locally (via the account functions),
> and remotely (via cpu_vm_stats_fold).
>
> Thanks to Aaron Tomlin for diagnosing issue 1 and writing
> the initial patch series.
>
> v2:
> - actually use LOCK CMPXCHG on counter mod/inc/dec functions
> (Christoph Lameter)
> - use try_cmpxchg for cmpxchg loops
> (Uros Bizjak / Matthew Wilcox)
>
>
> arch/arm64/include/asm/percpu.h | 16 ++-
> arch/loongarch/include/asm/percpu.h | 23 ++++
> arch/s390/include/asm/percpu.h | 5 +
> arch/x86/include/asm/percpu.h | 39 ++++----
> include/asm-generic/percpu.h | 17 +++
> include/linux/mmzone.h | 3
> kernel/fork.c | 2
> kernel/scs.c | 2
> mm/vmstat.c | 424 ++++++++++++++++++++++++++++++++++++++++++++++++------------------------------------------
> 9 files changed, 308 insertions(+), 223 deletions(-)
Friendly ping, any other concern with this series?
If not, ACKed-by or Reviewed-by's would be welcome.
On Fri, Feb 24, 2023 at 10:34:10AM +0800, Hillf Danton wrote:
> On Thu, Feb 09, 2023 at 12:01:50PM -0300, Marcelo Tosatti wrote:
> > This patch series addresses the following two problems:
> >
> > 1. A customer provided some evidence which indicates that
> > the idle tick was stopped; albeit, CPU-specific vmstat
> > counters still remained populated.
> >
> > Thus one can only assume quiet_vmstat() was not
> > invoked on return to the idle loop. If I understand
> > correctly, I suspect this divergence might erroneously
> > prevent a reclaim attempt by kswapd. If the number of
> > zone specific free pages are below their per-cpu drift
> > value then zone_page_state_snapshot() is used to
> > compute a more accurate view of the aforementioned
> > statistic. Thus any task blocked on the NUMA node
> > specific pfmemalloc_wait queue will be unable to make
> > significant progress via direct reclaim unless it is
> > killed after being woken up by kswapd
> > (see throttle_direct_reclaim())
> >
> > 2. With a SCHED_FIFO task that busy loops on a given CPU,
> > and kworker for that CPU at SCHED_OTHER priority,
> > queuing work to sync per-vmstats will either cause that
> > work to never execute, or stalld (i.e. stall daemon)
> > boosts kworker priority which causes a latency
> > violation
> >
> > By having vmstat_shepherd flush the per-CPU counters to the
> > global counters from remote CPUs.
> >
> > This is done using cmpxchg to manipulate the counters,
> > both CPU locally (via the account functions),
> > and remotely (via cpu_vm_stats_fold).
>
> Frankly another case of bandaid[1] ?
>
> [1] https://lore.kernel.org/lkml/[email protected]/
Only if you disable per-CPU vmstat counters for isolated CPUs
(then maintenance of the data structures in isolated CPUs is
not necessary).
Which would be terrible for performance, however.