Currently the memcg's status is also accounted into the socket's
memory pressure to alleviate the memcg's memstall. But there are
still cases that can be improved. Please check the patches for
detailed info.
Tested on Intel Xeon(R) Platinum 8260, a dual socket machine
containing 2 NUMA nodes each of which has 24C/48T. All the benchmarks
are done inside a separate 5-level depth memcg in a clean host.
Below shows the result of tbench4 and netperf:
tbench4 Throughput (misleading but traditional)
baseline patchset
Hmean 1 357.14 ( 0.00%) 360.31 * 0.89%*
Hmean 2 716.66 ( 0.00%) 724.57 * 1.10%*
Hmean 4 1408.82 ( 0.00%) 1424.31 * 1.10%*
Hmean 8 2826.02 ( 0.00%) 2832.64 * 0.23%*
Hmean 16 5413.68 ( 0.00%) 5347.72 * -1.22%*
Hmean 32 8692.74 ( 0.00%) 8684.26 ( -0.10%)
Hmean 64 10180.12 ( 0.00%) 10377.41 * 1.94%*
Hmean 128 22905.53 ( 0.00%) 22959.73 * 0.24%*
Hmean 256 22935.78 ( 0.00%) 23103.81 * 0.73%*
Hmean 384 22605.36 ( 0.00%) 22747.53 * 0.63%*
netperf-udp
baseline patchset
Hmean send-64 278.42 ( 0.00%) 277.05 ( -0.49%)
Hmean send-128 552.18 ( 0.00%) 553.51 ( 0.24%)
Hmean send-256 1096.38 ( 0.00%) 1095.84 ( -0.05%)
Hmean send-1024 4102.79 ( 0.00%) 4086.06 ( -0.41%)
Hmean send-2048 7727.20 ( 0.00%) 7769.95 ( 0.55%)
Hmean send-3312 11927.57 ( 0.00%) 11966.36 ( 0.33%)
Hmean send-4096 14218.54 ( 0.00%) 14193.51 ( -0.18%)
Hmean send-8192 23903.60 ( 0.00%) 24205.35 * 1.26%*
Hmean send-16384 39600.11 ( 0.00%) 39372.47 ( -0.57%)
Hmean recv-64 278.42 ( 0.00%) 277.05 ( -0.49%)
Hmean recv-128 552.18 ( 0.00%) 553.51 ( 0.24%)
Hmean recv-256 1096.38 ( 0.00%) 1095.84 ( -0.05%)
Hmean recv-1024 4102.79 ( 0.00%) 4086.06 ( -0.41%)
Hmean recv-2048 7727.19 ( 0.00%) 7769.94 ( 0.55%)
Hmean recv-3312 11927.57 ( 0.00%) 11966.36 ( 0.33%)
Hmean recv-4096 14218.45 ( 0.00%) 14193.50 ( -0.18%)
Hmean recv-8192 23903.45 ( 0.00%) 24205.21 * 1.26%*
Hmean recv-16384 39599.53 ( 0.00%) 39372.28 ( -0.57%)
netperf-tcp
baseline patchset
Hmean 64 1756.32 ( 0.00%) 1808.43 * 2.97%*
Hmean 128 3393.47 ( 0.00%) 3421.99 * 0.84%*
Hmean 256 6464.04 ( 0.00%) 6459.72 ( -0.07%)
Hmean 1024 19050.99 ( 0.00%) 19036.21 ( -0.08%)
Hmean 2048 26107.88 ( 0.00%) 26185.44 ( 0.30%)
Hmean 3312 30770.77 ( 0.00%) 30834.78 ( 0.21%)
Hmean 4096 32523.50 ( 0.00%) 32609.77 ( 0.27%)
Hmean 8192 40180.74 ( 0.00%) 39632.41 * -1.36%*
Hmean 16384 46117.02 ( 0.00%) 46259.69 ( 0.31%)
Seems no obvious regression.
v5:
- As Paolo pointed out, the cleanup paired with the patch that
removed in v4 should also be removed.
v4:
- Per Shakeel's suggestion, removed the patch that suppresses
allocation under net-memcg pressure to avoid further keeping
the senders waiting if SACKed segments get dropped from the
OFO queue.
v3:
- Fixed some coding style issues pointed out by Simon
- Fold dependency into memcg pressure func to improve readability
v2:
- Splited into several patches and modified commit log for
better readability.
- Make memcg's pressure consideration function-wide in
__sk_mem_raise_allocated().
v1: https://lore.kernel.org/lkml/[email protected]/
v2: https://lore.kernel.org/lkml/[email protected]/
v3: https://lore.kernel.org/lkml/[email protected]/
v4: https://lore.kernel.org/lkml/[email protected]/
Abel Wu (3):
net-memcg: Fold dependency into memcg pressure cond
sock: Always take memcg pressure into consideration
sock: Fix misuse of sk_under_memory_pressure()
include/linux/memcontrol.h | 2 ++
include/net/sock.h | 14 ++++++++------
include/net/tcp.h | 3 +--
net/core/sock.c | 2 +-
4 files changed, 12 insertions(+), 9 deletions(-)
--
2.37.3
The sk_under_memory_pressure() is called to check whether there is
memory pressure related to this socket. But now it ignores the net-
memcg's pressure if the proto of the socket doesn't care about the
global pressure, which may put burden on its memcg compaction or
reclaim path (also remember that socket memory is un-reclaimable).
So always check the memcg's vm status to alleviate memstalls when
it's in pressure.
Signed-off-by: Abel Wu <[email protected]>
---
include/net/sock.h | 6 ++----
1 file changed, 2 insertions(+), 4 deletions(-)
diff --git a/include/net/sock.h b/include/net/sock.h
index 3f63253ee092..ad1895ffbc4a 100644
--- a/include/net/sock.h
+++ b/include/net/sock.h
@@ -1411,13 +1411,11 @@ static inline bool sk_has_memory_pressure(const struct sock *sk)
static inline bool sk_under_memory_pressure(const struct sock *sk)
{
- if (!sk->sk_prot->memory_pressure)
- return false;
-
if (mem_cgroup_under_socket_pressure(sk->sk_memcg))
return true;
- return !!*sk->sk_prot->memory_pressure;
+ return sk->sk_prot->memory_pressure &&
+ *sk->sk_prot->memory_pressure;
}
static inline long
--
2.37.3
The callers of mem_cgroup_under_socket_pressure() should always make
sure that (mem_cgroup_sockets_enabled && sk->sk_memcg) is true. So
instead of coding around all the callsites, put the dependencies into
mem_cgroup_under_socket_pressure() to avoid redundancy and possibly
bugs.
This change might also introduce slight function call overhead *iff*
the function gets expanded in the future. But for now this change
doesn't make binaries different (checked by vimdiff) except the one
net/ipv4/tcp_input.o (by scripts/bloat-o-meter), which is probably
negligible to performance:
add/remove: 0/0 grow/shrink: 1/2 up/down: 5/-5 (0)
Function old new delta
tcp_grow_window 573 578 +5
tcp_try_rmem_schedule 1083 1081 -2
tcp_check_space 324 321 -3
Total: Before=44647, After=44647, chg +0.00%
So folding the dependencies into mem_cgroup_under_socket_pressure()
is generally a good thing and provides better readablility.
Signed-off-by: Abel Wu <[email protected]>
---
include/linux/memcontrol.h | 2 ++
include/net/sock.h | 3 +--
include/net/tcp.h | 3 +--
3 files changed, 4 insertions(+), 4 deletions(-)
diff --git a/include/linux/memcontrol.h b/include/linux/memcontrol.h
index 222d7370134c..a1aead140ff8 100644
--- a/include/linux/memcontrol.h
+++ b/include/linux/memcontrol.h
@@ -1743,6 +1743,8 @@ void mem_cgroup_sk_alloc(struct sock *sk);
void mem_cgroup_sk_free(struct sock *sk);
static inline bool mem_cgroup_under_socket_pressure(struct mem_cgroup *memcg)
{
+ if (!mem_cgroup_sockets_enabled || !memcg)
+ return false;
if (!cgroup_subsys_on_dfl(memory_cgrp_subsys) && memcg->tcpmem_pressure)
return true;
do {
diff --git a/include/net/sock.h b/include/net/sock.h
index 656ea89f60ff..3f63253ee092 100644
--- a/include/net/sock.h
+++ b/include/net/sock.h
@@ -1414,8 +1414,7 @@ static inline bool sk_under_memory_pressure(const struct sock *sk)
if (!sk->sk_prot->memory_pressure)
return false;
- if (mem_cgroup_sockets_enabled && sk->sk_memcg &&
- mem_cgroup_under_socket_pressure(sk->sk_memcg))
+ if (mem_cgroup_under_socket_pressure(sk->sk_memcg))
return true;
return !!*sk->sk_prot->memory_pressure;
diff --git a/include/net/tcp.h b/include/net/tcp.h
index 14fa716cac50..d4c358bc0c52 100644
--- a/include/net/tcp.h
+++ b/include/net/tcp.h
@@ -259,8 +259,7 @@ extern unsigned long tcp_memory_pressure;
/* optimized version of sk_under_memory_pressure() for TCP sockets */
static inline bool tcp_under_memory_pressure(const struct sock *sk)
{
- if (mem_cgroup_sockets_enabled && sk->sk_memcg &&
- mem_cgroup_under_socket_pressure(sk->sk_memcg))
+ if (mem_cgroup_under_socket_pressure(sk->sk_memcg))
return true;
return READ_ONCE(tcp_memory_pressure);
--
2.37.3
The status of global socket memory pressure is updated when:
a) __sk_mem_raise_allocated():
enter: sk_memory_allocated(sk) > sysctl_mem[1]
leave: sk_memory_allocated(sk) <= sysctl_mem[0]
b) __sk_mem_reduce_allocated():
leave: sk_under_memory_pressure(sk) &&
sk_memory_allocated(sk) < sysctl_mem[0]
So the conditions of leaving global pressure are inconstant, which
may lead to the situation that one pressured net-memcg prevents the
global pressure from being cleared when there is indeed no global
pressure, thus the global constrains are still in effect unexpectedly
on the other sockets.
This patch fixes this by ignoring the net-memcg's pressure when
deciding whether should leave global memory pressure.
Fixes: e1aab161e013 ("socket: initial cgroup code.")
Signed-off-by: Abel Wu <[email protected]>
---
include/net/sock.h | 9 +++++++--
net/core/sock.c | 2 +-
2 files changed, 8 insertions(+), 3 deletions(-)
diff --git a/include/net/sock.h b/include/net/sock.h
index ad1895ffbc4a..22695f776e76 100644
--- a/include/net/sock.h
+++ b/include/net/sock.h
@@ -1409,13 +1409,18 @@ static inline bool sk_has_memory_pressure(const struct sock *sk)
return sk->sk_prot->memory_pressure != NULL;
}
+static inline bool sk_under_global_memory_pressure(const struct sock *sk)
+{
+ return sk->sk_prot->memory_pressure &&
+ *sk->sk_prot->memory_pressure;
+}
+
static inline bool sk_under_memory_pressure(const struct sock *sk)
{
if (mem_cgroup_under_socket_pressure(sk->sk_memcg))
return true;
- return sk->sk_prot->memory_pressure &&
- *sk->sk_prot->memory_pressure;
+ return sk_under_global_memory_pressure(sk);
}
static inline long
diff --git a/net/core/sock.c b/net/core/sock.c
index 5440e67bcfe3..801df091e37a 100644
--- a/net/core/sock.c
+++ b/net/core/sock.c
@@ -3095,7 +3095,7 @@ void __sk_mem_reduce_allocated(struct sock *sk, int amount)
if (mem_cgroup_sockets_enabled && sk->sk_memcg)
mem_cgroup_uncharge_skmem(sk->sk_memcg, amount);
- if (sk_under_memory_pressure(sk) &&
+ if (sk_under_global_memory_pressure(sk) &&
(sk_memory_allocated(sk) < sk_prot_mem_limits(sk, 0)))
sk_leave_memory_pressure(sk);
}
--
2.37.3
On Fri, Jun 02, 2023 at 04:11:33PM +0800, Abel Wu wrote:
> The callers of mem_cgroup_under_socket_pressure() should always make
> sure that (mem_cgroup_sockets_enabled && sk->sk_memcg) is true. So
> instead of coding around all the callsites, put the dependencies into
> mem_cgroup_under_socket_pressure() to avoid redundancy and possibly
> bugs.
>
> This change might also introduce slight function call overhead *iff*
> the function gets expanded in the future. But for now this change
> doesn't make binaries different (checked by vimdiff) except the one
> net/ipv4/tcp_input.o (by scripts/bloat-o-meter), which is probably
> negligible to performance:
>
> add/remove: 0/0 grow/shrink: 1/2 up/down: 5/-5 (0)
> Function old new delta
> tcp_grow_window 573 578 +5
> tcp_try_rmem_schedule 1083 1081 -2
> tcp_check_space 324 321 -3
> Total: Before=44647, After=44647, chg +0.00%
>
> So folding the dependencies into mem_cgroup_under_socket_pressure()
> is generally a good thing and provides better readablility.
>
I don't see how it is improving readability. If you have removed the use
of mem_cgroup_sockets_enabled completely from the networking then I can
understand but this change IMHO will actually decrease the readability
because the later readers will have to reason why we are doing this
check at some places but not other.
+Willem
On Fri, Jun 02, 2023 at 04:11:34PM +0800, Abel Wu wrote:
> The sk_under_memory_pressure() is called to check whether there is
> memory pressure related to this socket. But now it ignores the net-
> memcg's pressure if the proto of the socket doesn't care about the
> global pressure, which may put burden on its memcg compaction or
> reclaim path (also remember that socket memory is un-reclaimable).
>
> So always check the memcg's vm status to alleviate memstalls when
> it's in pressure.
>
This is interesting. UDP is the only protocol which supports memory
accounting (i.e. udp_memory_allocated) but it does not define
memory_pressure. In addition, it does have sysctl_udp_mem. So
effectively UDP supports a hard limit and ignores memcg pressure at the
moment. This patch will change its behavior to consider memcg pressure
as well. I don't have any objection but let's get opinion of UDP
maintainer.
> Signed-off-by: Abel Wu <[email protected]>
> ---
> include/net/sock.h | 6 ++----
> 1 file changed, 2 insertions(+), 4 deletions(-)
>
> diff --git a/include/net/sock.h b/include/net/sock.h
> index 3f63253ee092..ad1895ffbc4a 100644
> --- a/include/net/sock.h
> +++ b/include/net/sock.h
> @@ -1411,13 +1411,11 @@ static inline bool sk_has_memory_pressure(const struct sock *sk)
>
> static inline bool sk_under_memory_pressure(const struct sock *sk)
> {
> - if (!sk->sk_prot->memory_pressure)
> - return false;
> -
> if (mem_cgroup_under_socket_pressure(sk->sk_memcg))
> return true;
>
> - return !!*sk->sk_prot->memory_pressure;
> + return sk->sk_prot->memory_pressure &&
> + *sk->sk_prot->memory_pressure;
> }
>
> static inline long
> --
> 2.37.3
>
On Fri, Jun 02, 2023 at 04:11:35PM +0800, Abel Wu wrote:
> The status of global socket memory pressure is updated when:
>
> a) __sk_mem_raise_allocated():
>
> enter: sk_memory_allocated(sk) > sysctl_mem[1]
> leave: sk_memory_allocated(sk) <= sysctl_mem[0]
>
> b) __sk_mem_reduce_allocated():
>
> leave: sk_under_memory_pressure(sk) &&
> sk_memory_allocated(sk) < sysctl_mem[0]
There is also sk_page_frag_refill() where we can enter the global
protocol memory pressure on actual global memory pressure i.e. page
allocation failed. However this might be irrelevant from this patch's
perspective as the focus is on the leaving part.
>
> So the conditions of leaving global pressure are inconstant, which
*inconsistent
> may lead to the situation that one pressured net-memcg prevents the
> global pressure from being cleared when there is indeed no global
> pressure, thus the global constrains are still in effect unexpectedly
> on the other sockets.
>
> This patch fixes this by ignoring the net-memcg's pressure when
> deciding whether should leave global memory pressure.
>
> Fixes: e1aab161e013 ("socket: initial cgroup code.")
> Signed-off-by: Abel Wu <[email protected]>
This patch looks good.
On Fri, Jun 2, 2023 at 10:42 PM Shakeel Butt <[email protected]> wrote:
>
> On Fri, Jun 02, 2023 at 04:11:34PM +0800, Abel Wu wrote:
> > The sk_under_memory_pressure() is called to check whether there is
> > memory pressure related to this socket. But now it ignores the net-
> > memcg's pressure if the proto of the socket doesn't care about the
> > global pressure, which may put burden on its memcg compaction or
> > reclaim path (also remember that socket memory is un-reclaimable).
> >
> > So always check the memcg's vm status to alleviate memstalls when
> > it's in pressure.
> >
>
> This is interesting. UDP is the only protocol which supports memory
> accounting (i.e. udp_memory_allocated) but it does not define
> memory_pressure. In addition, it does have sysctl_udp_mem. So
> effectively UDP supports a hard limit and ignores memcg pressure at the
> moment. This patch will change its behavior to consider memcg pressure
> as well. I don't have any objection but let's get opinion of UDP
> maintainer.
Others have more experience with memory pressure on UDP, for the
record. Paolo worked on UDP memory pressure in
https://lore.kernel.org/netdev/[email protected]/
It does seem odd to me to modify sk_under_memory_pressure only. See
for instance its use in __sk_mem_raise_allocated:
if (sk_has_memory_pressure(sk)) {
u64 alloc;
if (!sk_under_memory_pressure(sk))
return 1;
This is not even reached as sk_has_memory_pressure is false for UDP.
So this commit only affects the only other protocol-independent
caller, __sk_mem_reduce_allocated, to possibly call
sk_leave_memory_pressure if now under the global limit.
What is the expected behavioral change in practice of this commit?
> > Signed-off-by: Abel Wu <[email protected]>
> > ---
> > include/net/sock.h | 6 ++----
> > 1 file changed, 2 insertions(+), 4 deletions(-)
> >
> > diff --git a/include/net/sock.h b/include/net/sock.h
> > index 3f63253ee092..ad1895ffbc4a 100644
> > --- a/include/net/sock.h
> > +++ b/include/net/sock.h
> > @@ -1411,13 +1411,11 @@ static inline bool sk_has_memory_pressure(const struct sock *sk)
> >
> > static inline bool sk_under_memory_pressure(const struct sock *sk)
> > {
> > - if (!sk->sk_prot->memory_pressure)
> > - return false;
> > -
> > if (mem_cgroup_under_socket_pressure(sk->sk_memcg))
> > return true;
> >
> > - return !!*sk->sk_prot->memory_pressure;
> > + return sk->sk_prot->memory_pressure &&
> > + *sk->sk_prot->memory_pressure;
> > }
> >
> > static inline long
> > --
> > 2.37.3
> >
On 6/4/23 6:36 PM, Willem de Bruijn wrote:
> On Fri, Jun 2, 2023 at 10:42 PM Shakeel Butt <[email protected]> wrote:
>>
>> On Fri, Jun 02, 2023 at 04:11:34PM +0800, Abel Wu wrote:
>>> The sk_under_memory_pressure() is called to check whether there is
>>> memory pressure related to this socket. But now it ignores the net-
>>> memcg's pressure if the proto of the socket doesn't care about the
>>> global pressure, which may put burden on its memcg compaction or
>>> reclaim path (also remember that socket memory is un-reclaimable).
>>>
>>> So always check the memcg's vm status to alleviate memstalls when
>>> it's in pressure.
>>>
>>
>> This is interesting. UDP is the only protocol which supports memory
>> accounting (i.e. udp_memory_allocated) but it does not define
>> memory_pressure. In addition, it does have sysctl_udp_mem. So
>> effectively UDP supports a hard limit and ignores memcg pressure at the
>> moment. This patch will change its behavior to consider memcg pressure
>> as well. I don't have any objection but let's get opinion of UDP
>> maintainer.
>
> Others have more experience with memory pressure on UDP, for the
> record. Paolo worked on UDP memory pressure in
> https://lore.kernel.org/netdev/[email protected]/
>
> It does seem odd to me to modify sk_under_memory_pressure only. See
> for instance its use in __sk_mem_raise_allocated:
>
> if (sk_has_memory_pressure(sk)) {
> u64 alloc;
>
> if (!sk_under_memory_pressure(sk))
> return 1;
>
> This is not even reached as sk_has_memory_pressure is false for UDP.
I intended to make __sk_mem_raise_allocated() be aware of net-memcg
pressure instead of just this bit [1][2].
[1]
https://lore.kernel.org/lkml/[email protected]/
[2]
https://lore.kernel.org/lkml/[email protected]/
And TBH I am wondering why considering memcg's pressure here, as the
main part in this if statement is to allow the sockets that are below
average memory usage to raise from a *global* memory view, which seems
nothing to do with memcg.
> So this commit only affects the only other protocol-independent
> caller, __sk_mem_reduce_allocated, to possibly call
> sk_leave_memory_pressure if now under the global limit.
>
> What is the expected behavioral change in practice of this commit?
Be more conservative on sockmem alloc if under memcg pressure, to
avoid worse memstall/latency.
>
>
>>> Signed-off-by: Abel Wu <[email protected]>
>>> ---
>>> include/net/sock.h | 6 ++----
>>> 1 file changed, 2 insertions(+), 4 deletions(-)
>>>
>>> diff --git a/include/net/sock.h b/include/net/sock.h
>>> index 3f63253ee092..ad1895ffbc4a 100644
>>> --- a/include/net/sock.h
>>> +++ b/include/net/sock.h
>>> @@ -1411,13 +1411,11 @@ static inline bool sk_has_memory_pressure(const struct sock *sk)
>>>
>>> static inline bool sk_under_memory_pressure(const struct sock *sk)
>>> {
>>> - if (!sk->sk_prot->memory_pressure)
>>> - return false;
>>> -
>>> if (mem_cgroup_under_socket_pressure(sk->sk_memcg))
>>> return true;
>>>
>>> - return !!*sk->sk_prot->memory_pressure;
>>> + return sk->sk_prot->memory_pressure &&
>>> + *sk->sk_prot->memory_pressure;
>>> }
>>>
>>> static inline long
>>> --
>>> 2.37.3
>>>
On Mon, 2023-06-05 at 11:44 +0800, Abel Wu wrote:
> On 6/4/23 6:36 PM, Willem de Bruijn wrote:
> > On Fri, Jun 2, 2023 at 10:42 PM Shakeel Butt <[email protected]> wrote:
> > >
> > > On Fri, Jun 02, 2023 at 04:11:34PM +0800, Abel Wu wrote:
> > > > The sk_under_memory_pressure() is called to check whether there is
> > > > memory pressure related to this socket. But now it ignores the net-
> > > > memcg's pressure if the proto of the socket doesn't care about the
> > > > global pressure, which may put burden on its memcg compaction or
> > > > reclaim path (also remember that socket memory is un-reclaimable).
> > > >
> > > > So always check the memcg's vm status to alleviate memstalls when
> > > > it's in pressure.
> > > >
> > >
> > > This is interesting. UDP is the only protocol which supports memory
> > > accounting (i.e. udp_memory_allocated) but it does not define
> > > memory_pressure. In addition, it does have sysctl_udp_mem. So
> > > effectively UDP supports a hard limit and ignores memcg pressure at the
> > > moment. This patch will change its behavior to consider memcg pressure
> > > as well. I don't have any objection but let's get opinion of UDP
> > > maintainer.
Thanks for the head-up, I did not notice the side effect on UDP.
>
> > So this commit only affects the only other protocol-independent
> > caller, __sk_mem_reduce_allocated, to possibly call
> > sk_leave_memory_pressure if now under the global limit.
> >
> > What is the expected behavioral change in practice of this commit?
>
> Be more conservative on sockmem alloc if under memcg pressure, to
> avoid worse memstall/latency.
I guess the above is for TCP sockets only, right? Or at least not for
UDP sockets?
If so, I think we should avoid change of behaviour for UDP - e.g.
keeping the initial 'if (!sk->sk_prot->memory_pressure)' in
sk_under_memory_pressure(), with some comments about the rationale for
future memory. That should preserve the whole patchset effect for other
protocols, right?
If instead you are also interested into UDP sockets under pressure, how
that is going to work? UDP sockets can reclaim memory only at send and
close time. A memcg under pressure could starve some sockets forever if
the the ones keeping the memory busy are left untouched.
Cheers,
Paolo
On 6/3/23 4:53 AM, Shakeel Butt wrote:
> On Fri, Jun 02, 2023 at 04:11:35PM +0800, Abel Wu wrote:
>> The status of global socket memory pressure is updated when:
>>
>> a) __sk_mem_raise_allocated():
>>
>> enter: sk_memory_allocated(sk) > sysctl_mem[1]
>> leave: sk_memory_allocated(sk) <= sysctl_mem[0]
>>
>> b) __sk_mem_reduce_allocated():
>>
>> leave: sk_under_memory_pressure(sk) &&
>> sk_memory_allocated(sk) < sysctl_mem[0]
>
> There is also sk_page_frag_refill() where we can enter the global
> protocol memory pressure on actual global memory pressure i.e. page
> allocation failed. However this might be irrelevant from this patch's
> perspective as the focus is on the leaving part.
Leaving prot pressure or not under actual global vm pressure is
something similar to what you concerned last time (prot & memcg
is now intermingled), as this will mix prot & global together.
To decouple global info from prot level pressure, a new variable
might be needed. But I doubt the necessity as this seems to be a
rare case but a constant overhead on net core path to check the
global status (although can be relieved by static key).
And after a second thought, failing in skb_page_frag_refill()
doesn't necessarily mean there is global memory pressure since
it can due to the mpol/memset of the current task.
>
>>
>> So the conditions of leaving global pressure are inconstant, which
>
> *inconsistent
Will fix in next version.
>
>> may lead to the situation that one pressured net-memcg prevents the
>> global pressure from being cleared when there is indeed no global
>> pressure, thus the global constrains are still in effect unexpectedly
>> on the other sockets.
>>
>> This patch fixes this by ignoring the net-memcg's pressure when
>> deciding whether should leave global memory pressure.
>>
>> Fixes: e1aab161e013 ("socket: initial cgroup code.")
>> Signed-off-by: Abel Wu <[email protected]>
>
> This patch looks good.
Thanks!
Abel
On 6/5/23 4:27 PM, Paolo Abeni wrote:
> On Mon, 2023-06-05 at 11:44 +0800, Abel Wu wrote:
>> On 6/4/23 6:36 PM, Willem de Bruijn wrote:
>>> On Fri, Jun 2, 2023 at 10:42 PM Shakeel Butt <[email protected]> wrote:
>>>>
>>>> On Fri, Jun 02, 2023 at 04:11:34PM +0800, Abel Wu wrote:
>>>>> The sk_under_memory_pressure() is called to check whether there is
>>>>> memory pressure related to this socket. But now it ignores the net-
>>>>> memcg's pressure if the proto of the socket doesn't care about the
>>>>> global pressure, which may put burden on its memcg compaction or
>>>>> reclaim path (also remember that socket memory is un-reclaimable).
>>>>>
>>>>> So always check the memcg's vm status to alleviate memstalls when
>>>>> it's in pressure.
>>>>>
>>>>
>>>> This is interesting. UDP is the only protocol which supports memory
>>>> accounting (i.e. udp_memory_allocated) but it does not define
>>>> memory_pressure. In addition, it does have sysctl_udp_mem. So
>>>> effectively UDP supports a hard limit and ignores memcg pressure at the
>>>> moment. This patch will change its behavior to consider memcg pressure
>>>> as well. I don't have any objection but let's get opinion of UDP
>>>> maintainer.
>
> Thanks for the head-up, I did not notice the side effect on UDP.
>
>>
>>> So this commit only affects the only other protocol-independent
>>> caller, __sk_mem_reduce_allocated, to possibly call
>>> sk_leave_memory_pressure if now under the global limit.
>>>
>>> What is the expected behavioral change in practice of this commit?
>>
>> Be more conservative on sockmem alloc if under memcg pressure, to
>> avoid worse memstall/latency.
>
> I guess the above is for TCP sockets only, right? Or at least not for
> UDP sockets?
Yes, I started off with TCP but wondering if it is applicable to the
others too as the 'problem' sounds really generic to me.
>
> If so, I think we should avoid change of behaviour for UDP - e.g.
> keeping the initial 'if (!sk->sk_prot->memory_pressure)' in
> sk_under_memory_pressure(), with some comments about the rationale for
> future memory. That should preserve the whole patchset effect for other
> protocols, right?
Keeping the if statement as it is would imply the prot pressure as a
master 'switch' to all kinds of pressure. IMHO this might hurt other
protocols with pressure enabled if they are all used in one memcg which
happens to be under vmpressure, IOW UDP allocations are given higher
priority than others.
>
> If instead you are also interested into UDP sockets under pressure, how
> that is going to work? UDP sockets can reclaim memory only at send and
> close time. A memcg under pressure could starve some sockets forever if
> the the ones keeping the memory busy are left untouched.
Yes.. And it starts to get me confused that why&when should the memcg
pressure be used given that we don't want to put harsh constrains on
sockmem even under memcg pressure.
Thanks!
Abel
On 6/3/23 4:25 AM, Shakeel Butt wrote:
> On Fri, Jun 02, 2023 at 04:11:33PM +0800, Abel Wu wrote:
>> The callers of mem_cgroup_under_socket_pressure() should always make
>> sure that (mem_cgroup_sockets_enabled && sk->sk_memcg) is true. So
>> instead of coding around all the callsites, put the dependencies into
>> mem_cgroup_under_socket_pressure() to avoid redundancy and possibly
>> bugs.
>>
>> This change might also introduce slight function call overhead *iff*
>> the function gets expanded in the future. But for now this change
>> doesn't make binaries different (checked by vimdiff) except the one
>> net/ipv4/tcp_input.o (by scripts/bloat-o-meter), which is probably
>> negligible to performance:
>>
>> add/remove: 0/0 grow/shrink: 1/2 up/down: 5/-5 (0)
>> Function old new delta
>> tcp_grow_window 573 578 +5
>> tcp_try_rmem_schedule 1083 1081 -2
>> tcp_check_space 324 321 -3
>> Total: Before=44647, After=44647, chg +0.00%
>>
>> So folding the dependencies into mem_cgroup_under_socket_pressure()
>> is generally a good thing and provides better readablility.
>>
>
> I don't see how it is improving readability. If you have removed the use
> of mem_cgroup_sockets_enabled completely from the networking then I can
> understand but this change IMHO will actually decrease the readability
> because the later readers will have to reason why we are doing this
> check at some places but not other.
Yes, I agree. I am trying to let networking get rid of this macro
entirely, but get stuck on inet_csk_accept().. :(
Hello,
kernel test robot noticed a 2.8% improvement of netperf.Throughput_Mbps on:
commit: c89fa56a8776f98d8e4ed9310f5b178288005916 ("[PATCH net-next v5 3/3] sock: Fix misuse of sk_under_memory_pressure()")
url: https://github.com/intel-lab-lkp/linux/commits/Abel-Wu/net-memcg-Fold-dependency-into-memcg-pressure-cond/20230602-161424
base: https://git.kernel.org/cgit/linux/kernel/git/davem/net-next.git a395b8d1c7c3a074bfa83b9759a4a11901a295c5
patch link: https://lore.kernel.org/all/[email protected]/
patch subject: [PATCH net-next v5 3/3] sock: Fix misuse of sk_under_memory_pressure()
testcase: netperf
test machine: 128 threads 4 sockets Intel(R) Xeon(R) Gold 6338 CPU @ 2.00GHz (Ice Lake) with 256G memory
parameters:
ip: ipv4
runtime: 300s
nr_threads: 50%
cluster: cs-localhost
test: TCP_STREAM
cpufreq_governor: performance
Details are as below:
-------------------------------------------------------------------------------------------------->
To reproduce:
git clone https://github.com/intel/lkp-tests.git
cd lkp-tests
sudo bin/lkp install job.yaml # job file is attached in this email
bin/lkp split-job --compatible job.yaml # generate the yaml file for lkp run
sudo bin/lkp run generated-yaml-file
# if come across any failure that blocks the test,
# please remove ~/.lkp and /lkp dir to run from a clean state.
=========================================================================================
cluster/compiler/cpufreq_governor/ip/kconfig/nr_threads/rootfs/runtime/tbox_group/test/testcase:
cs-localhost/gcc-12/performance/ipv4/x86_64-rhel-8.3/50%/debian-11.1-x86_64-20220510.cgz/300s/lkp-icl-2sp2/TCP_STREAM/netperf
commit:
5ec3359188 ("sock: Always take memcg pressure into consideration")
c89fa56a87 ("sock: Fix misuse of sk_under_memory_pressure()")
5ec335918819adbd c89fa56a8776f98d8e4ed9310f5
---------------- ---------------------------
%stddev %change %stddev
\ | \
15.76 -0.4 15.31 turbostat.C1%
34.82 +1.4% 35.31 turbostat.RAMWatt
5050526 +1.3% 5115222 vmstat.system.cs
2620207 +1.1% 2648074 vmstat.system.in
1.676e+09 +2.8% 1.723e+09 proc-vmstat.numa_hit
1.676e+09 +2.8% 1.722e+09 proc-vmstat.numa_local
1.338e+10 +2.8% 1.375e+10 proc-vmstat.pgalloc_normal
1.338e+10 +2.8% 1.375e+10 proc-vmstat.pgfree
0.00 ? 22% -25.8% 0.00 ? 5% sched_debug.cpu.next_balance.stddev
0.00 ? 24% -29.2% 0.00 ? 46% sched_debug.rt_rq:.rt_time.avg
0.02 ? 24% -29.2% 0.01 ? 46% sched_debug.rt_rq:.rt_time.max
0.00 ? 24% -29.2% 0.00 ? 46% sched_debug.rt_rq:.rt_time.stddev
22841 +2.8% 23483 netperf.Throughput_Mbps
1461840 +2.8% 1502930 netperf.Throughput_total_Mbps
425.78 +3.6% 441.16 netperf.time.user_time
101776 ? 6% +19.6% 121747 ? 7% netperf.time.voluntary_context_switches
3.346e+09 +2.8% 3.44e+09 netperf.workload
2.182e+10 +2.5% 2.236e+10 perf-stat.i.branch-instructions
0.88 -0.0 0.87 perf-stat.i.branch-miss-rate%
2.55 -0.1 2.46 ? 2% perf-stat.i.cache-miss-rate%
6.286e+09 +2.2% 6.425e+09 perf-stat.i.cache-references
5099243 +1.3% 5164362 perf-stat.i.context-switches
2.59 -2.8% 2.52 perf-stat.i.cpi
12909 ? 3% -8.8% 11772 ? 3% perf-stat.i.cpu-migrations
3.253e+10 +2.6% 3.337e+10 perf-stat.i.dTLB-loads
1.888e+10 +2.7% 1.939e+10 perf-stat.i.dTLB-stores
1.142e+11 +2.6% 1.172e+11 perf-stat.i.instructions
0.39 +2.8% 0.40 perf-stat.i.ipc
621.17 +2.6% 637.06 perf-stat.i.metric.M/sec
50.27 ? 4% +6.5 56.79 ? 3% perf-stat.i.node-store-miss-rate%
26917446 ? 3% +7.6% 28967728 ? 2% perf-stat.i.node-store-misses
27529051 ? 6% -17.5% 22707159 ? 6% perf-stat.i.node-stores
0.87 -0.0 0.86 perf-stat.overall.branch-miss-rate%
2.52 -0.1 2.43 ? 2% perf-stat.overall.cache-miss-rate%
2.59 -2.8% 2.52 perf-stat.overall.cpi
0.39 +2.9% 0.40 perf-stat.overall.ipc
49.47 ? 4% +6.6 56.09 ? 3% perf-stat.overall.node-store-miss-rate%
2.174e+10 +2.5% 2.228e+10 perf-stat.ps.branch-instructions
6.267e+09 +2.2% 6.405e+09 perf-stat.ps.cache-references
5081226 +1.3% 5146095 perf-stat.ps.context-switches
12877 ? 3% -8.8% 11745 ? 3% perf-stat.ps.cpu-migrations
3.241e+10 +2.6% 3.325e+10 perf-stat.ps.dTLB-loads
1.881e+10 +2.7% 1.932e+10 perf-stat.ps.dTLB-stores
1.138e+11 +2.6% 1.168e+11 perf-stat.ps.instructions
26872617 ? 3% +7.6% 28926338 ? 2% perf-stat.ps.node-store-misses
27479983 ? 6% -17.5% 22669639 ? 6% perf-stat.ps.node-stores
3.431e+13 +2.6% 3.52e+13 perf-stat.total.instructions
86.42 -14.7 71.74 ? 20% perf-profile.calltrace.cycles-pp.main.__libc_start_main
86.46 -14.7 71.79 ? 20% perf-profile.calltrace.cycles-pp.__libc_start_main
51.74 -0.9 50.86 perf-profile.calltrace.cycles-pp.entry_SYSCALL_64_after_hwframe.__send.send_omni_inner.send_tcp_stream.main
1.08 ? 30% -0.6 0.44 ? 44% perf-profile.calltrace.cycles-pp.page_counter_try_charge.try_charge_memcg.mem_cgroup_charge_skmem.tcp_data_queue.tcp_rcv_established
51.54 -0.6 50.95 perf-profile.calltrace.cycles-pp.do_syscall_64.entry_SYSCALL_64_after_hwframe.__send.send_omni_inner.send_tcp_stream
2.00 -0.5 1.46 ? 8% perf-profile.calltrace.cycles-pp.__sk_mem_reduce_allocated.tcp_recvmsg_locked.tcp_recvmsg.inet_recvmsg.sock_recvmsg
49.60 -0.5 49.06 perf-profile.calltrace.cycles-pp.tcp_sendmsg.sock_sendmsg.__sys_sendto.__x64_sys_sendto.do_syscall_64
50.04 -0.5 49.56 perf-profile.calltrace.cycles-pp.sock_sendmsg.__sys_sendto.__x64_sys_sendto.do_syscall_64.entry_SYSCALL_64_after_hwframe
51.20 -0.5 50.74 perf-profile.calltrace.cycles-pp.__x64_sys_sendto.do_syscall_64.entry_SYSCALL_64_after_hwframe.__send.send_omni_inner
51.10 -0.5 50.64 perf-profile.calltrace.cycles-pp.__sys_sendto.__x64_sys_sendto.do_syscall_64.entry_SYSCALL_64_after_hwframe.__send
13.57 -0.4 13.20 perf-profile.calltrace.cycles-pp.release_sock.tcp_sendmsg.sock_sendmsg.__sys_sendto.__x64_sys_sendto
13.07 -0.3 12.72 perf-profile.calltrace.cycles-pp.__release_sock.release_sock.tcp_sendmsg.sock_sendmsg.__sys_sendto
2.48 ? 2% -0.3 2.17 ? 2% perf-profile.calltrace.cycles-pp.mem_cgroup_charge_skmem.__sk_mem_raise_allocated.__sk_mem_schedule.tcp_wmem_schedule.tcp_sendmsg_locked
1.36 -0.3 1.05 ? 2% perf-profile.calltrace.cycles-pp.__sk_mem_reduce_allocated.tcp_clean_rtx_queue.tcp_ack.tcp_rcv_established.tcp_v4_do_rcv
3.23 -0.3 2.96 perf-profile.calltrace.cycles-pp.__sk_mem_raise_allocated.__sk_mem_schedule.tcp_wmem_schedule.tcp_sendmsg_locked.tcp_sendmsg
3.28 -0.3 3.00 perf-profile.calltrace.cycles-pp.__sk_mem_schedule.tcp_wmem_schedule.tcp_sendmsg_locked.tcp_sendmsg.sock_sendmsg
3.33 -0.3 3.06 perf-profile.calltrace.cycles-pp.tcp_wmem_schedule.tcp_sendmsg_locked.tcp_sendmsg.sock_sendmsg.__sys_sendto
6.08 -0.3 5.82 perf-profile.calltrace.cycles-pp.tcp_rcv_established.tcp_v4_do_rcv.tcp_v4_rcv.ip_protocol_deliver_rcu.ip_local_deliver_finish
6.21 -0.3 5.96 perf-profile.calltrace.cycles-pp.tcp_v4_do_rcv.tcp_v4_rcv.ip_protocol_deliver_rcu.ip_local_deliver_finish.__netif_receive_skb_one_core
3.32 -0.2 3.07 perf-profile.calltrace.cycles-pp.tcp_clean_rtx_queue.tcp_ack.tcp_rcv_established.tcp_v4_do_rcv.__release_sock
1.40 ? 3% -0.2 1.17 ? 7% perf-profile.calltrace.cycles-pp.refill_stock.__sk_mem_reduce_allocated.tcp_recvmsg_locked.tcp_recvmsg.inet_recvmsg
2.90 ? 3% -0.2 2.67 ? 2% perf-profile.calltrace.cycles-pp.try_charge_memcg.mem_cgroup_charge_skmem.tcp_data_queue.tcp_rcv_established.tcp_v4_do_rcv
1.26 ? 3% -0.2 1.03 ? 7% perf-profile.calltrace.cycles-pp.page_counter_uncharge.drain_stock.refill_stock.__sk_mem_reduce_allocated.tcp_recvmsg_locked
3.69 -0.2 3.46 perf-profile.calltrace.cycles-pp.tcp_ack.tcp_rcv_established.tcp_v4_do_rcv.__release_sock.release_sock
1.27 ? 3% -0.2 1.05 ? 7% perf-profile.calltrace.cycles-pp.drain_stock.refill_stock.__sk_mem_reduce_allocated.tcp_recvmsg_locked.tcp_recvmsg
4.66 -0.2 4.46 perf-profile.calltrace.cycles-pp.tcp_data_queue.tcp_rcv_established.tcp_v4_do_rcv.tcp_v4_rcv.ip_protocol_deliver_rcu
2.15 ? 3% -0.2 1.98 ? 2% perf-profile.calltrace.cycles-pp.mem_cgroup_charge_skmem.tcp_data_queue.tcp_rcv_established.tcp_v4_do_rcv.tcp_v4_rcv
0.64 ? 2% -0.2 0.47 ? 45% perf-profile.calltrace.cycles-pp.tcp_rcv_established.tcp_v4_do_rcv.__release_sock.release_sock.tcp_recvmsg
1.52 -0.2 1.37 ? 7% perf-profile.calltrace.cycles-pp.__sk_flush_backlog.tcp_recvmsg_locked.tcp_recvmsg.inet_recvmsg.sock_recvmsg
1.70 ? 3% -0.1 1.55 ? 2% perf-profile.calltrace.cycles-pp.try_charge_memcg.mem_cgroup_charge_skmem.__sk_mem_raise_allocated.__sk_mem_schedule.tcp_wmem_schedule
1.42 -0.1 1.27 ? 7% perf-profile.calltrace.cycles-pp.tcp_v4_do_rcv.__release_sock.__sk_flush_backlog.tcp_recvmsg_locked.tcp_recvmsg
1.48 -0.1 1.33 ? 7% perf-profile.calltrace.cycles-pp.__release_sock.__sk_flush_backlog.tcp_recvmsg_locked.tcp_recvmsg.inet_recvmsg
1.29 -0.1 1.16 ? 7% perf-profile.calltrace.cycles-pp.tcp_rcv_established.tcp_v4_do_rcv.__release_sock.__sk_flush_backlog.tcp_recvmsg_locked
1.08 -0.1 0.96 ? 7% perf-profile.calltrace.cycles-pp.tcp_data_queue.tcp_rcv_established.tcp_v4_do_rcv.__release_sock.__sk_flush_backlog
0.96 ? 2% -0.1 0.85 ? 7% perf-profile.calltrace.cycles-pp.mem_cgroup_charge_skmem.tcp_data_queue.tcp_rcv_established.tcp_v4_do_rcv.__release_sock
1.01 ? 3% -0.1 0.90 ? 2% perf-profile.calltrace.cycles-pp.refill_stock.__sk_mem_reduce_allocated.tcp_clean_rtx_queue.tcp_ack.tcp_rcv_established
0.89 -0.1 0.78 ? 7% perf-profile.calltrace.cycles-pp.release_sock.tcp_recvmsg.inet_recvmsg.sock_recvmsg.__sys_recvfrom
0.92 ? 3% -0.1 0.82 ? 2% perf-profile.calltrace.cycles-pp.page_counter_uncharge.drain_stock.refill_stock.__sk_mem_reduce_allocated.tcp_clean_rtx_queue
0.93 ? 3% -0.1 0.83 ? 2% perf-profile.calltrace.cycles-pp.drain_stock.refill_stock.__sk_mem_reduce_allocated.tcp_clean_rtx_queue.tcp_ack
0.75 -0.1 0.65 ? 7% perf-profile.calltrace.cycles-pp.__release_sock.release_sock.tcp_recvmsg.inet_recvmsg.sock_recvmsg
0.71 -0.1 0.62 ? 7% perf-profile.calltrace.cycles-pp.tcp_v4_do_rcv.__release_sock.release_sock.tcp_recvmsg.inet_recvmsg
0.85 ? 4% -0.1 0.76 ? 2% perf-profile.calltrace.cycles-pp.page_counter_try_charge.try_charge_memcg.mem_cgroup_charge_skmem.__sk_mem_raise_allocated.__sk_mem_schedule
0.96 -0.0 0.92 perf-profile.calltrace.cycles-pp.skb_release_data.napi_consume_skb.net_rx_action.__do_softirq.do_softirq
1.04 -0.0 1.00 perf-profile.calltrace.cycles-pp.napi_consume_skb.net_rx_action.__do_softirq.do_softirq.__local_bh_enable_ip
0.54 +0.0 0.56 perf-profile.calltrace.cycles-pp.__alloc_skb.tcp_stream_alloc_skb.tcp_sendmsg_locked.tcp_sendmsg.sock_sendmsg
1.04 +0.0 1.08 ? 2% perf-profile.calltrace.cycles-pp.__schedule.schedule_idle.do_idle.cpu_startup_entry.start_secondary
1.10 +0.0 1.15 ? 2% perf-profile.calltrace.cycles-pp.schedule_idle.do_idle.cpu_startup_entry.start_secondary.secondary_startup_64_no_verify
2.68 +0.1 2.73 perf-profile.calltrace.cycles-pp.sysvec_call_function_single.asm_sysvec_call_function_single.acpi_safe_halt.acpi_idle_enter.cpuidle_enter_state
1.14 +0.1 1.20 perf-profile.calltrace.cycles-pp.skb_release_data.__kfree_skb.tcp_clean_rtx_queue.tcp_ack.tcp_rcv_established
1.16 +0.1 1.22 perf-profile.calltrace.cycles-pp.__kfree_skb.tcp_clean_rtx_queue.tcp_ack.tcp_rcv_established.tcp_v4_do_rcv
31.36 +0.2 31.59 perf-profile.calltrace.cycles-pp.sock_recvmsg.__sys_recvfrom.__x64_sys_recvfrom.do_syscall_64.entry_SYSCALL_64_after_hwframe
31.75 +0.2 31.99 perf-profile.calltrace.cycles-pp.__sys_recvfrom.__x64_sys_recvfrom.do_syscall_64.entry_SYSCALL_64_after_hwframe.recv
31.13 +0.2 31.37 perf-profile.calltrace.cycles-pp.tcp_recvmsg.inet_recvmsg.sock_recvmsg.__sys_recvfrom.__x64_sys_recvfrom
31.20 +0.2 31.45 perf-profile.calltrace.cycles-pp.inet_recvmsg.sock_recvmsg.__sys_recvfrom.__x64_sys_recvfrom.do_syscall_64
31.80 +0.2 32.05 perf-profile.calltrace.cycles-pp.__x64_sys_recvfrom.do_syscall_64.entry_SYSCALL_64_after_hwframe.recv.recv_omni
29.91 +0.3 30.16 perf-profile.calltrace.cycles-pp.tcp_recvmsg_locked.tcp_recvmsg.inet_recvmsg.sock_recvmsg.__sys_recvfrom
32.02 +0.3 32.28 perf-profile.calltrace.cycles-pp.do_syscall_64.entry_SYSCALL_64_after_hwframe.recv.recv_omni.process_requests
32.11 +0.3 32.37 perf-profile.calltrace.cycles-pp.entry_SYSCALL_64_after_hwframe.recv.recv_omni.process_requests.spawn_child
32.72 +0.3 32.99 perf-profile.calltrace.cycles-pp.recv.recv_omni.process_requests.spawn_child.accept_connection
33.18 +0.3 33.45 perf-profile.calltrace.cycles-pp.recv_omni.process_requests.spawn_child.accept_connection.accept_connections
15.32 +0.4 15.71 perf-profile.calltrace.cycles-pp.copyout._copy_to_iter.__skb_datagram_iter.skb_copy_datagram_iter.tcp_recvmsg_locked
15.54 +0.4 15.92 perf-profile.calltrace.cycles-pp._copy_to_iter.__skb_datagram_iter.skb_copy_datagram_iter.tcp_recvmsg_locked.tcp_recvmsg
17.05 +0.4 17.47 perf-profile.calltrace.cycles-pp.skb_copy_datagram_iter.tcp_recvmsg_locked.tcp_recvmsg.inet_recvmsg.sock_recvmsg
17.02 +0.4 17.44 perf-profile.calltrace.cycles-pp.__skb_datagram_iter.skb_copy_datagram_iter.tcp_recvmsg_locked.tcp_recvmsg.inet_recvmsg
86.46 -14.7 71.79 ? 20% perf-profile.children.cycles-pp.__libc_start_main
86.58 -14.7 71.92 ? 20% perf-profile.children.cycles-pp.main
3.52 -0.7 2.78 perf-profile.children.cycles-pp.__sk_mem_reduce_allocated
6.21 ? 2% -0.5 5.67 perf-profile.children.cycles-pp.mem_cgroup_charge_skmem
49.64 -0.5 49.10 perf-profile.children.cycles-pp.tcp_sendmsg
50.06 -0.5 49.58 perf-profile.children.cycles-pp.sock_sendmsg
51.22 -0.5 50.76 perf-profile.children.cycles-pp.__x64_sys_sendto
51.13 -0.5 50.67 perf-profile.children.cycles-pp.__sys_sendto
5.14 ? 3% -0.3 4.82 ? 2% perf-profile.children.cycles-pp.try_charge_memcg
18.19 -0.3 17.87 perf-profile.children.cycles-pp.tcp_rcv_established
18.52 -0.3 18.20 perf-profile.children.cycles-pp.tcp_v4_do_rcv
15.38 -0.3 15.08 perf-profile.children.cycles-pp.__release_sock
14.62 -0.3 14.34 perf-profile.children.cycles-pp.release_sock
19.19 -0.3 18.92 perf-profile.children.cycles-pp._copy_from_iter
18.90 -0.3 18.62 perf-profile.children.cycles-pp.copyin
3.45 -0.3 3.18 perf-profile.children.cycles-pp.__sk_mem_raise_allocated
3.50 -0.3 3.23 perf-profile.children.cycles-pp.__sk_mem_schedule
6.37 -0.2 6.12 perf-profile.children.cycles-pp.tcp_data_queue
19.75 -0.2 19.51 perf-profile.children.cycles-pp.skb_do_copy_data_nocache
2.58 ? 3% -0.2 2.34 ? 2% perf-profile.children.cycles-pp.refill_stock
2.30 ? 3% -0.2 2.06 ? 2% perf-profile.children.cycles-pp.page_counter_uncharge
2.32 ? 3% -0.2 2.08 ? 2% perf-profile.children.cycles-pp.drain_stock
3.35 -0.2 3.11 perf-profile.children.cycles-pp.tcp_wmem_schedule
3.65 -0.2 3.41 perf-profile.children.cycles-pp.tcp_clean_rtx_queue
2.70 ? 3% -0.2 2.49 perf-profile.children.cycles-pp.page_counter_try_charge
4.38 -0.2 4.17 perf-profile.children.cycles-pp.tcp_ack
0.47 ? 3% -0.1 0.37 ? 3% perf-profile.children.cycles-pp.mem_cgroup_uncharge_skmem
0.52 ? 2% -0.0 0.50 perf-profile.children.cycles-pp._raw_spin_trylock
0.39 -0.0 0.37 perf-profile.children.cycles-pp.select_task_rq_fair
0.10 ? 3% -0.0 0.09 ? 4% perf-profile.children.cycles-pp.security_socket_recvmsg
0.06 +0.0 0.07 perf-profile.children.cycles-pp._raw_spin_unlock_bh
0.18 ? 2% +0.0 0.20 ? 2% perf-profile.children.cycles-pp.ip_send_check
0.06 ? 6% +0.0 0.08 ? 6% perf-profile.children.cycles-pp.rb_next
0.17 ? 2% +0.0 0.19 ? 2% perf-profile.children.cycles-pp.tick_irq_enter
0.52 +0.0 0.54 perf-profile.children.cycles-pp.__fget_light
0.27 +0.0 0.29 perf-profile.children.cycles-pp.tcp_tso_segs
1.08 +0.0 1.10 perf-profile.children.cycles-pp.skb_page_frag_refill
0.65 +0.0 0.67 perf-profile.children.cycles-pp.tcp_stream_alloc_skb
1.11 +0.0 1.14 perf-profile.children.cycles-pp.sk_page_frag_refill
0.41 +0.0 0.43 ? 2% perf-profile.children.cycles-pp.native_sched_clock
0.64 +0.0 0.67 perf-profile.children.cycles-pp.sockfd_lookup_light
0.56 +0.0 0.58 perf-profile.children.cycles-pp.kmem_cache_alloc_node
0.47 +0.0 0.49 perf-profile.children.cycles-pp.tcp_schedule_loss_probe
0.45 +0.0 0.48 perf-profile.children.cycles-pp.sched_clock_cpu
0.29 ? 2% +0.0 0.32 perf-profile.children.cycles-pp.aa_sk_perm
0.76 +0.0 0.79 perf-profile.children.cycles-pp.ktime_get
0.72 +0.0 0.75 perf-profile.children.cycles-pp.dequeue_entity
0.22 ? 3% +0.0 0.25 ? 3% perf-profile.children.cycles-pp.set_next_entity
1.03 +0.0 1.06 perf-profile.children.cycles-pp.simple_copy_to_iter
0.86 +0.0 0.89 perf-profile.children.cycles-pp.__alloc_skb
0.48 +0.0 0.51 perf-profile.children.cycles-pp.irqtime_account_irq
0.81 +0.0 0.85 perf-profile.children.cycles-pp.dequeue_task_fair
0.53 ? 2% +0.0 0.57 ? 2% perf-profile.children.cycles-pp.tcp_event_new_data_sent
0.62 +0.0 0.66 perf-profile.children.cycles-pp.__mod_timer
0.66 +0.0 0.70 perf-profile.children.cycles-pp.sk_reset_timer
1.19 +0.0 1.23 perf-profile.children.cycles-pp.check_heap_object
0.33 ? 2% +0.0 0.37 ? 4% perf-profile.children.cycles-pp.propagate_protected_usage
1.47 +0.0 1.51 perf-profile.children.cycles-pp.__check_object_size
1.11 +0.0 1.16 ? 2% perf-profile.children.cycles-pp.schedule_idle
0.27 ? 3% +0.0 0.32 perf-profile.children.cycles-pp.security_socket_sendmsg
0.00 +0.1 0.05 perf-profile.children.cycles-pp.pick_next_entity
0.00 +0.1 0.05 perf-profile.children.cycles-pp.detach_if_pending
0.00 +0.1 0.05 perf-profile.children.cycles-pp.rb_erase
0.00 +0.1 0.05 perf-profile.children.cycles-pp.get_nohz_timer_target
2.72 +0.1 2.78 perf-profile.children.cycles-pp.sysvec_call_function_single
2.80 +0.1 2.85 perf-profile.children.cycles-pp.skb_release_data
0.23 ? 2% +0.1 0.29 perf-profile.children.cycles-pp.skb_attempt_defer_free
1.54 +0.1 1.62 perf-profile.children.cycles-pp.__kfree_skb
16.08 +0.1 16.20 perf-profile.children.cycles-pp.tcp_write_xmit
18.25 +0.1 18.39 perf-profile.children.cycles-pp.__tcp_transmit_skb
31.37 +0.2 31.60 perf-profile.children.cycles-pp.sock_recvmsg
31.16 +0.2 31.39 perf-profile.children.cycles-pp.tcp_recvmsg
31.82 +0.2 32.06 perf-profile.children.cycles-pp.__x64_sys_recvfrom
31.76 +0.2 32.00 perf-profile.children.cycles-pp.__sys_recvfrom
31.21 +0.2 31.45 perf-profile.children.cycles-pp.inet_recvmsg
29.93 +0.3 30.19 perf-profile.children.cycles-pp.tcp_recvmsg_locked
32.82 +0.3 33.08 perf-profile.children.cycles-pp.recv
33.18 +0.3 33.46 perf-profile.children.cycles-pp.recv_omni
33.18 +0.3 33.46 perf-profile.children.cycles-pp.accept_connections
33.18 +0.3 33.46 perf-profile.children.cycles-pp.accept_connection
33.18 +0.3 33.46 perf-profile.children.cycles-pp.spawn_child
33.18 +0.3 33.46 perf-profile.children.cycles-pp.process_requests
15.42 +0.4 15.80 perf-profile.children.cycles-pp.copyout
15.54 +0.4 15.93 perf-profile.children.cycles-pp._copy_to_iter
17.03 +0.4 17.45 perf-profile.children.cycles-pp.__skb_datagram_iter
17.05 +0.4 17.48 perf-profile.children.cycles-pp.skb_copy_datagram_iter
0.50 ? 3% -0.4 0.11 ? 4% perf-profile.self.cycles-pp.__sk_mem_reduce_allocated
18.79 -0.3 18.52 perf-profile.self.cycles-pp.copyin
2.12 ? 3% -0.3 1.85 ? 2% perf-profile.self.cycles-pp.page_counter_uncharge
2.54 ? 3% -0.2 2.30 perf-profile.self.cycles-pp.page_counter_try_charge
0.96 ? 3% -0.2 0.73 ? 3% perf-profile.self.cycles-pp.mem_cgroup_charge_skmem
2.40 ? 3% -0.1 2.28 ? 2% perf-profile.self.cycles-pp.try_charge_memcg
0.42 ? 3% -0.1 0.31 ? 3% perf-profile.self.cycles-pp.mem_cgroup_uncharge_skmem
0.12 ? 3% -0.0 0.10 perf-profile.self.cycles-pp.select_task_rq_fair
0.15 ? 3% -0.0 0.13 ? 2% perf-profile.self.cycles-pp.__entry_text_start
0.20 ? 2% -0.0 0.19 perf-profile.self.cycles-pp.loopback_xmit
0.05 +0.0 0.06 perf-profile.self.cycles-pp.set_next_entity
0.05 +0.0 0.06 perf-profile.self.cycles-pp.__xfrm_policy_check2
0.06 +0.0 0.07 perf-profile.self.cycles-pp.exit_to_user_mode_prepare
0.19 +0.0 0.20 perf-profile.self.cycles-pp.sock_put
0.12 ? 3% +0.0 0.14 ? 3% perf-profile.self.cycles-pp.tcp_event_new_data_sent
0.12 +0.0 0.13 ? 2% perf-profile.self.cycles-pp.tcp_v4_do_rcv
0.19 ? 3% +0.0 0.20 ? 2% perf-profile.self.cycles-pp.update_curr
0.51 +0.0 0.53 perf-profile.self.cycles-pp.__fget_light
0.26 +0.0 0.28 ? 2% perf-profile.self.cycles-pp.ktime_get
0.16 ? 2% +0.0 0.18 ? 4% perf-profile.self.cycles-pp.recv_data
0.21 ? 2% +0.0 0.23 perf-profile.self.cycles-pp.__do_softirq
0.16 ? 5% +0.0 0.18 ? 6% perf-profile.self.cycles-pp.cgroup_rstat_updated
0.39 +0.0 0.42 perf-profile.self.cycles-pp.native_sched_clock
0.89 +0.0 0.91 perf-profile.self.cycles-pp.__inet_lookup_established
0.96 +0.0 0.99 perf-profile.self.cycles-pp.__tcp_transmit_skb
0.82 +0.0 0.85 perf-profile.self.cycles-pp.check_heap_object
0.46 +0.0 0.48 perf-profile.self.cycles-pp.net_rx_action
0.23 ? 3% +0.0 0.26 perf-profile.self.cycles-pp.aa_sk_perm
0.54 +0.0 0.57 perf-profile.self.cycles-pp.tcp_recvmsg_locked
0.53 +0.0 0.56 perf-profile.self.cycles-pp.tcp_v4_rcv
0.32 ? 3% +0.0 0.37 ? 5% perf-profile.self.cycles-pp.propagate_protected_usage
0.10 ? 4% +0.1 0.17 ? 2% perf-profile.self.cycles-pp.skb_attempt_defer_free
1.14 +0.1 1.21 perf-profile.self.cycles-pp.skb_release_data
15.32 +0.4 15.71 perf-profile.self.cycles-pp.copyout
Disclaimer:
Results have been estimated based on internal Intel analysis and are provided
for informational purposes only. Any difference in system hardware or software
design or configuration may affect actual performance.
--
0-DAY CI Kernel Test Service
https://github.com/intel/lkp-tests/wiki