Received-SPF: pass (google.com: domain of linux-kernel-owner@vger.kernel.org designates 2620:137:e000::1:20 as permitted sender) client-ip=2620:137:e000::1:20;
MIME-Version: 1.0
References: <20221205184742.0952fc75@kernel.org> <202212071527223155626@zte.com.cn>
In-Reply-To: <202212071527223155626@zte.com.cn>
From:   Eric Dumazet <edumazet@google.com>
Date:   Wed, 7 Dec 2022 08:58:11 +0100
Message-ID: <CANn89iKqb64sLT2r+2YrpDyMfZ8T6z2Ygtby-ruVNNYvniaV0g@mail.gmail.com>
Subject: Re: [PATCH linux-next] net: record times of netdev_budget exhausted
To:     yang.yang29@zte.com.cn
Cc:     kuba@kernel.org, davem@davemloft.net, pabeni@redhat.com,
        bigeasy@linutronix.de, imagedong@tencent.com, kuniyu@amazon.com,
        petrm@nvidia.com, liu3101@purdue.edu, wujianguo@chinatelecom.cn,
        netdev@vger.kernel.org, linux-kernel@vger.kernel.org,
        tedheadster@gmail.com
Content-Type: text/plain; charset="UTF-8"
Precedence: bulk

On Wed, Dec 7, 2022 at 8:28 AM <yang.yang29@zte.com.cn> wrote:
>
> On Tue, 6 Dec 2022 10:47:07 +0800 (CST) kuba@kernel.org wrote:
> > But are you seeing actual performance wins in terms of throughput
> > or latency?
>
> I did a test and see 7~8% of performance difference with small and big
> netdev_budget. Detail:
> 1. machine
> In qemu. CPU is QEMU TCG CPU version 2.5+.
> 2. kernel
> Linux (none) 5.14.0-rc6+ #91 SMP Tue Dec 6 19:55:14 CST 2022 x86_64 GNU/Linux
> 3. test condition
> Run 5 rt tasks to simulate workload, task is test.sh:
> ---
> #!/bin/bash
>
> while [ 1 ]
> do
>       ls  > /dev/null
> done
> ---
> 4. test method
> Use ping -f to flood.
> # ping -f 192.168.1.201 -w 1800
>
> With netdev_buget is 500, and netdev_budget_usecs is 2000:
> 497913 packets transmitted, 497779 received, 0% packet loss, time 1799992ms
> rtt min/avg/max/mdev = 0.181/114.417/1915.343/246.098 ms, pipe 144, ipg/ewma 3.615/0.273 ms
>
> With netdev_budget is 1, and netdev_budget_usecs is 2000:
> 457530 packets transmitted, 457528 received, 0% packet loss, time 1799997ms
> rtt min/avg/max/mdev = 0.180/123.287/1914.186/253.883 ms, pipe 147, ipg/ewma 3.934/0.301 ms
>

Sure, but netdev_budget set to 1 is extreme, don't you think ???

Has anyone used such a setting ?

> With small netdev_budget, avg latency increases 7%, packets transmitted
> decreases 8%.
>
> > Have you tried threaded NAPI? (find files called 'threaded' in sysfs)
>
> Thanks, we had researched on threaded NAPI, much applaud for it!
> But we think someone maynot use it because some kinds of reasons.
> One is threaded NAPI is good for control, but maynot good for
> throughput, especially for those who not care real-time too much.
> Another reason is distribution kernel may too old to support
> threaded NAPI?
>
> >Well, we can't be sure if there's really nobody that uses them :(
>
> As we still retain netdev_budget*, and there maybe some using it,
> should it be improve? Netdev_budget* are sysctl for administrator,
> when administrator adjust them, they may want to see the effect in
> a direct or easy way. That's what this patch's purpose.

We prefer not changing /proc file format as much as we can, they are
deprecated/legacy.

Presumably, modern tracing techniques can let you do what you want
without adding new counters.

I think that a per-cpu counter is old-fashioned, and incurs a cost for
the vast majority of users who will
never look at the counters.