Received-SPF: pass (google.com: domain of linux-kernel-owner@vger.kernel.org designates 23.128.96.33 as permitted sender) client-ip=23.128.96.33;
Message-ID: <078f662d-a73f-766b-3a07-c82cd37026c5@linux.dev>
Date:   Mon, 9 Oct 2023 18:58:27 +0800
MIME-Version: 1.0
Subject: Re: [PATCH net-next v7] net/core: Introduce netdev_core_stats_inc()
Content-Language: en-US
To:     Eric Dumazet <edumazet@google.com>
Cc:     rostedt@goodmis.org, mhiramat@kernel.org, dennis@kernel.org,
        tj@kernel.org, cl@linux.com, mark.rutland@arm.com,
        davem@davemloft.net, kuba@kernel.org, pabeni@redhat.com,
        netdev@vger.kernel.org, linux-kernel@vger.kernel.org,
        Alexander Lobakin <aleksander.lobakin@intel.com>,
        linux-trace-kernel@vger.kernel.org, linux-mm@kvack.org
References: <20231007050621.1706331-1-yajun.deng@linux.dev>
 <917708b5-cb86-f233-e878-9233c4e6c707@linux.dev>
 <CANn89i+navyRe8-AV=ehM3qFce2hmnOEKBqvK5Xnev7KTaS5Lg@mail.gmail.com>
 <a53a3ff6-8c66-07c4-0163-e582d88843dd@linux.dev>
 <CANn89i+u5dXdYm_0_LwhXg5Nw+gHXx+nPUmbYhvT=k9P4+9JRQ@mail.gmail.com>
 <9f4fb613-d63f-9b86-fe92-11bf4dfb7275@linux.dev>
 <CANn89iK7bvQtGD=p+fHaWiiaNn=u8vWrt0YQ26pGQY=kZTdfJw@mail.gmail.com>
 <4a747fda-2bb9-4231-66d6-31306184eec2@linux.dev>
 <814b5598-5284-9558-8f56-12a6f7a67187@linux.dev>
 <CANn89iJCTgWTu0mzwj-8_-HiWm4uErY=VASDHoYaod9Nq-ayPA@mail.gmail.com>
 <508b33f7-3dc0-4536-21f6-4a5e7ade2b5c@linux.dev>
 <CANn89i+r-pQGpen1mUhybmj+6ybhxSsuoaB07NFzOWyHUMFDNw@mail.gmail.com>
 <296ca17d-cff0-2d19-f620-eedab004ddde@linux.dev>
 <CANn89iL=W3fyuH_KawfhKvLyw2Cw=qhHbEZtbKgQEYhHJChy3Q@mail.gmail.com>
 <68eb65c5-1870-0776-0878-694a8b002a6d@linux.dev>
 <CANn89iJHtYJjp6zPc2PVLAWuN88BQc5OntjrAf7f6QOcqP+B=g@mail.gmail.com>
From:   Yajun Deng <yajun.deng@linux.dev>
In-Reply-To: <CANn89iJHtYJjp6zPc2PVLAWuN88BQc5OntjrAf7f6QOcqP+B=g@mail.gmail.com>
Content-Type: text/plain; charset=UTF-8; format=flowed
Content-Transfer-Encoding: 8bit
Precedence: bulk


On 2023/10/9 18:16, Eric Dumazet wrote:
> On Mon, Oct 9, 2023 at 11:43 AM Yajun Deng <yajun.deng@linux.dev> wrote:
>>
>> On 2023/10/9 17:30, Eric Dumazet wrote:
>>> On Mon, Oct 9, 2023 at 10:36 AM Yajun Deng <yajun.deng@linux.dev> wrote:
>>>> On 2023/10/9 16:20, Eric Dumazet wrote:
>>>>> On Mon, Oct 9, 2023 at 10:14 AM Yajun Deng <yajun.deng@linux.dev> wrote:
>>>>>> On 2023/10/9 15:53, Eric Dumazet wrote:
>>>>>>> On Mon, Oct 9, 2023 at 5:07 AM Yajun Deng <yajun.deng@linux.dev> wrote:
>>>>>>>
>>>>>>>> 'this_cpu_read + this_cpu_write' and 'pr_info + this_cpu_inc' will make
>>>>>>>> the trace work well.
>>>>>>>>
>>>>>>>> They all have 'pop' instructions in them. This may be the key to making
>>>>>>>> the trace work well.
>>>>>>>>
>>>>>>>> Hi all,
>>>>>>>>
>>>>>>>> I need your help on percpu and ftrace.
>>>>>>>>
>>>>>>> I do not think you made sure netdev_core_stats_inc() was never inlined.
>>>>>>>
>>>>>>> Adding more code in it is simply changing how the compiler decides to
>>>>>>> inline or not.
>>>>>> Yes, you are right. It needs to add the 'noinline' prefix. The
>>>>>> disassembly code will have 'pop'
>>>>>>
>>>>>> instruction.
>>>>>>
>>>>> The function was fine, you do not need anything like push or pop.
>>>>>
>>>>> The only needed stuff was the call __fentry__.
>>>>>
>>>>> The fact that the function was inlined for some invocations was the
>>>>> issue, because the trace point
>>>>> is only planted in the out of line function.
>>>> But somehow the following code isn't inline? They didn't need to add the
>>>> 'noinline' prefix.
>>>>
>>>> +               field = (unsigned long *)((void *)this_cpu_ptr(p) + offset);
>>>> +               WRITE_ONCE(*field, READ_ONCE(*field) + 1);
>>>>
>>>> Or
>>>> +               (*(unsigned long *)((void *)this_cpu_ptr(p) + offset))++;
>>>>
>>> I think you are very confused.
>>>
>>> You only want to trace netdev_core_stats_inc() entry point, not
>>> arbitrary pieces of it.
>>
>> Yes, I will trace netdev_core_stats_inc() entry point. I mean to replace
>>
>> +                                       field = (__force unsigned long
>> __percpu *)((__force void *)p + offset);
>> +                                       this_cpu_inc(*field);
>>
>> with
>>
>> +               field = (unsigned long *)((void *)this_cpu_ptr(p) + offset);
>> +               WRITE_ONCE(*field, READ_ONCE(*field) + 1);
>>
>> Or
>> +               (*(unsigned long *)((void *)this_cpu_ptr(p) + offset))++;
>>
>> The netdev_core_stats_inc() entry point will work fine even if it doesn't
>> have 'noinline' prefix.
>>
>> I don't know why this code needs to add 'noinline' prefix.
>> +               field = (__force unsigned long __percpu *)((__force void *)p + offset);
>> +               this_cpu_inc(*field);
>>
> C compiler decides to inline or not, depending on various factors.
>
> The most efficient (and small) code is generated by this_cpu_inc()
> version, allowing the compiler to inline it.
>
> If you copy/paste this_cpu_inc()  twenty times, then the compiler
> would  not inline the function anymore.


Got it. Thank you.