Received-SPF: softfail (google.com: domain of transitioning linux-kernel-owner@vger.kernel.org does not designate 23.128.96.19 as permitted sender) client-ip=23.128.96.19;
MIME-Version: 1.0
References: <20220418043735.11441-1-patrick.wang.shcn@gmail.com>
 <20220418143404.55c8fcab@gandalf.local.home> <CAGcnep8bH4Nncbt3t+VRLff35ZyD8JyzHcEgkq-u6TEq47jqzQ@mail.gmail.com>
 <CAGcnep_Gx+3KiUvDVronYKn_divU3OM-RQEOPZakv7MRYh4EJw@mail.gmail.com>
 <20220420114454.69ab108c@gandalf.local.home> <20220420122633.55d4ff6b@gandalf.local.home>
 <CAGcnep86pd2trfTovU8_z1msOstx7gKCdndm=pHLDGTy+_K7=g@mail.gmail.com> <20220421085047.2cb8edf9@gandalf.local.home>
In-Reply-To: <20220421085047.2cb8edf9@gandalf.local.home>
From:   patrick wang <patrick.wang.shcn@gmail.com>
Date:   Fri, 22 Apr 2022 18:45:46 +0800
Message-ID: <CAGcnep9Ez-S962vukR_a=otzcSxDRo5HUW2vG_YCBanCfURGXw@mail.gmail.com>
Subject: Re: [PATCH] rcu: ftrace: avoid tracing a few functions executed in multi_cpu_stop()
To:     Steven Rostedt <rostedt@goodmis.org>
Cc:     paulmck@kernel.org, frederic@kernel.org, quic_neeraju@quicinc.com,
        josh@joshtriplett.org, mathieu.desnoyers@efficios.com,
        jiangshanlai@gmail.com, joel@joelfernandes.org,
        rcu@vger.kernel.org, linux-kernel@vger.kernel.org
Content-Type: text/plain; charset="UTF-8"
Precedence: bulk

On Thu, Apr 21, 2022 at 8:50 PM Steven Rostedt <rostedt@goodmis.org> wrote:
>
> On Thu, 21 Apr 2022 19:36:06 +0800
> patrick wang <patrick.wang.shcn@gmail.com> wrote:
>
> > On Thu, Apr 21, 2022 at 12:26 AM Steven Rostedt <rostedt@goodmis.org> wrote:
> > >
> > > On Wed, 20 Apr 2022 11:44:54 -0400
> > > Steven Rostedt <rostedt@goodmis.org> wrote:
> > >
> > >
> > > Interesting. On x86 when we used stop machine[*] it was not an issue to
> > > modify the code that is being executed in stop machine. I'm curious to
> > > exactly what the issue is if something does get traced in the stop machine
> > > processing. Why does it crash?
> >
> > I looked up the commit history. When x86 turned to breakpoints, stop
> > machine's state machine loop had no calls during waiting. So there
> > was no function being called during modifying code. Which means
> > there were no other cpus would load those instructions to be updated
> > in this period. While the stop machine currently will call other functions
> > and the call chain is fairly deep. So it may not quite be suitable for ftrace
> > now because it can not make sure the instructions within its own call
> > chain are updated atomically.
>
> Are you saying that the RISC-V stop machine implementation has code being
> called on the other CPUs while they wait?
>

Yes, and not just RISC-V. It's arch-independent. It's the internal change of
stop machine implementation.

----------------------------------------------------------------
Former stop machine wait loop:
do {
    cpu_relax(); => macro
    ...
} while (curstate != STOPMACHINE_EXIT);
-----------------------------------------------------------------
Current stop machine wait loop:
do {
    stop_machine_yield(cpumask); => function (notraced)
    ...
    touch_nmi_watchdog(); => function (notraced, inside calls also notraced)
    ...
    rcu_momentary_dyntick_idle(); => function (notraced, inside calls traced)
} while (curstate != MULTI_STOP_EXIT);
------------------------------------------------------------------

Thanks,
Patrick