Received-SPF: pass (google.com: domain of linux-kernel-owner@vger.kernel.org designates 2620:137:e000::1:18 as permitted sender) client-ip=2620:137:e000::1:18;
Date:   Fri, 11 Mar 2022 16:16:56 +0100
From:   Frederic Weisbecker <frederic@kernel.org>
To:     nicolas saenz julienne <nsaenz@kernel.org>
Cc:     LKML <linux-kernel@vger.kernel.org>,
        Peter Zijlstra <peterz@infradead.org>,
        Phil Auld <pauld@redhat.com>,
        Alex Belits <abelits@marvell.com>,
        Xiongfeng Wang <wangxiongfeng2@huawei.com>,
        Neeraj Upadhyay <quic_neeraju@quicinc.com>,
        Thomas Gleixner <tglx@linutronix.de>,
        Yu Liao <liaoyu15@huawei.com>,
        Boqun Feng <boqun.feng@gmail.com>,
        "Paul E . McKenney" <paulmck@kernel.org>,
        Marcelo Tosatti <mtosatti@redhat.com>,
        Paul Gortmaker <paul.gortmaker@windriver.com>,
        Uladzislau Rezki <uladzislau.rezki@sony.com>,
        Joel Fernandes <joel@joelfernandes.org>
Subject: Re: [PATCH 19/19] context_tracking: Exempt
 CONFIG_HAVE_CONTEXT_TRACKING_USER_OFFSTACK from non-active tracking
Message-ID: <20220311151656.GA227315@lothringen>
References: <20220302154810.42308-1-frederic@kernel.org>
 <20220302154810.42308-20-frederic@kernel.org>
 <e193b98f2e6a6de2178aa0e68325e54a62c804ec.camel@kernel.org>
MIME-Version: 1.0
Content-Type: text/plain; charset=us-ascii
Content-Disposition: inline
In-Reply-To: <e193b98f2e6a6de2178aa0e68325e54a62c804ec.camel@kernel.org>
Precedence: bulk

On Tue, Mar 08, 2022 at 05:15:14PM +0100, nicolas saenz julienne wrote:
> Hi Frederic,
> 
> On Wed, 2022-03-02 at 16:48 +0100, Frederic Weisbecker wrote:
> > Since a CPU may save the state of the context tracking using
> > exception_enter() before calling into schedule(), we need all CPUs in
> > the system to track user <-> kernel transitions and not just those that
> > really need it (nohz_full CPUs).
> > 
> > The following illustrates the issue that could otherwise happen:
> > 
> >      CPU 0 (not tracking)                       CPU 1 (tracking)
> >      -------------------                       --------------------
> >      // we are past user_enter()
> >      // but this CPU is always in
> >      // CONTEXT_KERNEL
> >      // because it doesn't track user <-> kernel
> > 
> >      ctx = exception_enter(); //ctx == CONTEXT_KERNEL
> >      schedule();
> >      ===========================================>
> >                                                return from schedule();
> >                                                exception_exit(ctx);
> >                                                //go to user in CONTEXT_KERNEL
> > 
> > However CONFIG_HAVE_CONTEXT_TRACKING_USER_OFFSTACK doesn't play those
> > games because schedule() can't be called between user_enter() and
> > user_exit() under such config. In this situation we can spare context
> > tracking on the CPUs that don't need it.
> > 
> > Signed-off-by: Frederic Weisbecker <frederic@kernel.org>
> > Cc: Paul E. McKenney <paulmck@kernel.org>
> > Cc: Peter Zijlstra <peterz@infradead.org>
> > Cc: Thomas Gleixner <tglx@linutronix.de>
> > Cc: Neeraj Upadhyay <quic_neeraju@quicinc.com>
> > Cc: Uladzislau Rezki <uladzislau.rezki@sony.com>
> > Cc: Joel Fernandes <joel@joelfernandes.org>
> > Cc: Boqun Feng <boqun.feng@gmail.com>
> > Cc: Nicolas Saenz Julienne <nsaenz@kernel.org>
> > Cc: Marcelo Tosatti <mtosatti@redhat.com>
> > Cc: Xiongfeng Wang <wangxiongfeng2@huawei.com>
> > Cc: Yu Liao<liaoyu15@huawei.com>
> > Cc: Phil Auld <pauld@redhat.com>
> > Cc: Paul Gortmaker<paul.gortmaker@windriver.com>
> > Cc: Alex Belits <abelits@marvell.com>
> > ---
> >  kernel/context_tracking.c | 7 ++++---
> >  1 file changed, 4 insertions(+), 3 deletions(-)
> > 
> > diff --git a/kernel/context_tracking.c b/kernel/context_tracking.c
> > index 87e7b748791c..b1934264f77f 100644
> > --- a/kernel/context_tracking.c
> > +++ b/kernel/context_tracking.c
> > @@ -374,7 +374,7 @@ void noinstr __ct_user_enter(enum ctx_state state)
> >  			 * when the CPU runs in userspace.
> >  			 */
> >  			ct_kernel_exit(true, RCU_DYNTICKS_IDX + state);
> > -		} else {
> > +		} else if (!IS_ENABLED(CONFIG_HAVE_CONTEXT_TRACKING_USER_OFFSTACK)) {
> 
> user entry code assumes that state will be kept on all CPUs as long as context
> tracking is enabled. See kernel/entry/common.c:
> 
>    static __always_inline void __enter_from_user_mode(struct pt_regs *regs)
>    {
>            arch_check_user_regs(regs);
>            lockdep_hardirqs_off(CALLER_ADDR0);
>            
>            CT_WARN_ON(ct_state() != CONTEXT_USER); <-- NOT HAPPY ABOUT THIS
>            CHANGE

Good point!

So I need to do:

#ifdef CONFIG_HAVE_CONTEXT_TRACKING_USER_OFFSTACK
#define CT_WARN_ON(cond) WARN_ON(context_tracking_enabled_this_cpu() && (cond))
#else
#define CT_WARN_ON(cond) WARN_ON(context_tracking_enabled() && (cond))
#endif

Thanks.

>            user_exit_irqoff();
>            
>            instrumentation_begin();
>            trace_hardirqs_off_finish();
>            instrumentation_end();
>    }
> 
> Regards,
> Nicolas