Date:   Fri, 27 Jan 2023 20:43:20 +0900
From:   Hyeonggon Yoo <42.hyeyoo@gmail.com>
To:     Michal Hocko <mhocko@suse.com>
Cc:     Vlastimil Babka <vbabka@suse.cz>,
        Andrew Morton <akpm@linux-foundation.org>,
        Christoph Lameter <cl@linux.com>,
        Pekka Enberg <penberg@kernel.org>,
        David Rientjes <rientjes@google.com>,
        Joonsoo Kim <iamjoonsoo.kim@lge.com>,
        Roman Gushchin <roman.gushchin@linux.dev>,
        Ingo Molnar <mingo@redhat.com>,
        Johannes Weiner <hannes@cmpxchg.org>,
        Shakeel Butt <shakeelb@google.com>,
        Muchun Song <muchun.song@linux.dev>,
        Matthew Wilcox <willy@infradead.org>, linux-mm@kvack.org,
        linux-kernel@vger.kernel.org,
        Peter Zijlstra <peterz@infradead.org>,
        Juri Lelli <juri.lelli@redhat.com>,
        Vincent Guittot <vincent.guittot@linaro.org>,
        Dietmar Eggemann <dietmar.eggemann@arm.com>,
        Steven Rostedt <rostedt@goodmis.org>,
        Ben Segall <bsegall@google.com>,
        Mel Gorman <mgorman@techsingularity.net>,
        Daniel Bristot de Oliveira <bristot@redhat.com>,
        Valentin Schneider <vschneid@redhat.com>,
        Dennis Zhou <dennis@kernel.org>, Tejun Heo <tj@kernel.org>
Subject: Re: [PATCH mm-unstable] lib/Kconfig.debug: do not enable
 DEBUG_PREEMPT by default
Message-ID: <Y9O42N9J9gAsbJ6T@hyeyoo>
References: <20230121033942.350387-1-42.hyeyoo@gmail.com>
 <86e68d3b-b029-5e82-5bbc-e0ccc2ae1d36@suse.cz>
 <Y8vSZ+gOFXWDKC8Z@hyeyoo>
 <Y85MNmZDc5czMRUJ@dhcp22.suse.cz>
 <Y9FNm50H+w1B26yR@hyeyoo>
 <Y9D7iXUuLsOURfFv@dhcp22.suse.cz>
MIME-Version: 1.0
Content-Type: text/plain; charset=us-ascii
Content-Disposition: inline
In-Reply-To: <Y9D7iXUuLsOURfFv@dhcp22.suse.cz>
Precedence: bulk

On Wed, Jan 25, 2023 at 10:51:05AM +0100, Michal Hocko wrote:
> On Thu 26-01-23 00:41:15, Hyeonggon Yoo wrote:
> [...]
> > > Do you happen to have any perf data collected during those runs? I
> > > would be interested in the memcg side of things. Maybe we can do
> > > something better there.
> > 
> > Yes, below is performance data I've collected.
> > 
> > 6.1.8-debug-preempt-dirty
> > =========================
> >   Overhead  Command       Shared Object     Symbol
> > +    9.14%  hackbench        [kernel.vmlinux]  [k] check_preemption_disabled
> 
> Thanks! Could you just add callers that are showing in the profile for
> this call please?

-   14.56%     9.14%  hackbench        [kernel.vmlinux]  [k] check_preemption_disabled                          
   - 6.37% check_preemption_disabled                                                                            
      + 3.48% mod_objcg_state                                                                                   
      + 1.10% obj_cgroup_charge                                                                                 
        1.02% refill_obj_stock                                                                                  
     0.67% memcg_slab_post_alloc_hook                                                                           
     0.58% mod_objcg_state      

According to perf, many memcg functions call this function
and that's because __this_cpu_xxxx checks if preemption is disabled.

in include/linux/percpu-defs.h:

/*
 * Operations for contexts that are safe from preemption/interrupts.  These
 * operations verify that preemption is disabled.
 */
#define __this_cpu_read(pcp)                                            \
({                                                                      \
        __this_cpu_preempt_check("read");                               \
        raw_cpu_read(pcp);                                              \
})

#define __this_cpu_write(pcp, val)                                      \
({                                                                      \
        __this_cpu_preempt_check("write");                              \
        raw_cpu_write(pcp, val);                                        \
})

#define __this_cpu_add(pcp, val)                                        \
({                                                                      \
        __this_cpu_preempt_check("add");                                \
        raw_cpu_add(pcp, val);                                          \
})

in lib/smp_processor_id.c:

noinstr void __this_cpu_preempt_check(const char *op)
{
        check_preemption_disabled("__this_cpu_", op);
}
EXPORT_SYMBOL(__this_cpu_preempt_check);


> -- 
> Michal Hocko
> SUSE Labs