Received-SPF: pass (google.com: domain of linux-kernel-owner@vger.kernel.org designates 23.128.96.18 as permitted sender) client-ip=23.128.96.18;
Sender: Tejun Heo <htejun@gmail.com>
Date:   Mon, 4 Oct 2021 07:44:24 -1000
From:   Tejun Heo <tj@kernel.org>
To:     Shakeel Butt <shakeelb@google.com>
Cc:     Johannes Weiner <hannes@cmpxchg.org>,
        Michal =?iso-8859-1?Q?Koutn=FD?= <mkoutny@suse.com>,
        Cgroups <cgroups@vger.kernel.org>,
        LKML <linux-kernel@vger.kernel.org>
Subject: Re: [PATCH] cgroup: rstat: optimize flush through speculative test
Message-ID: <YVs9eJnNJYwF/3f3@slm.duckdns.org>
References: <20210929235936.2859271-1-shakeelb@google.com>
 <YVszNI97NAAYpHpm@slm.duckdns.org>
 <CALvZod5OKz=7pFpxCt1CONPyJO4wR5t+PH0nzdbFBT1SYpjrsg@mail.gmail.com>
MIME-Version: 1.0
Content-Type: text/plain; charset=us-ascii
Content-Disposition: inline
In-Reply-To: <CALvZod5OKz=7pFpxCt1CONPyJO4wR5t+PH0nzdbFBT1SYpjrsg@mail.gmail.com>
Precedence: bulk

On Mon, Oct 04, 2021 at 10:25:12AM -0700, Shakeel Butt wrote:
> > > To evaluate the impact of this patch, an 8 GiB tmpfs file is created on
> > > a system with swap-on-zram and the file was pushed to swap through
> > > memory.force_empty interface. On reading the whole file, the memcg stat
> > > flush in the refault code path is triggered. With this patch, we
> > > observed 38% reduction in the read time of 8 GiB file.
> >
> > The patch looks fine to me but that's a lot of reduction in read time. Can
> > you elaborate a bit on why this makes such a huge difference? Who's hitting
> > on that lock so hard?
> 
> It was actually due to machine size. I ran a single threaded workload
> without any interference on a 112 cpus machine. So, most of the time
> the flush was acquiring and releasing the per-cpu rstat lock for empty
> trees.

Sorry for being so slow but can you point to the exact call path which gets
slowed down so significantly? I'm mostly wondering whether we need some sort
of time-batched flushes because even with lock avoidance the flush path
really isn't great when called frequently. We can mitigate it further if
necessary - e.g. by adding an "updated" bitmap so that the flusher doesn't
have to go around touching the cachelines for all the cpus.

Thanks.

-- 
tejun