Received-SPF: pass (google.com: domain of linux-kernel-owner@vger.kernel.org designates 2620:137:e000::1:20 as permitted sender) client-ip=2620:137:e000::1:20;
MIME-Version: 1.0
References: <20230530020626.186192-1-zhiyin.chen@intel.com> <20230530-wortbruch-extra-88399a74392e@brauner>
In-Reply-To: <20230530-wortbruch-extra-88399a74392e@brauner>
From:   Amir Goldstein <amir73il@gmail.com>
Date:   Tue, 30 May 2023 13:02:06 +0300
Message-ID: <CAOQ4uxhAn9JOGioLwqt0W6AvS532B5KOFzanWfPOBEuYHsDPTA@mail.gmail.com>
Subject: Re: [PATCH] fs.h: Optimize file struct to prevent false sharing
To:     chenzhiyin <zhiyin.chen@intel.com>
Cc:     viro@zeniv.linux.org.uk, linux-fsdevel@vger.kernel.org,
        linux-kernel@vger.kernel.org, nanhai.zou@intel.com,
        Christian Brauner <brauner@kernel.org>
Content-Type: text/plain; charset="UTF-8"
Content-Transfer-Encoding: quoted-printable
Precedence: bulk

On Tue, May 30, 2023 at 12:31=E2=80=AFPM Christian Brauner <brauner@kernel.=
org> wrote:
>
> On Mon, May 29, 2023 at 10:06:26PM -0400, chenzhiyin wrote:
> > In the syscall test of UnixBench, performance regression occurred
> > due to false sharing.
> >
> > The lock and atomic members, including file::f_lock, file::f_count
> > and file::f_pos_lock are highly contended and frequently updated
> > in the high-concurrency test scenarios. perf c2c indentified one
> > affected read access, file::f_op.
> > To prevent false sharing, the layout of file struct is changed as
> > following
> > (A) f_lock, f_count and f_pos_lock are put together to share the
> > same cache line.
> > (B) The read mostly members, including f_path, f_inode, f_op are
> > put into a separate cache line.
> > (C) f_mode is put together with f_count, since they are used
> > frequently at the same time.
> >
> > The optimization has been validated in the syscall test of
> > UnixBench. performance gain is 30~50%, when the number of parallel
> > jobs is 16.
> >
> > Signed-off-by: chenzhiyin <zhiyin.chen@intel.com>
> > ---
>
> Sounds interesting, but can we see the actual numbers, please?
> So struct file is marked with __randomize_layout which seems to make
> this whole reordering pointless or at least only useful if the
> structure randomization Kconfig is turned off. Is there any precedence
> to optimizing structures that are marked as randomizable?

Good question!

Also does the impressive improvement is gained only with (A)+(B)+(C)?

(A) and (B) make sense, but something about the claim (C) does not sit righ=
t.
Can you explain this claim?

Putting the read mostly f_mode with frequently updated f_count seems
counter to the goal of your patch.
Aren't f_mode and f_flags just as frequently accessed as f_op?
Shouldn't f_mode belong with the read-mostly members?

What am I missing?

Thanks,
Amir.