Return-Path: Received: (majordomo@vger.kernel.org) by vger.kernel.org via listexpand id S1753244AbdLMOUY (ORCPT ); Wed, 13 Dec 2017 09:20:24 -0500 Received: from mail.kernel.org ([198.145.29.99]:35980 "EHLO mail.kernel.org" rhost-flags-OK-OK-OK-OK) by vger.kernel.org with ESMTP id S1752175AbdLMOUW (ORCPT ); Wed, 13 Dec 2017 09:20:22 -0500 DMARC-Filter: OpenDMARC Filter v1.3.2 mail.kernel.org 825AE20C0F Authentication-Results: mail.kernel.org; dmarc=none (p=none dis=none) header.from=kernel.org Authentication-Results: mail.kernel.org; spf=none smtp.mailfrom=jlayton@kernel.org From: Jeff Layton To: linux-fsdevel@vger.kernel.org Cc: linux-kernel@vger.kernel.org, hch@lst.de, neilb@suse.de, bfields@fieldses.org, amir73il@gmail.com, jack@suse.de, viro@zeniv.linux.org.uk Subject: [PATCH 00/19] fs: rework and optimize i_version handling in filesystems Date: Wed, 13 Dec 2017 09:19:58 -0500 Message-Id: <20171213142017.23653-1-jlayton@kernel.org> X-Mailer: git-send-email 2.14.3 Sender: linux-kernel-owner@vger.kernel.org List-ID: X-Mailing-List: linux-kernel@vger.kernel.org Content-Length: 6297 Lines: 143 From: Jeff Layton About a year ago, I sent a pile of patches that overhauled how the inode->i_version field is handled in filesystems. This is a follow up to that initial series. tl;dr: I think we can greatly reduce the cost of the inode->i_version counter, by exploiting the fact that we don't need to increment it if no one is looking at it. We can also clean up the code to prepare to eventually expose this value via statx(). The inode->i_version field is supposed to be a value that changes whenever there is any data or metadata change to the inode. Some filesystems use it internally to detect directory changes during readdir. knfsd will use it if the filesystem has MS_I_VERSION set. IMA will also use it to optimize away some remeasurement if it's available. Only btrfs, ext4, and xfs implement it for data changes. Because of this, these filesystems must log the inode to disk whenever the i_version counter changes. That has a non-zero performance impact, especially on write-heavy workloads, because we end up dirtying the inode metadata on every write, not just when the times change. [1] It turns out though that none of these users of i_version require that i_version change on every change to the file. The only real requirement is that it be different if _something_ changed since the last time we queried for it. If we keep track of when something queries the value, we can avoid bumping the counter and an on-disk update when nothing else has changed if no one has queried it since it was last incremented. This patchset changes the code to only bump the i_version counter when it's strictly necessary, or when we're updating the inode metadata anyway (e.g. when times change). It takes the approach of converting the existing accessors of i_version to use a new API, while leaving the underlying implementation mostly the same. The last patch then converts the existing implementation to keep track of whether the value has been queried since it was last incremented and uses that to avoid incrementing the counter when it can. With this, we reduce inode metadata updates across all 3 filesystems down to roughly the frequency of the timestamp granularity, particularly when it's not being queried (the vastly common case). The pessimal workload here is 1 byte writes, and it helps that significantly. Of course, that's not what we'd consider a real-world workload. A tiobench-example.fio workload also shows some modest performance gains, and I've gotten mails from the kernel test robot that show some significant performance gains on some microbenchmarks (case-msync-mt in the vm-scalability testsuite to be specific), with an earlier version of this set. With larger writes, the gains with this patchset mostly vaporize, but it does not seem to cause performance to regress anywhere, AFAICT. I'm happy to run other workloads if anyone can suggest them. At this point, the patchset works and does what it's expected to do in my own testing. It seems like it's at least a modest performance win across all 3 major disk-based filesystems. It may also encourage others to implement i_version as well since it reduces the cost. [1]: On ext4 it must be turned on with the i_version mount option, mostly due to fears of incurring this impact, AFAICT. Jeff Layton (19): fs: new API for handling inode->i_version fs: don't take the i_lock in inode_inc_iversion fat: convert to new i_version API affs: convert to new i_version API afs: convert to new i_version API btrfs: convert to new i_version API exofs: switch to new i_version API ext2: convert to new i_version API ext4: convert to new i_version API nfs: convert to new i_version API nfsd: convert to new i_version API ocfs2: convert to new i_version API ufs: use new i_version API xfs: convert to new i_version API IMA: switch IMA over to new i_version API fs: only set S_VERSION when updating times if necessary xfs: avoid setting XFS_ILOG_CORE if i_version doesn't need incrementing btrfs: only dirty the inode in btrfs_update_time if something was changed fs: handle inode->i_version more efficiently fs/affs/amigaffs.c | 4 +- fs/affs/dir.c | 4 +- fs/affs/super.c | 2 +- fs/afs/fsclient.c | 2 +- fs/afs/inode.c | 4 +- fs/btrfs/delayed-inode.c | 6 +- fs/btrfs/inode.c | 11 +- fs/btrfs/tree-log.c | 3 +- fs/exofs/dir.c | 8 +- fs/exofs/super.c | 2 +- fs/ext2/dir.c | 8 +- fs/ext2/super.c | 4 +- fs/ext4/dir.c | 8 +- fs/ext4/inline.c | 6 +- fs/ext4/inode.c | 12 +- fs/ext4/ioctl.c | 2 +- fs/ext4/namei.c | 4 +- fs/ext4/super.c | 2 +- fs/ext4/xattr.c | 4 +- fs/fat/dir.c | 2 +- fs/fat/inode.c | 8 +- fs/fat/namei_msdos.c | 6 +- fs/fat/namei_vfat.c | 20 +-- fs/inode.c | 9 +- fs/nfs/delegation.c | 2 +- fs/nfs/fscache-index.c | 4 +- fs/nfs/inode.c | 16 +-- fs/nfs/nfs4proc.c | 9 +- fs/nfs/nfstrace.h | 4 +- fs/nfs/write.c | 7 +- fs/nfsd/nfsfh.h | 2 +- fs/ocfs2/dir.c | 14 +-- fs/ocfs2/inode.c | 2 +- fs/ocfs2/namei.c | 2 +- fs/ocfs2/quota_global.c | 2 +- fs/ufs/dir.c | 8 +- fs/ufs/inode.c | 2 +- fs/ufs/super.c | 2 +- fs/xfs/libxfs/xfs_inode_buf.c | 5 +- fs/xfs/xfs_icache.c | 4 +- fs/xfs/xfs_inode.c | 2 +- fs/xfs/xfs_inode_item.c | 2 +- fs/xfs/xfs_trans_inode.c | 14 ++- include/linux/fs.h | 250 ++++++++++++++++++++++++++++++++++++-- security/integrity/ima/ima_api.c | 2 +- security/integrity/ima/ima_main.c | 2 +- 46 files changed, 371 insertions(+), 127 deletions(-) -- 2.14.3