Hi Linus,
So sorry for a bit late pull request. I had to take a look at the test results
run during the weekend. Could you please consider this?
Thanks,
The following changes since commit 2d618bdf71635463a4aa4ad0fe46ec852292bc0c:
Merge branch 'for-linus' of git://git.kernel.org/pub/scm/linux/kernel/git/rkuo/linux-hexagon-kernel (2018-05-01 19:54:22 -0700)
are available in the Git repository at:
git://git.kernel.org/pub/scm/linux/kernel/git/jaegeuk/f2fs.git tags/f2fs-for-4.18
for you to fetch changes up to dfa742803fbbd8b82790c152600e09c5ad99caca:
f2fs: fix to clear FI_VOLATILE_FILE correctly (2018-06-04 14:33:27 -0700)
----------------------------------------------------------------
f2fs-for-4.18-rc1
In this round, we've mainly focused on discard, aka unmap, control along with
fstrim for Android-specific usage model. In addition, we've fixed writepage flow
which returned EAGAIN previously resulting in EIO of fsync(2) due to mapping's
error state. In order to avoid old MM bug [1], we decided not to use __GFP_ZERO
for the mapping for node and meta page caches. As always, we've cleaned up many
places for future fsverity and symbol conflicts.
Enhancement:
- do discard/fstrim in lower priority considering fs utilization
- split large discard commands into smaller ones for better responsiveness
- add more sanity checks to address syzbot reports
- add a mount option, fsync_mode=nobarrier, which can reduce # of cache flushes
- clean up symbol namespace with modified function names
- be strict on block allocation and IO control in corner cases
Bug fix:
- don't use __GFP_ZERO for mappings
- fix error reports in writepage to avoid fsync() failure
- avoid selinux denial on CAP_RESOURCE on resgid/resuid
- fix some subtle race conditions in GC/atomic writes/shutdown
- fix overflow bugs in sanity_check_raw_super
- fix missing bits on get_flags
Clean-up:
- prepare the generic flow for future fsverity integration
- fix some broken coding standard
[1] https://lkml.org/lkml/2018/4/8/661
----------------------------------------------------------------
Anatoly Pugachev (1):
disable loading f2fs module on PAGE_SIZE > 4KB
Chao Yu (39):
f2fs: don't split checkpoint in fstrim
f2fs: turn down IO priority of discard from background
f2fs: introduce private inode status mapping
f2fs: remove redundant block plug
f2fs: don't use GFP_ZERO for page caches
f2fs: remove unneeded F2FS_PROJINHERIT_FL
f2fs: fix to show missing bits in FS_IOC_GETFLAGS
f2fs: correct return value of f2fs_trim_fs
f2fs: fix race in between GC and atomic open
f2fs: fix return value in f2fs_ioc_commit_atomic_write
f2fs: fix to detect failure of dquot_initialize
f2fs: clean up commit_inmem_pages()
f2fs: rename dio_rwsem to i_gc_rwsem
f2fs: fix potential overflow
f2fs: introduce release_discard_addr() for cleanup
f2fs: treat volatile file's data as hot one
Revert "f2fs: add ovp valid_blocks check for bg gc victim to fg_gc"
f2fs: fix to initialize i_current_depth according to inode type
f2fs: fix to let checkpoint guarantee atomic page persistence
f2fs: fix to initialize min_mtime with ULLONG_MAX
f2fs: clean up with is_valid_blkaddr()
f2fs: detect synchronous writeback more earlier
f2fs: fix to wait page writeback during revoking atomic write
f2fs: keep migration IO order in LFS mode
f2fs: avoid stucking GC due to atomic write
f2fs: fix to don't trigger writeback during recovery
f2fs: clean up with clear_radix_tree_dirty_tag
f2fs: fix to avoid race during access gc_thread pointer
f2fs: don't drop dentry pages after fs shutdown
f2fs: fix error path of move_data_page
f2fs: fix to let caller retry allocating block address
f2fs: fix to avoid accessing cross the boundary
f2fs: make __f2fs_write_data_pages() static
f2fs: make set_de_type() static
f2fs: clean up symbol namespace
f2fs: fix to update mtime correctly
f2fs: don't change wbc->sync_mode
f2fs: let sync node IO interrupt async one
f2fs: fix to clear FI_VOLATILE_FILE correctly
Colin Ian King (1):
f2fs: fix spelling mistake: "extenstion" -> "extension"
Eric Biggers (4):
fscrypt: allow synchronous bio decryption
f2fs: refactor read path to allow multiple postprocessing steps
f2fs: call unlock_new_inode() before d_instantiate()
f2fs: fix features filename in sysfs documentation
Jaegeuk Kim (16):
Revert "f2fs: introduce f2fs_set_page_dirty_nobuffer"
f2fs: check cap_resource only for data blocks
f2fs: clear PageError on writepage
f2fs: avoid fsync() failure caused by EAGAIN in writepage()
f2fs: let fstrim issue discard commands in lower priority
f2fs: add fsync_mode=nobarrier for non-atomic files
f2fs: issue discard commands proactively in high fs utilization
f2fs: run fstrim asynchronously if runtime discard is on
f2fs: give message and set need_fsck given broken node id
f2fs: avoid bug_on on corrupted inode
f2fs: sanity check on sit entry
f2fs: sanity check for total valid node blocks
f2fs: enforce fsync_mode=strict for renamed directory
f2fs: enhance sanity_check_raw_super() to avoid potential overflows
f2fs: don't drop any page on f2fs_cp_error() case
f2fs: introduce sbi->gc_mode to determine the policy
Sahitya Tummala (1):
f2fs: Fix deadlock in shutdown ioctl
Sheng Yong (4):
f2fs: check if inmem_pages list is empty correctly
f2fs: remove duplicated dquot_initialize and fix error handling
f2fs: do not check F2FS_INLINE_DOTS in recover
f2fs: clear discard_wake earlier
Souptick Joarder (1):
fs: f2fs: Adding new return type vm_fault_t
Yunlei He (6):
f2fs: issue all big range discards in umount process
f2fs: allocate hot_data for atomic write more strictly
f2fs: stop issue discard if something wrong with f2fs
f2fs: fix missing clear FI_NO_PREALLOC in some error case
f2fs: move mnt_want_write_file after range check
f2fs: let discard thread wait a little longer if dev is busy
Yunlong Song (1):
f2fs: remove unmatched zero_user_segment when convert inline dentry
Zhikang Zhang (2):
f2fs: check cur_valid_map_mir & raw_sit block count when flush sit entries
f2fs: change le32 to le16 of f2fs_inode->i_extra_size
youngjun yoo (3):
fs: f2fs: changed variable type of offset "unsigned" to "loff_t"
fs: f2fs: add missing blank lines after declarations
fs: f2fs: insert space around that ':' and ', '
Documentation/ABI/testing/sysfs-fs-f2fs | 3 +-
Documentation/filesystems/f2fs.txt | 16 +-
fs/crypto/bio.c | 35 +-
fs/crypto/crypto.c | 8 +-
fs/crypto/fscrypt_private.h | 1 -
fs/ext4/readpage.c | 2 +-
fs/f2fs/checkpoint.c | 149 ++++----
fs/f2fs/data.c | 399 ++++++++++++--------
fs/f2fs/debug.c | 6 +
fs/f2fs/dir.c | 76 ++--
fs/f2fs/extent_cache.c | 22 +-
fs/f2fs/f2fs.h | 460 +++++++++++++----------
fs/f2fs/file.c | 335 +++++++++--------
fs/f2fs/gc.c | 183 +++++-----
fs/f2fs/gc.h | 2 -
fs/f2fs/inline.c | 78 ++--
fs/f2fs/inode.c | 115 +++---
fs/f2fs/namei.c | 75 ++--
fs/f2fs/node.c | 285 ++++++++-------
fs/f2fs/recovery.c | 65 ++--
fs/f2fs/segment.c | 621 ++++++++++++++++++--------------
fs/f2fs/segment.h | 31 +-
fs/f2fs/shrinker.c | 4 +-
fs/f2fs/super.c | 198 +++++++---
fs/f2fs/sysfs.c | 63 +++-
fs/f2fs/xattr.c | 26 +-
include/linux/fscrypt_notsupp.h | 13 +-
include/linux/fscrypt_supp.h | 5 +-
28 files changed, 1884 insertions(+), 1392 deletions(-)
On Mon, Jun 11, 2018 at 9:52 AM Jaegeuk Kim <[email protected]> wrote:
>
> In order to avoid old MM bug [1], we decided not to use __GFP_ZERO
> for the mapping for node and meta page caches.
>
> [1] https://lkml.org/lkml/2018/4/8/661
Hmm. The patch there looks correct, and anybody who relies on slab
constructors is crazy.
That email from April says it should go into stable, but it never even
went upstream..
Adding Willy to the cc, for comments, since he's been active there and
is probably the maintainer even if the MAINTAINERS file doesn't say
so..
Willy?
Linus
On Mon, Jun 11, 2018 at 9:52 AM Jaegeuk Kim <[email protected]> wrote:
>
> Eric Biggers (4):
> f2fs: call unlock_new_inode() before d_instantiate()
Btw, in my merge, this got superseded by commit 1e2e547a93a00ebc2158
("do d_instantiate/unlock_new_inode combinations safely"), because the
ordering of that d_instantiate() and unlock_new_inode() was subtler
than just doing the unlock first.
I can't test the end result, so you guys should double-check whatever
xfstest you ran originall, ok?
Linus
On 06/11, Linus Torvalds wrote:
> On Mon, Jun 11, 2018 at 9:52 AM Jaegeuk Kim <[email protected]> wrote:
> >
> > In order to avoid old MM bug [1], we decided not to use __GFP_ZERO
> > for the mapping for node and meta page caches.
> >
> > [1] https://lkml.org/lkml/2018/4/8/661
>
> Hmm. The patch there looks correct, and anybody who relies on slab
> constructors is crazy.
>
> That email from April says it should go into stable, but it never even
> went upstream..
Heh, I forgot to link the latest results. The MM fix was merged by:
abc1be13fd11 ("mm/filemap.c: fix NULL pointer in page_cache_tree_insert()")
Thanks,
>
> Adding Willy to the cc, for comments, since he's been active there and
> is probably the maintainer even if the MAINTAINERS file doesn't say
> so..
>
> Willy?
>
> Linus
On Mon, Jun 11, 2018 at 10:40 AM Jaegeuk Kim <[email protected]> wrote:
>
> Heh, I forgot to link the latest results. The MM fix was merged by:
Ahh. Different fix. I did check that that original patch you mentioned
wasn't merged, and didn't react to the other patches.
Thanks,
Linus
Argh, you used my Outlook email address, so you get top-posting ????
The radix tree does rely on constructors still. One of the things on my todo list is to verify whether it actually makes sense to do that or not. It may continue to make sense; the radix tree node is 576 bytes (9 cache lines) and the normal usage of a freshly allocated radix tree node will touch only one or two cachelines. Radix tree nodes naturally come back to the cache zeroed, so there's no cache pollution when destroying them either.
We didn't send that patch upstream; instead we said that anybody who specifies __GFP_ZERO on a slab with constructors is crazy and fixed it with commits abc1be13fd113ddef5e2d807a466286b864caed3 and 128227e7fe4087b60f1bd31f762e61237eb23790
I don't really understand why Jaegeuk is changing f2fs now when we've just fixed the core kernel to make what he's doing perfectly legal.
> -----Original Message-----
> From: Linus Torvalds <[email protected]>
> Sent: Monday, June 11, 2018 1:28 PM
> To: Jaegeuk Kim <[email protected]>; Matthew Wilcox
> <[email protected]>
> Cc: Linux F2FS DEV, Mailing List <[email protected]>; Linux
> Kernel Mailing List <[email protected]>
> Subject: Re: [GIT PULL] f2fs update for 4.18-rc1
>
> On Mon, Jun 11, 2018 at 9:52 AM Jaegeuk Kim <[email protected]> wrote:
> >
> > In order to avoid old MM bug [1], we decided not to use __GFP_ZERO
> > for the mapping for node and meta page caches.
> >
> > [1]
> https://na01.safelinks.protection.outlook.com/?url=https%3A%2F%2Flkml.org
> %2Flkml%2F2018%2F4%2F8%2F661&data=02%7C01%7Cmawilcox%40microso
> ft.com%7Ce69c1f9a8d6a44bda36e08d5cfc0b1a5%7C72f988bf86f141af91ab2d
> 7cd011db47%7C1%7C0%7C636643348874608385&sdata=xbAg0cOMxyrY4NY
> ysTj1X0DPTdxwT24oYu1cxO45nkQ%3D&reserved=0
>
> Hmm. The patch there looks correct, and anybody who relies on slab
> constructors is crazy.
>
> That email from April says it should go into stable, but it never even
> went upstream..
>
> Adding Willy to the cc, for comments, since he's been active there and
> is probably the maintainer even if the MAINTAINERS file doesn't say
> so..
>
> Willy?
>
> Linus
On 06/11, Linus Torvalds wrote:
> On Mon, Jun 11, 2018 at 9:52 AM Jaegeuk Kim <[email protected]> wrote:
> >
> > Eric Biggers (4):
> > f2fs: call unlock_new_inode() before d_instantiate()
>
> Btw, in my merge, this got superseded by commit 1e2e547a93a00ebc2158
> ("do d_instantiate/unlock_new_inode combinations safely"), because the
> ordering of that d_instantiate() and unlock_new_inode() was subtler
> than just doing the unlock first.
>
> I can't test the end result, so you guys should double-check whatever
> xfstest you ran originall, ok?
https://lkml.org/lkml/2018/5/13/248
Got it. Thanks.
>
> Linus
On 06/11, Matthew Wilcox wrote:
> Argh, you used my Outlook email address, so you get top-posting ????
:P
>
> The radix tree does rely on constructors still. One of the things on my todo list is to verify whether it actually makes sense to do that or not. It may continue to make sense; the radix tree node is 576 bytes (9 cache lines) and the normal usage of a freshly allocated radix tree node will touch only one or two cachelines. Radix tree nodes naturally come back to the cache zeroed, so there's no cache pollution when destroying them either.
>
> We didn't send that patch upstream; instead we said that anybody who specifies __GFP_ZERO on a slab with constructors is crazy and fixed it with commits abc1be13fd113ddef5e2d807a466286b864caed3 and 128227e7fe4087b60f1bd31f762e61237eb23790
>
> I don't really understand why Jaegeuk is changing f2fs now when we've just fixed the core kernel to make what he's doing perfectly legal.
Cause Sometimes I can't wait for propagation of MM-stable patches into old
-stable kernels, going into android common kernels [1], in order to resolve
huge number of kernel panic reports in time. Instead, since I keep backporting
mainline f2fs into them, I'd prefer to change f2fs to address it quickly in this
case -- not always tho.
[1] 3.18, 4.4, 4.9, and 4.14 in:
https://android.googlesource.com/kernel/common/
>
> > -----Original Message-----
> > From: Linus Torvalds <[email protected]>
> > Sent: Monday, June 11, 2018 1:28 PM
> > To: Jaegeuk Kim <[email protected]>; Matthew Wilcox
> > <[email protected]>
> > Cc: Linux F2FS DEV, Mailing List <[email protected]>; Linux
> > Kernel Mailing List <[email protected]>
> > Subject: Re: [GIT PULL] f2fs update for 4.18-rc1
> >
> > On Mon, Jun 11, 2018 at 9:52 AM Jaegeuk Kim <[email protected]> wrote:
> > >
> > > In order to avoid old MM bug [1], we decided not to use __GFP_ZERO
> > > for the mapping for node and meta page caches.
> > >
> > > [1]
> > https://na01.safelinks.protection.outlook.com/?url=https%3A%2F%2Flkml.org
> > %2Flkml%2F2018%2F4%2F8%2F661&data=02%7C01%7Cmawilcox%40microso
> > ft.com%7Ce69c1f9a8d6a44bda36e08d5cfc0b1a5%7C72f988bf86f141af91ab2d
> > 7cd011db47%7C1%7C0%7C636643348874608385&sdata=xbAg0cOMxyrY4NY
> > ysTj1X0DPTdxwT24oYu1cxO45nkQ%3D&reserved=0
> >
> > Hmm. The patch there looks correct, and anybody who relies on slab
> > constructors is crazy.
> >
> > That email from April says it should go into stable, but it never even
> > went upstream..
> >
> > Adding Willy to the cc, for comments, since he's been active there and
> > is probably the maintainer even if the MAINTAINERS file doesn't say
> > so..
> >
> > Willy?
> >
> > Linus
On Mon, Jun 11, 2018 at 11:21 AM Matthew Wilcox <[email protected]> wrote:
>
> Argh, you used my Outlook email address, so you get top-posting
Hey, I think it's what you uise for your commit messages, I think
that's where I got it from..
> The radix tree does rely on constructors still. One of the things on my todo list is to verify whether it actually makes sense to do that or not.
Ok, thanks.
> I don't really understand why Jaegeuk is changing f2fs now when we've just fixed the core kernel to make what he's doing perfectly legal.
I'm assuming it was based on the state when it wasn't legal yet.
Linus