Here's where bcachefs is at and what I'd like to get merged:
https://evilpiepirate.org/git/bcachefs.git/log/?h=bcachefs-for-review
Non bcachefs prep patches:
Compiler Attributes: add __flatten
locking: SIX locks (shared/intent/exclusive)
mm: export find_get_pages_range()
mm: Add a mechanism to disable faults for a specific mapping
mm: Bring back vmalloc_exec
fs: insert_inode_locked2()
fs: factor out d_mark_tmpfile()
block: Add some exports for bcachefs
block: Add blk_status_to_str()
bcache: move closures to lib/
closures: closure_wait_event()
block/bio.c | 2 +
block/blk-core.c | 13 +-
drivers/md/bcache/Kconfig | 10 +-
drivers/md/bcache/Makefile | 4 +-
drivers/md/bcache/bcache.h | 2 +-
drivers/md/bcache/super.c | 1 -
drivers/md/bcache/util.h | 3 +-
fs/dcache.c | 10 +-
fs/inode.c | 40 ++
include/linux/blkdev.h | 1 +
{drivers/md/bcache => include/linux}/closure.h | 39 +-
include/linux/compiler_attributes.h | 5 +
include/linux/dcache.h | 1 +
include/linux/fs.h | 1 +
include/linux/sched.h | 1 +
include/linux/six.h | 197 +++++++++
include/linux/vmalloc.h | 1 +
init/init_task.c | 1 +
kernel/Kconfig.locks | 3 +
kernel/locking/Makefile | 1 +
kernel/locking/six.c | 553 +++++++++++++++++++++++++
kernel/module.c | 4 +-
lib/Kconfig | 3 +
lib/Kconfig.debug | 9 +
lib/Makefile | 2 +
{drivers/md/bcache => lib}/closure.c | 35 +-
mm/filemap.c | 1 +
mm/gup.c | 7 +
mm/nommu.c | 18 +
mm/vmalloc.c | 21 +
30 files changed, 937 insertions(+), 52 deletions(-)
rename {drivers/md/bcache => include/linux}/closure.h (94%)
create mode 100644 include/linux/six.h
create mode 100644 kernel/locking/six.c
rename {drivers/md/bcache => lib}/closure.c (89%)
New since last posting that's relevant to the rest of the kernel:
- Re: the DIO cache coherency issue, we finally have a solution that hopefully
everyone will find palatable. We no longer try to do any fancy recursive
locking stuff: if userspace issues a DIO read/write where the buffer points
to the same address space as the file being read/written to, we just return
an error.
This requires a small change to gup.c, to add the check after the VMA lookup.
My patch passes the mapping to check against via a new task_struct member,
which is ugly because plumbing a new argument all the way to __get_user_pages
is also going to be ugly and if I have to do that I'm likely to go on a
refactoring binge, which gup.c looks like it needs anyways.
- vmalloc_exec() is needed because bcachefs dynamically generates x86 machine
code - per btree node unpack functions.
Bcachefs changes since last posting:
- lots
- reflink is done
- erasure coding (reed solomon raid5/6) is maturing; I have declared it ready
for beta testers and gotten some _very_ positive feedback on its performance.
- btree key cache code is done and merged, big improvements to multithreaded
write workloads
- inline data extents
- major improvements to how the btree code handles extents (still todo:
re-implement extent merging)
- huge improvements to mount/unmount times on huge filesystems
- many, many bugfixes; bug reports are slowing and the bugs that are being
reported look less and less concerning. In particular repair code is getting
better and more polished.
TODO:
- scrub, repair of replicated data when one of the replicas fail the checksum
check
- erasure coding needs repair code (it'll do reconstruct reads, but we don't
have code to rewrite bad blocks in a stripe yet. this is going to be a hassle
until we get backpointers)
- fsck isn't checking refcounts of reflinked extents yet
- bcachefs tests in ktest need to be moved to xfstests
- user docs are still very minimal
So that's roughly where things are at. I think erasure coding is going to to be
bcachefs's killer feature (or at least one of them), and I'm pretty excited
about it: it's a completely new approach unlike ZFS and btrfs, no write hole (we
don't update existing stripes in place) and we don't have to fragment writes
either like ZFS does. Add to that the caching that we already do and it's
turning into a pretty amazing tool for managing a whole bunch of mixed storage.
Hi.
On 27.10.2020 21:04, Kent Overstreet wrote:
> Here's where bcachefs is at and what I'd like to get merged:
>
> https://evilpiepirate.org/git/bcachefs.git/log/?h=bcachefs-for-review
Please excuse my ignorance if I missed things in other discussions, but
if this is what's expected to be reviewed, why the submission is not
splitted into reviewable patches?
>
> Non bcachefs prep patches:
>
> Compiler Attributes: add __flatten
> locking: SIX locks (shared/intent/exclusive)
> mm: export find_get_pages_range()
> mm: Add a mechanism to disable faults for a specific mapping
> mm: Bring back vmalloc_exec
> fs: insert_inode_locked2()
> fs: factor out d_mark_tmpfile()
> block: Add some exports for bcachefs
> block: Add blk_status_to_str()
> bcache: move closures to lib/
> closures: closure_wait_event()
>
> block/bio.c | 2 +
> block/blk-core.c | 13 +-
> drivers/md/bcache/Kconfig | 10 +-
> drivers/md/bcache/Makefile | 4 +-
> drivers/md/bcache/bcache.h | 2 +-
> drivers/md/bcache/super.c | 1 -
> drivers/md/bcache/util.h | 3 +-
> fs/dcache.c | 10 +-
> fs/inode.c | 40 ++
> include/linux/blkdev.h | 1 +
> {drivers/md/bcache => include/linux}/closure.h | 39 +-
> include/linux/compiler_attributes.h | 5 +
> include/linux/dcache.h | 1 +
> include/linux/fs.h | 1 +
> include/linux/sched.h | 1 +
> include/linux/six.h | 197 +++++++++
> include/linux/vmalloc.h | 1 +
> init/init_task.c | 1 +
> kernel/Kconfig.locks | 3 +
> kernel/locking/Makefile | 1 +
> kernel/locking/six.c | 553
> +++++++++++++++++++++++++
> kernel/module.c | 4 +-
> lib/Kconfig | 3 +
> lib/Kconfig.debug | 9 +
> lib/Makefile | 2 +
> {drivers/md/bcache => lib}/closure.c | 35 +-
> mm/filemap.c | 1 +
> mm/gup.c | 7 +
> mm/nommu.c | 18 +
> mm/vmalloc.c | 21 +
> 30 files changed, 937 insertions(+), 52 deletions(-)
> rename {drivers/md/bcache => include/linux}/closure.h (94%)
> create mode 100644 include/linux/six.h
> create mode 100644 kernel/locking/six.c
> rename {drivers/md/bcache => lib}/closure.c (89%)
>
> New since last posting that's relevant to the rest of the kernel:
> - Re: the DIO cache coherency issue, we finally have a solution that
> hopefully
> everyone will find palatable. We no longer try to do any fancy
> recursive
> locking stuff: if userspace issues a DIO read/write where the buffer
> points
> to the same address space as the file being read/written to, we just
> return
> an error.
>
> This requires a small change to gup.c, to add the check after the
> VMA lookup.
> My patch passes the mapping to check against via a new task_struct
> member,
> which is ugly because plumbing a new argument all the way to
> __get_user_pages
> is also going to be ugly and if I have to do that I'm likely to go
> on a
> refactoring binge, which gup.c looks like it needs anyways.
>
> - vmalloc_exec() is needed because bcachefs dynamically generates x86
> machine
> code - per btree node unpack functions.
>
> Bcachefs changes since last posting:
> - lots
> - reflink is done
> - erasure coding (reed solomon raid5/6) is maturing; I have declared
> it ready
> for beta testers and gotten some _very_ positive feedback on its
> performance.
> - btree key cache code is done and merged, big improvements to
> multithreaded
> write workloads
> - inline data extents
> - major improvements to how the btree code handles extents (still
> todo:
> re-implement extent merging)
> - huge improvements to mount/unmount times on huge filesystems
> - many, many bugfixes; bug reports are slowing and the bugs that are
> being
> reported look less and less concerning. In particular repair code is
> getting
> better and more polished.
>
> TODO:
> - scrub, repair of replicated data when one of the replicas fail the
> checksum
> check
> - erasure coding needs repair code (it'll do reconstruct reads, but we
> don't
> have code to rewrite bad blocks in a stripe yet. this is going to be
> a hassle
> until we get backpointers)
> - fsck isn't checking refcounts of reflinked extents yet
> - bcachefs tests in ktest need to be moved to xfstests
> - user docs are still very minimal
>
> So that's roughly where things are at. I think erasure coding is going
> to to be
> bcachefs's killer feature (or at least one of them), and I'm pretty
> excited
> about it: it's a completely new approach unlike ZFS and btrfs, no write
> hole (we
> don't update existing stripes in place) and we don't have to fragment
> writes
> either like ZFS does. Add to that the caching that we already do and
> it's
> turning into a pretty amazing tool for managing a whole bunch of mixed
> storage.
--
Oleksandr Natalenko (post-factum)
On Tue, Oct 27, 2020 at 04:04:33PM -0400, Kent Overstreet wrote:
> Here's where bcachefs is at and what I'd like to get merged:
>
> https://evilpiepirate.org/git/bcachefs.git/log/?h=bcachefs-for-review
When building for x86-32 (ie make allnoconfig, enable BLOCK, enable BCACHEFS):
../fs/bcachefs/util.h:40: warning: "memcpy" redefined
40 | #define memcpy(dst, src, len) \
../arch/x86/include/asm/string_32.h:182: note: this is the location of the previous definition
182 | #define memcpy(t, f, n) __builtin_memcpy(t, f, n)
In function ‘journal_seq_copy’,
inlined from ‘__bch2_create’ at ../fs/bcachefs/fs.c:291:3:
../arch/x86/include/asm/cmpxchg.h:128:3: error: call to ‘__cmpxchg_wrong_size’ declared with attribute error: Bad argument size for cmpxchg
which comes from:
../fs/bcachefs/fs.c:52:16: note: in expansion of macro ‘cmpxchg’
52 | } while ((v = cmpxchg(&dst->ei_journal_seq, old, journal_seq)) != old);
... and similar problems elsewhere in fs.c.