[resend, email exploded, sorry...]
Hi,
I want to discuss a few FS related topics that I haven't already seen on
the mailing lists:
* Shared pagecache pages for reflinked files (and by extension making dax
work with reflink on xfs)
* Providing a simple interface for scrubbing filesystem metadata in the
background (the online check thing). Ideally we'd make it easy to discover
what kind of metadata there is to check and provide a simple interface to
check the metadata, once discovered. This is a tricky interface topic
since FS design differs pretty widely.
* Rudimentary online repair and rebuilding (xfs) from secondary metadata
* Working out the variances in the btrfs/xfs/ocfs2/nfs reflink implementations
and making sure they all get test coverage
I would also like participate in some of the proposed discussions:
* The ext4 summit (and whatever meeting of XFS devs may happen)
* Integrating existing filesystems into pmem, or hallway bofs about designing
new filesystems for pmem
* Actually seeing the fs developers (well, everyone!) in person again :)
--Darrick
Since SMB3 protocol has at least three ways to do copy offload (server
side copy),
the reflink topic is of interest to me and likely useful to discuss
for Samba server as
well as client (cifs.ko)
On Wed, Feb 10, 2016 at 1:19 PM Darrick J. Wong <[email protected]> wrote:
>
> [resend, email exploded, sorry...]
>
> Hi,
>
> I want to discuss a few FS related topics that I haven't already seen on
> the mailing lists:
>
> * Shared pagecache pages for reflinked files (and by extension making dax
> work with reflink on xfs)
>
> * Providing a simple interface for scrubbing filesystem metadata in the
> background (the online check thing). Ideally we'd make it easy to discover
> what kind of metadata there is to check and provide a simple interface to
> check the metadata, once discovered. This is a tricky interface topic
> since FS design differs pretty widely.
>
> * Rudimentary online repair and rebuilding (xfs) from secondary metadata
>
> * Working out the variances in the btrfs/xfs/ocfs2/nfs reflink implementations
> and making sure they all get test coverage
>
> I would also like participate in some of the proposed discussions:
>
> * The ext4 summit (and whatever meeting of XFS devs may happen)
>
> * Integrating existing filesystems into pmem, or hallway bofs about designing
> new filesystems for pmem
>
> * Actually seeing the fs developers (well, everyone!) in person again :)
>
> --Darrick
> --
> To unsubscribe from this list: send the line "unsubscribe linux-fsdevel" in
> the body of a message to [email protected]
> More majordomo info at http://vger.kernel.org/majordomo-info.html
--
Thanks,
Steve
Didn't see the original mail, so reply here.
On 2020/1/21 上午8:58, Steve French wrote:
> Since SMB3 protocol has at least three ways to do copy offload (server
> side copy),
> the reflink topic is of interest to me and likely useful to discuss
> for Samba server as
> well as client (cifs.ko)
>
> On Wed, Feb 10, 2016 at 1:19 PM Darrick J. Wong <[email protected]> wrote:
>>
>> [resend, email exploded, sorry...]
>>
>> Hi,
>>
>> I want to discuss a few FS related topics that I haven't already seen on
>> the mailing lists:
>>
>> * Shared pagecache pages for reflinked files (and by extension making dax
>> work with reflink on xfs)
IIRC Goldwyn Rodrigues <[email protected]> is working on this, mostly
for btrfs, but should also apply to other fses.
>>
>> * Providing a simple interface for scrubbing filesystem metadata in the
>> background (the online check thing). Ideally we'd make it easy to discover
>> what kind of metadata there is to check and provide a simple interface to
>> check the metadata, once discovered. This is a tricky interface topic
>> since FS design differs pretty widely.
Although btrfs has already implemented scrub for a long time, and btrfs
is coming over the point where kernel can detects more problems than
btrfs-check, I still hesitate to call it "check the metadata".
It looks more like "extended metadata sanity verification", other than
full cross-ref check implemented in user-space tools.
Currently, btrfs (and I guess xfs too) can detect corrupted metadata by:
- checksum
All modern fses have similar mechanism.
- internal fields checking at read time
At least btrfs is trying to do a byte-per-byte check for each fields.
And latest such check has killed tons of fuzzed images (I guess that's
why a lot of such CVE/fuzzed image reporters only want to report on
older kernels).
But this is mostly based on the fact that btrfs on-disk fields has a
lot of redundancy. E.g. btrfs uses bytenr, not block count, allowing
us to detect bit flips in lower bits easily.
So not all fs could follow this step.
But this provides us a good centralized place to validate most tree
blocks.
Maybe other fs devs could share such details too?
- runtime sanity check across metadata boundaries
This is the traditional methods.
So is that xfs online scrub just a similar thing, or a full cross-ref check?
And if so, can we find a less confusing naming for the interface first?
>>
>> * Rudimentary online repair and rebuilding (xfs) from secondary metadata
My guess is, it's checksum and copy based.
Or just like btrfs, if checksum passes but internal checks still failed,
just try next copy?
>>
>> * Working out the variances in the btrfs/xfs/ocfs2/nfs reflink implementations
>> and making sure they all get test coverage
That would be great!
Thanks,
Qu
>>
>> I would also like participate in some of the proposed discussions:
>>
>> * The ext4 summit (and whatever meeting of XFS devs may happen)
>>
>> * Integrating existing filesystems into pmem, or hallway bofs about designing
>> new filesystems for pmem
>>
>> * Actually seeing the fs developers (well, everyone!) in person again :)
>>
>> --Darrick
>> --
>> To unsubscribe from this list: send the line "unsubscribe linux-fsdevel" in
>> the body of a message to [email protected]
>> More majordomo info at http://vger.kernel.org/majordomo-info.html
>
>
>
On Tue, Jan 21, 2020 at 3:19 AM Qu Wenruo <[email protected]> wrote:
>
> Didn't see the original mail, so reply here.
Heh! Original email was from 2016, but most of Darrick's wish list is
still relevant in 2020 :)
I for one would be very interested in getting an update on the
progress of pagecache
page sharing if there is anyone working on it.
Thanks,
Amir.
On Tue, Jan 21, 2020 at 09:35:22AM +0200, Amir Goldstein wrote:
> On Tue, Jan 21, 2020 at 3:19 AM Qu Wenruo <[email protected]> wrote:
> >
> > Didn't see the original mail, so reply here.
>
> Heh! Original email was from 2016, but most of Darrick's wish list is
> still relevant in 2020 :)
Grumble grumble stable behavior of clonerange/deduperange ioctls across
filesystems grumble grumble.
> I for one would be very interested in getting an update on the
> progress of pagecache
> page sharing if there is anyone working on it.
Me too. I guess it's the 21st, I should really send in a proposal for
*this year's* LSFMMBPFLOLBBQ.
--D
> Thanks,
> Amir.
On Tue, Jan 21, 2020 at 08:18:40AM -0800, Darrick J. Wong wrote:
> On Tue, Jan 21, 2020 at 09:35:22AM +0200, Amir Goldstein wrote:
> > On Tue, Jan 21, 2020 at 3:19 AM Qu Wenruo <[email protected]> wrote:
> > >
> > > Didn't see the original mail, so reply here.
> >
> > Heh! Original email was from 2016, but most of Darrick's wish list is
> > still relevant in 2020 :)
>
> Grumble grumble stable behavior of clonerange/deduperange ioctls across
> filesystems grumble grumble.
>
> > I for one would be very interested in getting an update on the
> > progress of pagecache
> > page sharing if there is anyone working on it.
>
> Me too. I guess it's the 21st, I should really send in a proposal for
> *this year's* LSFMMBPFLOLBBQ.
I still have Strong Opinions on how pagecache page sharing should be done
... and half a dozen more important projects ahead of it in my queue.
So I have no update on this.
On Tue, Jan 21, 2020 at 2:02 PM Matthew Wilcox <[email protected]> wrote:
>
> On Tue, Jan 21, 2020 at 08:18:40AM -0800, Darrick J. Wong wrote:
> > On Tue, Jan 21, 2020 at 09:35:22AM +0200, Amir Goldstein wrote:
> > > On Tue, Jan 21, 2020 at 3:19 AM Qu Wenruo <[email protected]> wrote:
> > > >
> > > > Didn't see the original mail, so reply here.
> > >
> > > Heh! Original email was from 2016, but most of Darrick's wish list is
> > > still relevant in 2020 :)
> >
> > Grumble grumble stable behavior of clonerange/deduperange ioctls across
> > filesystems grumble grumble.
> >
> > > I for one would be very interested in getting an update on the
> > > progress of pagecache
> > > page sharing if there is anyone working on it.
> >
> > Me too. I guess it's the 21st, I should really send in a proposal for
> > *this year's* LSFMMBPFLOLBBQ.
>
> I still have Strong Opinions on how pagecache page sharing should be done
> ... and half a dozen more important projects ahead of it in my queue.
> So I have no update on this.
We should plan to huddle on this especially if I don't get an RFC for
dax-reflink support out before the summit.
On Tue, Jan 21, 2020 at 04:47:27PM -0800, Dan Williams wrote:
> On Tue, Jan 21, 2020 at 2:02 PM Matthew Wilcox <[email protected]> wrote:
> >
> > On Tue, Jan 21, 2020 at 08:18:40AM -0800, Darrick J. Wong wrote:
> > > On Tue, Jan 21, 2020 at 09:35:22AM +0200, Amir Goldstein wrote:
> > > > On Tue, Jan 21, 2020 at 3:19 AM Qu Wenruo <[email protected]> wrote:
> > > > >
> > > > > Didn't see the original mail, so reply here.
> > > >
> > > > Heh! Original email was from 2016, but most of Darrick's wish list is
> > > > still relevant in 2020 :)
> > >
> > > Grumble grumble stable behavior of clonerange/deduperange ioctls across
> > > filesystems grumble grumble.
> > >
> > > > I for one would be very interested in getting an update on the
> > > > progress of pagecache
> > > > page sharing if there is anyone working on it.
> > >
> > > Me too. I guess it's the 21st, I should really send in a proposal for
> > > *this year's* LSFMMBPFLOLBBQ.
> >
> > I still have Strong Opinions on how pagecache page sharing should be done
> > ... and half a dozen more important projects ahead of it in my queue.
> > So I have no update on this.
>
> We should plan to huddle on this especially if I don't get an RFC for
> dax-reflink support out before the summit.
It would be a good idea to share your ideas about how you plan to
solve this problem earlier rather than later so you don't waste time
going down a path that is incompatible with what the filesystems
need to/want to/can do.
Cheers,
Dave.
--
Dave Chinner
[email protected]