Hello all,
I'm interested in implementing the draft specification for NFS v4.2 as a
Google Summer of Code project. That includes server-side copying, sparse
files, and carrying fadvise() calls through to the server, among other
things.
The current document can be found here:
http://tools.ietf.org/html/draft-ietf-nfsv4-minorversion2-07
Is this something that you need to be done? If so, I'd very much like to
be involved. :)
Thanks for your time,
Calvin Owens
On 04/13/2012 06:05 PM, Dean wrote:
>>
>>
>>
>> The efficient sparse file read and fadvise support might be nice too,
>> but I'd like to see numbers for how they improve matters before I feel
>> comfortable saying yea or nay to adding those specific features.
>
> Instead of '...but I'd like to see...' I assume you meant, '...since I'd like to see...', as it would be hard to see how they improve matters without actually implementing them.
>
> Although for sparse file reads, I think the low overhead design to avoid bloating every file upon read in NFSv4.2 makes it easy to see the benefit without any implementation. For ioadvise, I think the benefit will be dependent upon support from the exported file system (or possibly nfs server). So any prototype would have to be combined with appropriate support in the server file system or nfs server.
>
> Dean
Well, that would make my project a lot more straightforward: if you guys want the ADB sparse file reads, I could just call it "Implementing Sparse File Support for NFS", and implement sections 3, 4, and 6 of the v4.2 internet-draft. That seems like enough.
fadvise() has the same behaviour for all filesystems, doesn't it? If wanting more efficient VM virtual disks over NFS is driving the sparse file stuff, I can see how POSIX_FADV_RANDOM on the server end would help with that.
Calvin
On 04/05/2012 10:17 PM, Myklebust, Trond wrote:
> On Thu, 2012-04-05 at 19:03 -0500, Calvin Owens wrote:
>> Hello all,
>>
>> I'm interested in implementing the draft specification for NFS v4.2 as a
>> Google Summer of Code project. That includes server-side copying, sparse
>> files, and carrying fadvise() calls through to the server, among other
>> things.
>>
>> The current document can be found here:
>> http://tools.ietf.org/html/draft-ietf-nfsv4-minorversion2-07
>>
>> Is this something that you need to be done? If so, I'd very much like to
>> be involved. :)
>
> Hi Calvin,
>
> Labelled NFS is likely to be merged into 3.5 (if Dave Q finds the time
> to port his existing code).
>
> Copy offload already exists in prototype form. The main remaining issue
> is working out the user syscall interface, which really requires getting
> all the interested filesystem maintainers to agree (we've started on
> doing that).
>
> If you'd like to contribute, then I'd suggest looking into SEEK (for
> providing lseek(SEEK_HOLE/SEEK_DATA) support. There is also the hole
> punching/space reservation, that should fit nicely into the fallocate()
> system call.
Yes, that sounds good. I'll start looking into that.
> The efficient sparse file read and fadvise support might be nice too,
> but I'd like to see numbers for how they improve matters before I feel
> comfortable saying yea or nay to adding those specific features.
I definitely see your point. It does seem to be a lot of added
complexity for a negligible benefit.
> Note that there are also a bunch of NFSv4.1 features that have yet to be
> implemented, so the above list of tasks is not exhaustive. I'd be happy
> to work with you to find something...
What else, precisely, would you like to get done? I need to get some
more detailed objectives to put into my application. (Forgive me for so
blatantly not looking myself, but it seems more efficient than me
digging through the code finding missing features and lobbing them at
you for approval, when I'm sure you know exactly where your priorities
are...)
Thanks very much,
Calvin Owens
On 04/09/2012 20:27, Calvin Owens wrote:
> On 04/05/2012 10:17 PM, Myklebust, Trond wrote:
>> On Thu, 2012-04-05 at 19:03 -0500, Calvin Owens wrote:
>>> Hello all,
>>>
>>> I'm interested in implementing the draft specification for NFS v4.2
>>> as a
>>> Google Summer of Code project. That includes server-side copying,
>>> sparse
>>> files, and carrying fadvise() calls through to the server, among
>>> other
>>> things.
>>>
>>> The current document can be found here:
>>> http://tools.ietf.org/html/draft-ietf-nfsv4-minorversion2-07
>>>
>>> Is this something that you need to be done? If so, I'd very much
>>> like to
>>> be involved. :)
>>
>> Hi Calvin,
>>
>> Labelled NFS is likely to be merged into 3.5 (if Dave Q finds the
>> time
>> to port his existing code).
>>
>> Copy offload already exists in prototype form. The main remaining
>> issue
>> is working out the user syscall interface, which really requires
>> getting
>> all the interested filesystem maintainers to agree (we've started on
>> doing that).
>>
>> If you'd like to contribute, then I'd suggest looking into SEEK (for
>> providing lseek(SEEK_HOLE/SEEK_DATA) support. There is also the hole
>> punching/space reservation, that should fit nicely into the
>> fallocate()
>> system call.
>
> Yes, that sounds good. I'll start looking into that.
>
>> The efficient sparse file read and fadvise support might be nice
>> too,
>> but I'd like to see numbers for how they improve matters before I
>> feel
>> comfortable saying yea or nay to adding those specific features.
>
> I definitely see your point. It does seem to be a lot of added
> complexity for a negligible benefit.
>
>> Note that there are also a bunch of NFSv4.1 features that have yet
>> to be
>> implemented, so the above list of tasks is not exhaustive. I'd be
>> happy
>> to work with you to find something...
>
> What else, precisely, would you like to get done? I need to get some
> more detailed objectives to put into my application. (Forgive me for
> so blatantly not looking myself, but it seems more efficient than me
> digging through the code finding missing features and lobbing them at
> you for approval, when I'm sure you know exactly where your
> priorities
> are...)
>
> Thanks very much,
> Calvin Owens
> --
> To unsubscribe from this list: send the line "unsubscribe linux-nfs"
> in
> the body of a message to [email protected]
> More majordomo info at http://vger.kernel.org/majordomo-info.html
Finishing up the port of Labeled NFS would be some low hanging fruit if
someone wants to help tackle it. I'll be trying to find time to work on
it but its unclear how soon I'll be able to get around to it.
Dave
On 4/10/2012 7:32 PM, Calvin Owens wrote:
> On 04/09/2012 07:34 PM, David Quigley wrote:
>> On 04/09/2012 20:27, Calvin Owens wrote:
>>> On 04/05/2012 10:17 PM, Myklebust, Trond wrote:
>>>> On Thu, 2012-04-05 at 19:03 -0500, Calvin Owens wrote:
>>>>> Hello all,
>>>>>
>>>>> I'm interested in implementing the draft specification for NFS v4.2
>>>>> as a
>>>>> Google Summer of Code project. That includes server-side copying,
>>>>> sparse
>>>>> files, and carrying fadvise() calls through to the server, among other
>>>>> things.
>>>>>
>>>>> The current document can be found here:
>>>>> http://tools.ietf.org/html/draft-ietf-nfsv4-minorversion2-07
>>>>>
>>>>> Is this something that you need to be done? If so, I'd very much
>>>>> like to
>>>>> be involved. :)
>>>>
>>>> Hi Calvin,
>>>>
>>>> Labelled NFS is likely to be merged into 3.5 (if Dave Q finds the time
>>>> to port his existing code).
>>>>
>>>> Copy offload already exists in prototype form. The main remaining issue
>>>> is working out the user syscall interface, which really requires
>>>> getting
>>>> all the interested filesystem maintainers to agree (we've started on
>>>> doing that).
>>>>
>>>> If you'd like to contribute, then I'd suggest looking into SEEK (for
>>>> providing lseek(SEEK_HOLE/SEEK_DATA) support. There is also the hole
>>>> punching/space reservation, that should fit nicely into the fallocate()
>>>> system call.
>>>
>>> Yes, that sounds good. I'll start looking into that.
>>>
>>>> The efficient sparse file read and fadvise support might be nice too,
>>>> but I'd like to see numbers for how they improve matters before I feel
>>>> comfortable saying yea or nay to adding those specific features.
>>>
>>> I definitely see your point. It does seem to be a lot of added
>>> complexity for a negligible benefit.
>>>
>>>> Note that there are also a bunch of NFSv4.1 features that have yet
>>>> to be
>>>> implemented, so the above list of tasks is not exhaustive. I'd be happy
>>>> to work with you to find something...
>>>
>>> What else, precisely, would you like to get done? I need to get some
>>> more detailed objectives to put into my application. (Forgive me for
>>> so blatantly not looking myself, but it seems more efficient than me
>>> digging through the code finding missing features and lobbing them at
>>> you for approval, when I'm sure you know exactly where your priorities
>>> are...)
>>>
>>> Thanks very much,
>>> Calvin Owens
>>> --
>>> To unsubscribe from this list: send the line "unsubscribe linux-nfs" in
>>> the body of a message to [email protected]
>>> More majordomo info at http://vger.kernel.org/majordomo-info.html
>>
>> Finishing up the port of Labeled NFS would be some low hanging fruit if
>> someone wants to help tackle it. I'll be trying to find time to work on
>> it but its unclear how soon I'll be able to get around to it.
>>
>> Dave
>
> Okay. I took a look at your lnfs git tree - just to make sure I
> understand: the patchset was originally against 2.6.33, and you're
> working to make it apply to the current kernel, with the same
> functionality? And stuff has changed enough since then that it's not
> trivial? Or am I completely missing the point?
>
> Thanks,
> Calvin Owens
> --
> To unsubscribe from this list: send the line "unsubscribe linux-nfs" in
> the body of a message to [email protected]
> More majordomo info at http://vger.kernel.org/majordomo-info.html
>
Sorry I somehow completely missed this email. The latest tree should be
against 3.2 or something like that. Currently there is an issue where I
couldn't port one of the parts over in lookup (see a previous email I
sent to trond on list for the details). Once that issue is fixed I would
assume the forward port to 3.5 would be mostly trivial.
The Labeled NFS tree is managed as two git trees for people's
convenience. The tree listed first below [1] is the guilt patch set
which I keep track of. The second tree is a git tree which has had the
patches applied to it[2]. The tree currently has a v3.2-lnfs branch
which needs the one fix figured out. The way I do development is to
check out the kernel tag and the patch-set and just work from there.
Another issue is that currently, SELinux is really the only thing you
can use to test Labeled NFS. I have a setup script which should give a
small test environment to mess around with as well. If you're not
familiar with SELinux I can help you with that part as well.
If you find this interesting and want to try to fix the issue and do the
forward port I'd be glad to help you. It would be nice to get these
changes upstream.
[1]
http://git.selinuxproject.org/git/?p=users/dpquigl/lnfs-patchset/.git;a=summary
[2] http://git.selinuxproject.org/git/?p=users/dpquigl/lnfs/.git;a=summary
Dave
On 04/12/2012 01:15 AM, Dave Quigley wrote:
> On 4/10/2012 7:32 PM, Calvin Owens wrote:
>> On 04/09/2012 07:34 PM, David Quigley wrote:
>>> On 04/09/2012 20:27, Calvin Owens wrote:
>>>> On 04/05/2012 10:17 PM, Myklebust, Trond wrote:
>>>>> On Thu, 2012-04-05 at 19:03 -0500, Calvin Owens wrote:
>>>>>> Hello all,
>>>>>>
>>>>>> I'm interested in implementing the draft specification for NFS v4.2
>>>>>> as a
>>>>>> Google Summer of Code project. That includes server-side copying,
>>>>>> sparse
>>>>>> files, and carrying fadvise() calls through to the server, among other
>>>>>> things.
>>>>>>
>>>>>> The current document can be found here:
>>>>>> http://tools.ietf.org/html/draft-ietf-nfsv4-minorversion2-07
>>>>>>
>>>>>> Is this something that you need to be done? If so, I'd very much
>>>>>> like to
>>>>>> be involved. :)
>>>>>
>>>>> Hi Calvin,
>>>>>
>>>>> Labelled NFS is likely to be merged into 3.5 (if Dave Q finds the time
>>>>> to port his existing code).
>>>>>
>>>>> Copy offload already exists in prototype form. The main remaining issue
>>>>> is working out the user syscall interface, which really requires
>>>>> getting
>>>>> all the interested filesystem maintainers to agree (we've started on
>>>>> doing that).
>>>>>
>>>>> If you'd like to contribute, then I'd suggest looking into SEEK (for
>>>>> providing lseek(SEEK_HOLE/SEEK_DATA) support. There is also the hole
>>>>> punching/space reservation, that should fit nicely into the fallocate()
>>>>> system call.
>>>>
>>>> Yes, that sounds good. I'll start looking into that.
>>>>
>>>>> The efficient sparse file read and fadvise support might be nice too,
>>>>> but I'd like to see numbers for how they improve matters before I feel
>>>>> comfortable saying yea or nay to adding those specific features.
>>>>
>>>> I definitely see your point. It does seem to be a lot of added
>>>> complexity for a negligible benefit.
>>>>
>>>>> Note that there are also a bunch of NFSv4.1 features that have yet
>>>>> to be
>>>>> implemented, so the above list of tasks is not exhaustive. I'd be happy
>>>>> to work with you to find something...
>>>>
>>>> What else, precisely, would you like to get done? I need to get some
>>>> more detailed objectives to put into my application. (Forgive me for
>>>> so blatantly not looking myself, but it seems more efficient than me
>>>> digging through the code finding missing features and lobbing them at
>>>> you for approval, when I'm sure you know exactly where your priorities
>>>> are...)
>>>>
>>>> Thanks very much,
>>>> Calvin Owens
>>>> --
>>>> To unsubscribe from this list: send the line "unsubscribe linux-nfs" in
>>>> the body of a message to [email protected]
>>>> More majordomo info at http://vger.kernel.org/majordomo-info.html
>>>
>>> Finishing up the port of Labeled NFS would be some low hanging fruit if
>>> someone wants to help tackle it. I'll be trying to find time to work on
>>> it but its unclear how soon I'll be able to get around to it.
>>>
>>> Dave
>>
>> Okay. I took a look at your lnfs git tree - just to make sure I
>> understand: the patchset was originally against 2.6.33, and you're
>> working to make it apply to the current kernel, with the same
>> functionality? And stuff has changed enough since then that it's not
>> trivial? Or am I completely missing the point?
>>
>> Thanks,
>> Calvin Owens
>> --
>> To unsubscribe from this list: send the line "unsubscribe linux-nfs" in
>> the body of a message to [email protected]
>> More majordomo info at http://vger.kernel.org/majordomo-info.html
>>
>
> Sorry I somehow completely missed this email. The latest tree should be against 3.2 or something like that. Currently there is an issue where I couldn't port one of the parts over in lookup (see a previous email I sent to trond on list for the details). Once that issue is fixed I would assume the forward port to 3.5 would be mostly trivial.
>
> The Labeled NFS tree is managed as two git trees for people's convenience. The tree listed first below [1] is the guilt patch set which I keep track of. The second tree is a git tree which has had the patches applied to it[2]. The tree currently has a v3.2-lnfs branch which needs the one fix figured out. The way I do development is to check out the kernel tag and the patch-set and just work from there.
>
> Another issue is that currently, SELinux is really the only thing you can use to test Labeled NFS. I have a setup script which should give a small test environment to mess around with as well. If you're not familiar with SELinux I can help you with that part as well.
>
> If you find this interesting and want to try to fix the issue and do the forward port I'd be glad to help you. It would be nice to get these changes upstream.
>
>
> [1] http://git.selinuxproject.org/git/?p=users/dpquigl/lnfs-patchset/.git;a=summary
> [2] http://git.selinuxproject.org/git/?p=users/dpquigl/lnfs/.git;a=summary
>
> Dave
Well, after spending some time looking at it, I think it's probably better left to somebody more familiar with SELinux... I have absolutely no experience with it whatsoever.
Thanks very much for the info though.
Calvin
>
>
>
> The efficient sparse file read and fadvise support might be nice too,
> but I'd like to see numbers for how they improve matters before I feel
> comfortable saying yea or nay to adding those specific features.
Instead of '...but I'd like to see...' I assume you meant, '...since I'd
like to see...', as it would be hard to see how they improve matters
without actually implementing them.
Although for sparse file reads, I think the low overhead design to avoid
bloating every file upon read in NFSv4.2 makes it easy to see the
benefit without any implementation. For ioadvise, I think the benefit
will be dependent upon support from the exported file system (or
possibly nfs server). So any prototype would have to be combined with
appropriate support in the server file system or nfs server.
Dean
On 04/09/2012 07:34 PM, David Quigley wrote:
> On 04/09/2012 20:27, Calvin Owens wrote:
>> On 04/05/2012 10:17 PM, Myklebust, Trond wrote:
>>> On Thu, 2012-04-05 at 19:03 -0500, Calvin Owens wrote:
>>>> Hello all,
>>>>
>>>> I'm interested in implementing the draft specification for NFS v4.2
>>>> as a
>>>> Google Summer of Code project. That includes server-side copying,
>>>> sparse
>>>> files, and carrying fadvise() calls through to the server, among other
>>>> things.
>>>>
>>>> The current document can be found here:
>>>> http://tools.ietf.org/html/draft-ietf-nfsv4-minorversion2-07
>>>>
>>>> Is this something that you need to be done? If so, I'd very much
>>>> like to
>>>> be involved. :)
>>>
>>> Hi Calvin,
>>>
>>> Labelled NFS is likely to be merged into 3.5 (if Dave Q finds the time
>>> to port his existing code).
>>>
>>> Copy offload already exists in prototype form. The main remaining issue
>>> is working out the user syscall interface, which really requires getting
>>> all the interested filesystem maintainers to agree (we've started on
>>> doing that).
>>>
>>> If you'd like to contribute, then I'd suggest looking into SEEK (for
>>> providing lseek(SEEK_HOLE/SEEK_DATA) support. There is also the hole
>>> punching/space reservation, that should fit nicely into the fallocate()
>>> system call.
>>
>> Yes, that sounds good. I'll start looking into that.
>>
>>> The efficient sparse file read and fadvise support might be nice too,
>>> but I'd like to see numbers for how they improve matters before I feel
>>> comfortable saying yea or nay to adding those specific features.
>>
>> I definitely see your point. It does seem to be a lot of added
>> complexity for a negligible benefit.
>>
>>> Note that there are also a bunch of NFSv4.1 features that have yet to be
>>> implemented, so the above list of tasks is not exhaustive. I'd be happy
>>> to work with you to find something...
>>
>> What else, precisely, would you like to get done? I need to get some
>> more detailed objectives to put into my application. (Forgive me for
>> so blatantly not looking myself, but it seems more efficient than me
>> digging through the code finding missing features and lobbing them at
>> you for approval, when I'm sure you know exactly where your priorities
>> are...)
>>
>> Thanks very much,
>> Calvin Owens
>> --
>> To unsubscribe from this list: send the line "unsubscribe linux-nfs" in
>> the body of a message to [email protected]
>> More majordomo info at http://vger.kernel.org/majordomo-info.html
>
> Finishing up the port of Labeled NFS would be some low hanging fruit if
> someone wants to help tackle it. I'll be trying to find time to work on
> it but its unclear how soon I'll be able to get around to it.
>
> Dave
Okay. I took a look at your lnfs git tree - just to make sure I
understand: the patchset was originally against 2.6.33, and you're
working to make it apply to the current kernel, with the same
functionality? And stuff has changed enough since then that it's not
trivial? Or am I completely missing the point?
Thanks,
Calvin Owens
T24gVGh1LCAyMDEyLTA0LTA1IGF0IDE5OjAzIC0wNTAwLCBDYWx2aW4gT3dlbnMgd3JvdGU6DQo+
IEhlbGxvIGFsbCwNCj4gDQo+IEknbSBpbnRlcmVzdGVkIGluIGltcGxlbWVudGluZyB0aGUgZHJh
ZnQgc3BlY2lmaWNhdGlvbiBmb3IgTkZTIHY0LjIgYXMgYSANCj4gR29vZ2xlIFN1bW1lciBvZiBD
b2RlIHByb2plY3QuIFRoYXQgaW5jbHVkZXMgc2VydmVyLXNpZGUgY29weWluZywgc3BhcnNlIA0K
PiBmaWxlcywgYW5kIGNhcnJ5aW5nIGZhZHZpc2UoKSBjYWxscyB0aHJvdWdoIHRvIHRoZSBzZXJ2
ZXIsIGFtb25nIG90aGVyIA0KPiB0aGluZ3MuDQo+IA0KPiBUaGUgY3VycmVudCBkb2N1bWVudCBj
YW4gYmUgZm91bmQgaGVyZToNCj4gaHR0cDovL3Rvb2xzLmlldGYub3JnL2h0bWwvZHJhZnQtaWV0
Zi1uZnN2NC1taW5vcnZlcnNpb24yLTA3DQo+IA0KPiBJcyB0aGlzIHNvbWV0aGluZyB0aGF0IHlv
dSBuZWVkIHRvIGJlIGRvbmU/IElmIHNvLCBJJ2QgdmVyeSBtdWNoIGxpa2UgdG8gDQo+IGJlIGlu
dm9sdmVkLiA6KQ0KDQpIaSBDYWx2aW4sDQoNCkxhYmVsbGVkIE5GUyBpcyBsaWtlbHkgdG8gYmUg
bWVyZ2VkIGludG8gMy41IChpZiBEYXZlIFEgZmluZHMgdGhlIHRpbWUNCnRvIHBvcnQgaGlzIGV4
aXN0aW5nIGNvZGUpLg0KDQpDb3B5IG9mZmxvYWQgYWxyZWFkeSBleGlzdHMgaW4gcHJvdG90eXBl
IGZvcm0uIFRoZSBtYWluIHJlbWFpbmluZyBpc3N1ZQ0KaXMgd29ya2luZyBvdXQgdGhlIHVzZXIg
c3lzY2FsbCBpbnRlcmZhY2UsIHdoaWNoIHJlYWxseSByZXF1aXJlcyBnZXR0aW5nDQphbGwgdGhl
IGludGVyZXN0ZWQgZmlsZXN5c3RlbSBtYWludGFpbmVycyB0byBhZ3JlZSAod2UndmUgc3RhcnRl
ZCBvbg0KZG9pbmcgdGhhdCkuDQoNCklmIHlvdSdkIGxpa2UgdG8gY29udHJpYnV0ZSwgdGhlbiBJ
J2Qgc3VnZ2VzdCBsb29raW5nIGludG8gU0VFSyAoZm9yDQpwcm92aWRpbmcgbHNlZWsoU0VFS19I
T0xFL1NFRUtfREFUQSkgc3VwcG9ydC4gVGhlcmUgaXMgYWxzbyB0aGUgaG9sZQ0KcHVuY2hpbmcv
c3BhY2UgcmVzZXJ2YXRpb24sIHRoYXQgc2hvdWxkIGZpdCBuaWNlbHkgaW50byB0aGUgZmFsbG9j
YXRlKCkNCnN5c3RlbSBjYWxsLg0KVGhlIGVmZmljaWVudCBzcGFyc2UgZmlsZSByZWFkIGFuZCBm
YWR2aXNlIHN1cHBvcnQgbWlnaHQgYmUgbmljZSB0b28sDQpidXQgSSdkIGxpa2UgdG8gc2VlIG51
bWJlcnMgZm9yIGhvdyB0aGV5IGltcHJvdmUgbWF0dGVycyBiZWZvcmUgSSBmZWVsDQpjb21mb3J0
YWJsZSBzYXlpbmcgeWVhIG9yIG5heSB0byBhZGRpbmcgdGhvc2Ugc3BlY2lmaWMgZmVhdHVyZXMu
DQoNCk5vdGUgdGhhdCB0aGVyZSBhcmUgYWxzbyBhIGJ1bmNoIG9mIE5GU3Y0LjEgZmVhdHVyZXMg
dGhhdCBoYXZlIHlldCB0byBiZQ0KaW1wbGVtZW50ZWQsIHNvIHRoZSBhYm92ZSBsaXN0IG9mIHRh
c2tzIGlzIG5vdCBleGhhdXN0aXZlLiBJJ2QgYmUgaGFwcHkNCnRvIHdvcmsgd2l0aCB5b3UgdG8g
ZmluZCBzb21ldGhpbmcuLi4NCg0KQ2hlZXJzDQogIFRyb25kDQotLSANClRyb25kIE15a2xlYnVz
dA0KTGludXggTkZTIGNsaWVudCBtYWludGFpbmVyDQoNCk5ldEFwcA0KVHJvbmQuTXlrbGVidXN0
QG5ldGFwcC5jb20NCnd3dy5uZXRhcHAuY29tDQoNCg==