LinuxLists.cc - [PATCH 2.6.32-131.6.1.el6.x86_64] idmapd.c: deactivate the ASCII characters check

2012-02-23 09:15:30

Subject: [PATCH 2.6.32-131.6.1.el6.x86_64] idmapd.c: deactivate the ASCII characters check

Customers are using Unicode characters umlauts (?,?,?) in group names and this
creates problems with NFS ID Mapping. Groups with umlauts will be redirected to
the group NFS "nobody".
This patch deactivate the ASCII characters check. It's maybe more like a temporary fix and I think it would be nicer to
adjust the check to support Unicode characters instead to deactivate the check.

========================================================================================================

diff -Nurp nfs-utils-1.2.2/utils/idmapd/idmapd.c nfs-utils-1.2.2-fag/utils/idmapd/idmapd.c
--- nfs-utils-1.2.2/utils/idmapd/idmapd.c 2010-02-18 13:35:00.000000000 +0100
+++ nfs-utils-1.2.2-fag/utils/idmapd/idmapd.c 2011-10-05 09:09:05.815103100 +0200
@@ -850,9 +850,6 @@ validateascii(char *string, u_int32_t le
for (i = 0; i< len; i++) {
if (string[i] == '\0')
break;
-
- if (string[i]& 0x80)
- return (-1);
}

if ((i>= len) || string[i] != '\0')

=========================================================================================================

Developer's Certificate of Origin 1.1

By making a contribution to this project, I certify that:

(a) The contribution was created in whole or in part by me and I
have the right to submit it under the open source license
indicated in the file; or

(b) The contribution is based upon previous work that, to the best
of my knowledge, is covered under an appropriate open source
license and I have the right under that license to submit that
work with modifications, whether created in whole or in part
by me, under the same open source license (unless I am
permitted to submit under a different license), as indicated
in the file; or

(c) The contribution was provided directly to me by some other
person who certified (a), (b) or (c) and I have not modified
it.

(d) I understand and agree that this project and the contribution
are public and that a record of the contribution (including all
personal information I submit with it, including my sign-off) is
maintained indefinitely and may be redistributed consistent with
this project or the open source license(s) involved.

Signed-off-by: Gregor Gruener<[email protected]>

2012-02-28 19:55:37

by J. Bruce Fields

[permalink] [raw]

Subject: Re: [PATCH 2.6.32-131.6.1.el6.x86_64] idmapd.c: deactivate the ASCII characters check

On Thu, Feb 23, 2012 at 08:02:45AM -0500, Jim Rees wrote:
> Gregor Gruener wrote:
>
> Customers are using Unicode characters umlauts (ö,ä,ü) in group names and this
> creates problems with NFS ID Mapping. Groups with umlauts will be redirected to
> the group NFS "nobody".
> This patch deactivate the ASCII characters check. It's maybe more like a temporary fix and I think it would be nicer to
> adjust the check to support Unicode characters instead to deactivate the
> check.
>
> Maybe the name of the routine should also be changed to something other than
> validateascii(). But I'm curious why that check was put in there.

Sounds totally wrong.

By the nfs rfc's of course, if we're going to check anything we should
be checking for utf8.

Though even then, it's hard to see how failing and mapping to nobody
really helps anyone here once they already have a non-utf8 name.

Best might be to allow a non-utf8 mapping and print a one-time warning.
And work on fixing any account-creation tools to enforce utf8.

>
> I found this in rfc5661 section 22.1, which seems to be a bug in the spec:
>
> 1. A US-ASCII string name that is the actual name of the attribute.
> This name must be unique. This string name can be 1 to 128 UTF-8
> characters long.

Oops.

--b.

2012-02-29 21:38:51

by Jim Rees

[permalink] [raw]

Subject: Re: [PATCH 2.6.32-131.6.1.el6.x86_64] idmapd.c: deactivate the ASCII characters check

Steve Dickson wrote:

On 02/23/2012 04:16 AM, Gregor Gruener wrote:
> Customers are using Unicode characters umlauts (?,?,?) in group names and this
> creates problems with NFS ID Mapping. Groups with umlauts will be redirected to
> the group NFS "nobody".
> This patch deactivate the ASCII characters check. It's maybe more like a temporary fix and I think it would be nicer to
> adjust the check to support Unicode characters instead to deactivate the check.
> Signed-off-by: Gregor Gruener<[email protected]>
...
Committed...

I still think the name should be changed. The only thing validateascii() is
doing now is verifying that the string is null terminated.

2012-02-29 20:48:57

by Steve Dickson

[permalink] [raw]

Subject: Re: [PATCH 2.6.32-131.6.1.el6.x86_64] idmapd.c: deactivate the ASCII characters check

On 02/23/2012 04:16 AM, Gregor Gruener wrote:
> Customers are using Unicode characters umlauts (?,?,?) in group names and this
> creates problems with NFS ID Mapping. Groups with umlauts will be redirected to
> the group NFS "nobody".
> This patch deactivate the ASCII characters check. It's maybe more like a temporary fix and I think it would be nicer to
> adjust the check to support Unicode characters instead to deactivate the check.
>
> ========================================================================================================
>
> diff -Nurp nfs-utils-1.2.2/utils/idmapd/idmapd.c nfs-utils-1.2.2-fag/utils/idmapd/idmapd.c
> --- nfs-utils-1.2.2/utils/idmapd/idmapd.c 2010-02-18 13:35:00.000000000 +0100
> +++ nfs-utils-1.2.2-fag/utils/idmapd/idmapd.c 2011-10-05 09:09:05.815103100 +0200
> @@ -850,9 +850,6 @@ validateascii(char *string, u_int32_t le
> for (i = 0; i< len; i++) {
> if (string[i] == '\0')
> break;
> -
> - if (string[i]& 0x80)
> - return (-1);
> }
>
> if ((i>= len) || string[i] != '\0')
>
>
>
> =========================================================================================================
>
>
>
>
> Developer's Certificate of Origin 1.1
>
> By making a contribution to this project, I certify that:
>
> (a) The contribution was created in whole or in part by me and I
> have the right to submit it under the open source license
> indicated in the file; or
>
> (b) The contribution is based upon previous work that, to the best
> of my knowledge, is covered under an appropriate open source
> license and I have the right under that license to submit that
> work with modifications, whether created in whole or in part
> by me, under the same open source license (unless I am
> permitted to submit under a different license), as indicated
> in the file; or
>
> (c) The contribution was provided directly to me by some other
> person who certified (a), (b) or (c) and I have not modified
> it.
>
> (d) I understand and agree that this project and the contribution
> are public and that a record of the contribution (including all
> personal information I submit with it, including my sign-off) is
> maintained indefinitely and may be redistributed consistent with
> this project or the open source license(s) involved.
>
> Signed-off-by: Gregor Gruener<[email protected]>
Committed...

steved.

>
> --
> To unsubscribe from this list: send the line "unsubscribe linux-nfs" in
> the body of a message to [email protected]
> More majordomo info at http://vger.kernel.org/majordomo-info.html

2012-02-23 13:02:50

by Jim Rees

[permalink] [raw]

Subject: Re: [PATCH 2.6.32-131.6.1.el6.x86_64] idmapd.c: deactivate the ASCII characters check

Gregor Gruener wrote:

Customers are using Unicode characters umlauts (?,?,?) in group names and this
creates problems with NFS ID Mapping. Groups with umlauts will be redirected to
the group NFS "nobody".
This patch deactivate the ASCII characters check. It's maybe more like a temporary fix and I think it would be nicer to
adjust the check to support Unicode characters instead to deactivate the
check.

Maybe the name of the routine should also be changed to something other than
validateascii(). But I'm curious why that check was put in there.

I found this in rfc5661 section 22.1, which seems to be a bug in the spec:

1. A US-ASCII string name that is the actual name of the attribute.
This name must be unique. This string name can be 1 to 128 UTF-8
characters long.

2012-03-01 15:22:32

by Steve Dickson

[permalink] [raw]

Subject: Re: [PATCH 2.6.32-131.6.1.el6.x86_64] idmapd.c: deactivate the ASCII characters check

Hey,

On 02/29/2012 04:38 PM, Jim Rees wrote:
> Steve Dickson wrote:
>
> On 02/23/2012 04:16 AM, Gregor Gruener wrote:
> > Customers are using Unicode characters umlauts (?,?,?) in group names and this
> > creates problems with NFS ID Mapping. Groups with umlauts will be redirected to
> > the group NFS "nobody".
> > This patch deactivate the ASCII characters check. It's maybe more like a temporary fix and I think it would be nicer to
> > adjust the check to support Unicode characters instead to deactivate the check.
> > Signed-off-by: Gregor Gruener<[email protected]>
> ...
> Committed...
>
> I still think the name should be changed. The only thing validateascii() is
> doing now is verifying that the string is null terminated.
Sorry Jim... I kinda overlooked that part of the discussion...
I guess validateascii() is not a published interface so we
could change it... any suggestions? (Patches always welcome! ;-) )

steved.

2012-05-17 17:43:27

by Sorin Faibish

[permalink] [raw]

Subject: pNFS block performance evaluation (sorry for the wrong subject title)

Team,

These are some preliminary results with pNFS block. They look very
promising both performance and scalability. They uncovered some
instability but mainly due to HW issues; no SW issues. These results are
our prove that the latest pNFS block code is stable and performing
reasonably well.

Single client pNFS performance was 86MB/sec write performance for
reference. Thank you very much for your patience and support

We’ve finished the testing and analysis of pNFS vs. MPFS. It took much
longer than expected due to several stability issues uncovered on both the
client and the server. Some of the issues occurred during read testing,
which had never been a focus of previous testing efforts. Anyway, please
find the results below. To help with the below interpretation, please
reference the following key:

SW8xN-MPFS Sequential Write 8 clients by N threads using MPFS
(special build using latest kernel on Fedora 15)
SR8xN-MPFS Sequential Read 8 clients by N threads using MPFS
(special build on Fedora 15)
SW8xN-pNFS Sequential Write 8 clients by N threads using pNFS
(on Fedora 15)
SR8xN-pNFS Sequential Read 8 clients by N threads using pNFS
(on Fedora 15)
SW8xN-pNFS2 Sequential Write 8 clients by N threads using pNFS
(on Fedora 15)
SR8xN-pNFS2 Sequential Read 8 clients by N threads using pNFS
(on Fedora 15)
SR8xN-pNFS3 Sequential Read 8 clients by N threads using pNFS
(on Fedora 15)
SW8xN-MPFS2 Sequential Write 8 clients by N threads using MPFS
SR8xN-MPFS2 Sequential Read 8 clients by N threads using MPFS

I think/hope the rest is self-explanatory based on labels, etc. To
summarize:

pNFS2 pNFS4vs.MPFS
SW8x1 653121.85 -8%
SW8x2 835101.05 -10%
SW8x3 846188.63 -8%
SW8x4 852617.66 -6%
SR8x1 674129.88 43%
SR8x2 634800.33 34%
SR8x3 629843.20 28%
SR8x4 577197.70 19%

Let me know what you think or if you have any questions. I’m moving on to
other projects that I’ve committed to and am currently behind on.

--
--
Best Regards

Sorin Faibish
Corporate Distinguished Engineer
Fast Data Group - Office of CTO
EMC2
where information lives

Phone: 508-249-5745
Mobile: 617-510-0422
email: [email protected]

2012-05-18 09:37:34

by Peng, Tao

[permalink] [raw]

Subject: RE: pNFS block performance evaluation (sorry for the wrong subject title)

> -----Original Message-----
> From: [email protected] [mailto:[email protected]] On Behalf Of Sorin
> Faibish
> Sent: Friday, May 18, 2012 1:43 AM
> To: [email protected]
> Subject: pNFS block performance evaluation (sorry for the wrong subject title)
>
> Team,
>
> These are some preliminary results with pNFS block. They look very
> promising both performance and scalability. They uncovered some
> instability but mainly due to HW issues; no SW issues. These results are
> our prove that the latest pNFS block code is stable and performing
> reasonably well.
>
To be specific, the performance test was run with some additional patches that make client send real IO size in layoutget requests. I attached the patches in the email for your reference. These patches were sent out before and got some objections. I will rework them to address comments and send again later.

Thanks,
Tao
> Single client pNFS performance was 86MB/sec write performance for
> reference. Thank you very much for your patience and support
>
>
>
> We’ve finished the testing and analysis of pNFS vs. MPFS. It took much
> longer than expected due to several stability issues uncovered on both the
> client and the server. Some of the issues occurred during read testing,
> which had never been a focus of previous testing efforts. Anyway, please
> find the results below. To help with the below interpretation, please
> reference the following key:
>
> SW8xN-MPFS Sequential Write 8 clients by N threads using MPFS
> (special build using latest kernel on Fedora 15)
> SR8xN-MPFS Sequential Read 8 clients by N threads using MPFS
> (special build on Fedora 15)
> SW8xN-pNFS Sequential Write 8 clients by N threads using pNFS
> (on Fedora 15)
> SR8xN-pNFS Sequential Read 8 clients by N threads using pNFS
> (on Fedora 15)
> SW8xN-pNFS2 Sequential Write 8 clients by N threads using pNFS
> (on Fedora 15)
> SR8xN-pNFS2 Sequential Read 8 clients by N threads using pNFS
> (on Fedora 15)
> SR8xN-pNFS3 Sequential Read 8 clients by N threads using pNFS
> (on Fedora 15)
> SW8xN-MPFS2 Sequential Write 8 clients by N threads using MPFS
> SR8xN-MPFS2 Sequential Read 8 clients by N threads using MPFS
>
> I think/hope the rest is self-explanatory based on labels, etc. To
> summarize:
>
>
> pNFS2 pNFS4vs.MPFS
> SW8x1 653121.85 -8%
> SW8x2 835101.05 -10%
> SW8x3 846188.63 -8%
> SW8x4 852617.66 -6%
> SR8x1 674129.88 43%
> SR8x2 634800.33 34%
> SR8x3 629843.20 28%
> SR8x4 577197.70 19%
>
> Let me know what you think or if you have any questions. I’m moving on to
> other projects that I’ve committed to and am currently behind on.
>
> --
> --
> Best Regards
>
> Sorin Faibish
> Corporate Distinguished Engineer
> Fast Data Group - Office of CTO
> EMC2
> where information lives
>
> Phone: 508-249-5745
> Mobile: 617-510-0422
> email: [email protected]
> --
> To unsubscribe from this list: send the line "unsubscribe linux-nfs" in
> the body of a message to [email protected]
> More majordomo info at http://vger.kernel.org/majordomo-info.html

Attachments:

0001-nfsv41-export-pnfs_find_alloc_layout.patch (1.46 kB)
0001-nfsv41-export-pnfs_find_alloc_layout.patch 0002-nfsv41-add-and-export-pnfs_find_get_layout_locked.patch (1.77 kB)
0002-nfsv41-add-and-export-pnfs_find_get_layout_locked.patch 0003-nfsv41-get-lseg-before-issue-LD-IO-if-pgio-doesn-t-c.patch (2.89 kB)
0003-nfsv41-get-lseg-before-issue-LD-IO-if-pgio-doesn-t-c.patch 0004-pnfsblock-do-not-ask-for-layout-in-pg_init.patch (3.04 kB)
0004-pnfsblock-do-not-ask-for-layout-in-pg_init.patch Download all attachments

2012-05-17 17:40:47

by Sorin Faibish

[permalink] [raw]