Received-SPF: pass (google.com: best guess record for domain of linux-kernel-owner@vger.kernel.org designates 209.132.180.67 as permitted sender) client-ip=209.132.180.67;
Date:   Mon, 20 Jan 2020 00:33:48 +0100
From:   Pali =?utf-8?B?Um9ow6Fy?= <pali.rohar@gmail.com>
To:     Al Viro <viro@zeniv.linux.org.uk>
Cc:     linux-kernel@vger.kernel.org, linux-fsdevel@vger.kernel.org,
        "Theodore Y. Ts'o" <tytso@mit.edu>,
        OGAWA Hirofumi <hirofumi@mail.parknet.co.jp>,
        Namjae Jeon <linkinjeon@gmail.com>,
        Gabriel Krisman Bertazi <krisman@collabora.com>
Subject: Re: vfat: Broken case-insensitive support for UTF-8
Message-ID: <20200119233348.es5m63kapdvyesal@pali>
References: <20200119221455.bac7dc55g56q2l4r@pali>
 <20200119230809.GW8904@ZenIV.linux.org.uk>
MIME-Version: 1.0
Content-Type: multipart/signed; micalg=pgp-sha1;
        protocol="application/pgp-signature"; boundary="lp2jkxbsd2wdqylt"
Content-Disposition: inline
In-Reply-To: <20200119230809.GW8904@ZenIV.linux.org.uk>
User-Agent: NeoMutt/20180716
Sender: linux-kernel-owner@vger.kernel.org
Precedence: bulk


--lp2jkxbsd2wdqylt
Content-Type: text/plain; charset=utf-8
Content-Disposition: inline
Content-Transfer-Encoding: quoted-printable

On Sunday 19 January 2020 23:08:09 Al Viro wrote:
> On Sun, Jan 19, 2020 at 11:14:55PM +0100, Pali Roh=C3=A1r wrote:
>=20
> > So when UTF-8 on VFS for VFAT is enabled, then for VFS <--> VFAT
> > conversion are used utf16s_to_utf8s() and utf8s_to_utf16s() functions.
> > But in fat_name_match(), vfat_hashi() and vfat_cmpi() functions is used
> > NLS table (default iso8859-1) with nls_strnicmp() and nls_tolower().
> >=20
> > Which means that fat_name_match(), vfat_hashi() and vfat_cmpi() are
> > broken for vfat in UTF-8 mode.
> >=20
> > I was thinking how to fix it, and the only possible way is to write a
> > uni_tolower() function which takes one Unicode code point and returns
> > lowercase of input's Unicode code point. We cannot do any Unicode
> > normalization as VFAT specification does not say anything about it and
> > MS reference fastfat.sys implementation does not do it neither.
>=20
> Then how can that possibly be broken?  If it matches the native behaviour,
> that's it.

VFAT is case insensitive.

> > As you can see lowercase 'd' and uppercase 'D' are same, but lowercase
> > '=C4=8D' and uppercase '=C4=8C' are not same. This is because '=C4=8D' =
is two bytes
> > 0xc4 0x8d sequence and comparing is done by Latin1 table. 0xc4 is in
> > Latin '=C3=84' which is already in uppercase. 0x8d is control char so i=
s not
> > changed by tolower/toupper function.
>=20
> Again, who the hell cares?

All users who use also non-Linux fat implementations.

> Does the behaviour match how Windows handles that thing?

Linux behavior does not match Windows behavior.

On Windows is FAT32 (fastfat.sys) case insensitive and file names "=C4=8D"
and "=C4=8C" are treated as same file. Windows does not allow you to create
both files. It says that file already exists.

> "Case" is not something well-defined; the only definition
> is "whatever weird crap does the native implementation choose to do".

You are right that case sensitiveness is not well-defined, but in
Unicode we have also language-independent and basically well-defined
conversion.

And because VFAT is Unicode fs (internally UTF-16) it make sense that
well-defined Unicode folding should be used.

> That's the only reason to support that garbage at all...

What do you mean by garbage? Where? All filenames which I specified are
valid UTF-8 sequences, valid Unicode code points and therefore have
valid UTF-16 representation stored in VFAT fs.

Sorry, but I did not understand your comment.

--=20
Pali Roh=C3=A1r
pali.rohar@gmail.com

--lp2jkxbsd2wdqylt
Content-Type: application/pgp-signature; name="signature.asc"

-----BEGIN PGP SIGNATURE-----

iF0EARECAB0WIQS4VrIQdKium2krgIWL8Mk9A+RDUgUCXiTnWgAKCRCL8Mk9A+RD
UnD3AKC/BUng/hJ5a6c7TzJLVXLIgm70lwCghBL7vuR3nHniaBYhpuYQuQ32okg=
=eo4j
-----END PGP SIGNATURE-----

--lp2jkxbsd2wdqylt--