Received-SPF: pass (google.com: best guess record for domain of linux-kernel-owner@vger.kernel.org designates 209.132.180.67 as permitted sender) client-ip=209.132.180.67;
From:   David Laight <David.Laight@ACULAB.COM>
To:     'Al Viro' <viro@zeniv.linux.org.uk>
CC:     =?iso-8859-1?Q?=27Pali_Roh=E1r=27?= <pali.rohar@gmail.com>,
        OGAWA Hirofumi <hirofumi@mail.parknet.co.jp>,
        "linux-kernel@vger.kernel.org" <linux-kernel@vger.kernel.org>,
        "linux-fsdevel@vger.kernel.org" <linux-fsdevel@vger.kernel.org>,
        "Theodore Y. Ts'o" <tytso@mit.edu>,
        "Namjae Jeon" <linkinjeon@gmail.com>,
        Gabriel Krisman Bertazi <krisman@collabora.com>
Subject: RE: vfat: Broken case-insensitive support for UTF-8
Thread-Topic: vfat: Broken case-insensitive support for UTF-8
Thread-Index: AQHVz4FsiOVqsS4Qp0SucuDN4afIhKfzph4wgAAFE4CAAADOMIAADbUAgAAIwlA=
Date:   Mon, 20 Jan 2020 16:51:34 +0000
Message-ID: <db8161512f33468981cbc49e71b7bf05@AcuMS.aculab.com>
References: <20200119221455.bac7dc55g56q2l4r@pali>
 <87sgkan57p.fsf@mail.parknet.co.jp> <20200120110438.ak7jpyy66clx5v6x@pali>
 <89eba9906011446f8441090f496278d2@AcuMS.aculab.com>
 <20200120152009.5vbemgmvhke4qupq@pali>
 <1a4c545dc7f14e33b7e59321a0aab868@AcuMS.aculab.com>
 <20200120161206.GC8904@ZenIV.linux.org.uk>
In-Reply-To: <20200120161206.GC8904@ZenIV.linux.org.uk>
Accept-Language: en-GB, en-US
Content-Language: en-US
MIME-Version: 1.0
Content-Type: text/plain; charset=UTF-8
Content-Transfer-Encoding: 8BIT
Sender: linux-kernel-owner@vger.kernel.org
Precedence: bulk

From: Al Viro
> Sent: 20 January 2020 16:12
> > From: Pali Rohár
> > > Sent: 20 January 2020 15:20
> > ...
> > > This is not possible. There is 1:1 mapping between UTF-8 sequence and
> > > Unicode code point. wchar_t in kernel represent either one Unicode code
> > > point (limited up to U+FFFF in NLS framework functions) or 2bytes in
> > > UTF-16 sequence (only in utf8s_to_utf16s() and utf16s_to_utf8s()
> > > functions).
> >
> > Unfortunately there is neither a 1:1 mapping of all possible byte sequences
> > to wchar_t (or unicode code points), nor a 1:1 mapping of all possible
> > wchar_t values to UTF-8.
> > Really both need to be defined - even for otherwise 'invalid' sequences.
> 
> Who.  Cares?
> 
> Filename is a sequence of octets, not codepoints.  Its interpretation is
> entirely up to the userland.

For filesystems that really ought to be true.
Saves a lot of problems in the kernel.

I guess the fat driver has to do something to convert the UCS-16 on-disk filenames
to/from a sequence of octets.

Even Microsoft have made it much easier to have case-dependant
NTS4 filesystems in windows 10.
(Ever watched the number of different cases in the list of c:/windows/system32/drivers/*.sys
filenames output when windows boots? They are nearly all different!)

-
Registered Address Lakeside, Bramley Road, Mount Farm, Milton Keynes, MK1 1PT, UK
Registration No: 1397386 (Wales)