Received-SPF: pass (google.com: best guess record for domain of linux-kernel-owner@vger.kernel.org designates 209.132.180.67 as permitted sender) client-ip=209.132.180.67;
Date:   Tue, 7 Jan 2020 12:52:02 +0100
From:   Pali =?utf-8?B?Um9ow6Fy?= <pali.rohar@gmail.com>
To:     Gabriel Krisman Bertazi <krisman@collabora.com>
Cc:     Namjae Jeon <namjae.jeon@samsung.com>,
        linux-kernel@vger.kernel.org, linux-fsdevel@vger.kernel.org,
        gregkh@linuxfoundation.org, valdis.kletnieks@vt.edu, hch@lst.de,
        sj1557.seo@samsung.com, linkinjeon@gmail.com, tytso@mit.edu
Subject: Re: [PATCH v9 10/13] exfat: add nls operations
Message-ID: <20200107115202.shjpp6g3gsrhhkuy@pali>
References: <20200102082036.29643-1-namjae.jeon@samsung.com>
 <CGME20200102082407epcas1p4cf10cd3d0ca2903707ab01b1cc523a05@epcas1p4.samsung.com>
 <20200102082036.29643-11-namjae.jeon@samsung.com>
 <20200105165115.37dyrcwtgf6zgc6r@pali>
 <85woa4jrl2.fsf@collabora.com>
MIME-Version: 1.0
Content-Type: text/plain; charset=utf-8
Content-Disposition: inline
Content-Transfer-Encoding: 8bit
In-Reply-To: <85woa4jrl2.fsf@collabora.com>
User-Agent: NeoMutt/20180716
Sender: linux-kernel-owner@vger.kernel.org
Precedence: bulk

On Monday 06 January 2020 14:46:33 Gabriel Krisman Bertazi wrote:
> Pali Rohár <pali.rohar@gmail.com> writes:
> 
> > What do you think what should kernel's exfat driver do in this case?
> >
> > To prevent such thing we need to use some kind of Unicode normalization
> > form here.
> >
> > CCing Gabriel as he was implementing some Unicode normalization for ext4
> > driver and maybe should bring some light to new exfat driver too.
> 
> We have an in-kernel implementation of the canonical decomposition
> normalization (NFD) in fs/unicode, which is what we use for f2fs and
> ext4.  It is heated argument what is the best form for filesystem usage,
> and from what I researched, every proprietary filesystem does a
> different (and crazy in their unique way) thing.
> 
> For exfat, even though the specification is quite liberal, I think the
> reasonable answer is to follow closely whatever behavior the Windows
> implementation has, whether it does normalization at all or not. Even if
> it is just an in-memory format used internally for lookups, assuming a
> different format or treating differently invalid file names can result
> in awkward results in a filesystem created on another operating system,
> like filename collisions or false misses in lookups.
> 

Hi Gabriel! Thank you for your input. AFAIK Windows exfat implementation
does not do any Unicode normalization and allow to store any sequence of
16bit numbers excluding some "bad chars" as filename (so including also
unpaired half of UTF-16 surrogate pair) if such upper cased filename
(according to upcase table stored in FS) does not conflict with another
upper cased filename already stored in directory.

So based on your suggestion, I understood that we should not do any
Unicode Normalization even just for comparing filenames if it exists.

-- 
Pali Rohár
pali.rohar@gmail.com