Received-SPF: pass (google.com: domain of linux-kernel-owner@vger.kernel.org designates 2620:137:e000::3:3 as permitted sender) client-ip=2620:137:e000::3:3;
Message-ID: <f7510474a007223811da8b04efabf13376e53728.camel@perches.com>
Subject: Re: [PATCH v2] get_maintainer: correctly parse UTF-8 encoded names
 in files
From:   Joe Perches <joe@perches.com>
To:     Alvin =?UTF-8?Q?=C5=A0ipraga?= <alvin@pqrs.dk>,
        Linus Torvalds <torvalds@linux-foundation.org>
Cc:     Duje =?UTF-8?Q?Mihanovi=C4=87?= <duje.mihanovic@skole.hr>,
        Konstantin Ryabitsev <konstantin@linuxfoundation.org>,
        linux-kernel@vger.kernel.org,
        Alvin =?UTF-8?Q?=C5=A0ipraga?= <alsi@bang-olufsen.dk>,
        Shawn Guo <shawnguo@kernel.org>,
        Andrew Morton <akpm@linux-foundation.org>
Date:   Thu, 14 Dec 2023 07:57:54 -0800
In-Reply-To: <20231214-get-maintainers-utf8-v2-1-b188dc7042a4@bang-olufsen.dk>
References: <20231214-get-maintainers-utf8-v2-1-b188dc7042a4@bang-olufsen.dk>
Content-Type: text/plain; charset="UTF-8"
Content-Transfer-Encoding: 8BIT
User-Agent: Evolution 3.48.4 (3.48.4-1.fc38) 
MIME-Version: 1.0
Precedence: bulk

On Thu, 2023-12-14 at 16:06 +0100, Alvin Šipraga wrote:
> From: Alvin Šipraga <alsi@bang-olufsen.dk>
> 
> While the script correctly extracts UTF-8 encoded names from the
> MAINTAINERS file, the regular expressions damage my name when parsing
> from .yaml files. Fix this by replacing the Latin-1-compatible regular
> expressions with the unicode property matcher \p{L}, which matches on
> any letter according to the Unicode General Category of letters.

OK

> It's also necessary to instruct Perl to open all files with UTF-8 encoding.

I doubt this.

> ---
> Changes in v2:
> - use '\p{L}' rather than '\p{Latin}', so that matching is even more
>   inclusive (i.e. match also Greek letters, CJK, etc.)
> - fix commit message to refer to tools mailing list, not b4 mailing list
> - Link to v1: https://lore.kernel.org/r/20231014-get-maintainers-utf8-v1-1-3af8c7aeb239@bang-olufsen.dk

OK

> diff --git a/scripts/get_maintainer.pl b/scripts/get_maintainer.pl
[]
> @@ -20,6 +20,7 @@ use Getopt::Long qw(:config no_auto_abbrev);
>  use Cwd;
>  use File::Find;
>  use File::Spec::Functions;
> +use open qw(:std :encoding(UTF-8));

I think this global use is unnecessary.


> @@ -442,7 +443,7 @@ sub maintainers_in_file {
>  	my $text = do { local($/) ; <$f> };
>  	close($f);
>  
> -	my @poss_addr = $text =~ m$[A-Za-zÀ-ÿ\"\' \,\.\+-]*\s*[\,]*\s*[\(\<\{]{0,1}[A-Za-z0-9_\.\+-]+\@[A-Za-z0-9\.-]+\.[A-Za-z0-9]+[\)\>\}]{0,1}$g;
> +	my @poss_addr = $text =~ m$[\p{L}\"\' \,\.\+-]*\s*[\,]*\s*[\(\<\{]{0,1}[A-Za-z0-9_\.\+-]+\@[A-Za-z0-9\.-]+\.[A-Za-z0-9]+[\)\>\}]{0,1}$g;
>  	push(@file_emails, clean_file_emails(@poss_addr));
>      }
>  }

Rather than open _all_ files in utf-8, perhaps the block
that opens a specific file to find maintainers

sub maintainers_in_file {
    my ($file) = @_;

    return if ($file =~ m@\bMAINTAINERS$@);

    if (-f $file && ($email_file_emails || $file =~ /\.yaml$/)) {
	open(my $f, '<', $file)
	    or die "$P: Can't open $file: $!\n";
	my $text = do { local($/) ; <$f> };
	close($f);
	...

should change the

	open(my $f...
to
	use open qw(:std :encoding(UTF-8));
	open(my $f...


And unrelated and secondarily, perhaps the
	$file =~ /\.yaml$/
test should be
	$file =~ /\.(?:yaml|dtsi?)$/

to also find any maintainer address in the dts* files

https://lore.kernel.org/lkml/20231028174656.GA3310672@bill-the-cat/T/