Received-SPF: pass (google.com: domain of linux-kernel-owner@vger.kernel.org designates 23.128.96.36 as permitted sender) client-ip=23.128.96.36;
Message-ID: <74126e6e301d2f4a0e5a546caa54961dbc2d492c.camel@perches.com>
Subject: Re: [PATCH] checkpatch: use utf-8 match for spell checking
From:   Joe Perches <joe@perches.com>
To:     Antonio Borneo <antonio.borneo@foss.st.com>,
        Andy Whitcroft <apw@canonical.com>,
        Dwaipayan Ray <dwaipayanray1@gmail.com>,
        Lukas Bulwahn <lukas.bulwahn@gmail.com>,
        Andrew Morton <akpm@linux-foundation.org>
Cc:     linux-kernel@vger.kernel.org,
        =?ISO-8859-1?Q?Cl=E9ment_L=E9ger?= <clement.leger@bootlin.com>,
        =?ISO-8859-1?Q?Cl=E9ment?= Le Goffic 
        <clement.legoffic@foss.st.com>,
        linux-stm32@st-md-mailman.stormreply.com
Date:   Tue, 12 Dec 2023 11:07:22 -0800
In-Reply-To: <20231212094310.3633-1-antonio.borneo@foss.st.com>
References: <20231212094310.3633-1-antonio.borneo@foss.st.com>
Content-Type: text/plain; charset="ISO-8859-1"
Content-Transfer-Encoding: 8BIT
User-Agent: Evolution 3.48.4 (3.48.4-1.fc38) 
MIME-Version: 1.0
Precedence: bulk

On Tue, 2023-12-12 at 10:43 +0100, Antonio Borneo wrote:
> The current code that checks for misspelling verifies, in a more
> complex regex, if $rawline matches [^\w]($misspellings)[^\w]
> 
> Being $rawline a byte-string, a utf-8 character in $rawline can
> match the non-word-char [^\w].
> E.g.:
> 	./script/checkpatch.pl --git 81c2f059ab9
> 	WARNING: 'ment' may be misspelled - perhaps 'meant'?
> 	#36: FILE: MAINTAINERS:14360:
> 	+M:     Cl?ment L?ger <clement.leger@bootlin.com>
> 	            ^^^^
> 
> Use a utf-8 version of $rawline for spell checking.
> 
> Signed-off-by: Antonio Borneo <antonio.borneo@foss.st.com>
> Reported-by: Cl?ment Le Goffic <clement.legoffic@foss.st.com>

Seems sensible, thanks, but:

> diff --git a/scripts/checkpatch.pl b/scripts/checkpatch.pl
[]
> @@ -3477,7 +3477,8 @@ sub process {
>  # Check for various typo / spelling mistakes
>  		if (defined($misspellings) &&
>  		    ($in_commit_log || $line =~ /^(?:\+|Subject:)/i)) {
> -			while ($rawline =~ /(?:^|[^\w\-'`])($misspellings)(?:[^\w\-'`]|$)/gi) {
> +			my $rawline_utf8 = decode("utf8", $rawline);
> +			while ($rawline_utf8 =~ /(?:^|[^\w\-'`])($misspellings)(?:[^\w\-'`]|$)/gi) {
>  				my $typo = $1;
>  				my $blank = copy_spacing($rawline);

Maybe this needs to use $rawline_utf8 ?

>  				my $ptr = substr($blank, 0, $-[1]) . "^" x length($typo);

And may now the $fix bit will not always work properly