Received-SPF: pass (google.com: domain of linux-kernel-owner@vger.kernel.org designates 23.128.96.18 as permitted sender) client-ip=23.128.96.18;
From:   Aditya Srivastava <yashsri421@gmail.com>
To:     joe@perches.com
Cc:     linux-kernel@vger.kernel.org,
        linux-kernel-mentees@lists.linuxfoundation.org,
        lukas.bulwahn@gmail.com, dwaipayanray1@gmail.com,
        Aditya Srivastava <yashsri421@gmail.com>
Subject: [PATCH v2] checkpatch: fix false positives in REPEATED_WORD warning
Date:   Thu, 22 Oct 2020 20:20:21 +0530
Message-Id: <20201022145021.28211-1-yashsri421@gmail.com>
Precedence: bulk

Presence of hexadecimal address or symbol results in false warning
message by checkpatch.pl.

For example, running checkpatch on commit b8ad540dd4e4 ("mptcp: fix
memory leak in mptcp_subflow_create_socket()") results in warning:

WARNING:REPEATED_WORD: Possible repeated word: 'ff'
    00 00 00 00 00 00 00 00 00 2f 30 0a 81 88 ff ff  ........./0.....

Similarly, the presence of list command output in commit results in
an unnecessary warning.

For example, running checkpatch on commit 899e5ffbf246 ("perf record:
Introduce --switch-output-event") gives:

WARNING:REPEATED_WORD: Possible repeated word: 'root'
  dr-xr-x---. 12 root root    4096 Apr 27 17:46 ..

Here, it reports 'ff' and 'root to be repeated, but it is in fact part
of some address or code, where it has to be repeated.

In these cases, the intent of the warning to find stylistic issues in
commit messages is not met and the warning is just completely wrong in
this case.

To avoid these warnings, add additional regex check for the
directory permission pattern and avoid checking the line for this
class of warning. Similarly, to avoid hex pattern, check if the word
consists of hex symbols and skip this warning if, it forms a pattern
of repeating sequence or contains more special character after pattern
than non hex characters.

A quick evaluation on v5.6..v5.8 showed that this fix reduces
REPEATED_WORD warnings from 2797 to 907.

A quick manual check found all cases are related to hex output or
list command outputs in commit messages.

Signed-off-by: Aditya Srivastava <yashsri421@gmail.com>
---
 scripts/checkpatch.pl | 33 ++++++++++++++++++++++++++++++++-
 1 file changed, 32 insertions(+), 1 deletion(-)

diff --git a/scripts/checkpatch.pl b/scripts/checkpatch.pl
index 9b9ffd876e8a..0f57e99ed670 100755
--- a/scripts/checkpatch.pl
+++ b/scripts/checkpatch.pl
@@ -3051,7 +3051,10 @@ sub process {
 		}
 
 # check for repeated words separated by a single space
-		if ($rawline =~ /^\+/ || $in_commit_log) {
+# avoid false positive from list command eg, '-rw-r--r-- 1 root root'
+		if (($rawline =~ /^\+/ || $in_commit_log) &&
+		$rawline !~ /[bcCdDlMnpPs\?-][rwxsStT-]{9}/) {
+			my @hex_seq = $rawline =~ /\b[0-9a-f]{2,} \b/g;
 			while ($rawline =~ /\b($word_pattern) (?=($word_pattern))/g) {
 
 				my $first = $1;
@@ -3065,6 +3068,34 @@ sub process {
 				next if ($first ne $second);
 				next if ($first eq 'long');
 
+				# avoid repeating hex occurrences like 'ff ff fe 09 ...'
+				if ($first =~ /\b[0-9a-f]{2,}/) {
+					# if such sequence occurs more than 4, it is most probably part of some of code
+					next if ((scalar @hex_seq)>4);
+
+					# for hex occurrences which are less than 4
+					# get first hex word in the line
+					if ($rawline =~ /\b[0-9a-f]{2,} /) {
+						my $post_hex_seq = $';
+
+						# set suffieciently high default values to avoid ignoring or counting in absence of another
+						my $non_hex_char_pos = 1000;
+						my $special_chars_pos = 500;
+
+						if ($post_hex_seq =~ /[g-z]+/) {
+							# first non hex character in post_hex_seq
+							$non_hex_char_pos = $-[0];
+						}
+						if($post_hex_seq =~ /[^a-zA-Z0-9]{2,}/) {
+							# first occurrence of 2 or more special chars
+							$special_chars_pos = $-[0];
+						}
+
+						# as such occurrences are not common, it is more likely to be a part of some code
+						next if ($special_chars_pos<$non_hex_char_pos);
+					}
+				}
+
 				if (WARN("REPEATED_WORD",
 					 "Possible repeated word: '$first'\n" . $herecurr) &&
 				    $fix) {
-- 
2.17.1