Received-SPF: pass (google.com: best guess record for domain of linux-kernel-owner@vger.kernel.org designates 209.132.180.67 as permitted sender) client-ip=209.132.180.67;
Date:   Wed, 12 Dec 2018 13:14:10 -0500
From:   Jeremy Cline <jcline@redhat.com>
To:     Thierry Reding <thierry.reding@gmail.com>
Cc:     Andrew Morton <akpm@linux-foundation.org>,
        Thomas Gleixner <tglx@linutronix.de>,
        Jonathan Corbet <corbet@lwn.net>,
        Joe Perches <joe@perches.com>,
        Uwe =?utf-8?Q?Kleine-K=C3=B6nig?= 
        <u.kleine-koenig@pengutronix.de>, linux-kernel@vger.kernel.org
Subject: Re: [PATCH 1/2] scripts/spdxcheck.py: Always open files in binary
 mode
Message-ID: <20181212181410.GC2352@laptop.jcline.org>
References: <20181212131210.28024-1-thierry.reding@gmail.com>
MIME-Version: 1.0
Content-Type: text/plain; charset=us-ascii
Content-Disposition: inline
In-Reply-To: <20181212131210.28024-1-thierry.reding@gmail.com>
User-Agent: Mutt/1.10.1 (2018-07-13)
Sender: linux-kernel-owner@vger.kernel.org
Precedence: bulk

Hi,

On Wed, Dec 12, 2018 at 02:12:09PM +0100, Thierry Reding wrote:
> From: Thierry Reding <treding@nvidia.com>
> 
> The spdxcheck script currently falls over when confronted with a binary
> file (such as Documentation/logo.gif). To avoid that, always open files
> in binary mode and decode line-by-line, ignoring encoding errors.
> 
> One tricky case is when piping data into the script and reading it from
> standard input. By default, standard input will be opened in text mode,
> so we need to reopen it in binary mode.
> 
> Signed-off-by: Thierry Reding <treding@nvidia.com>
> ---
>  scripts/spdxcheck.py | 6 ++++--
>  1 file changed, 4 insertions(+), 2 deletions(-)
> 
> diff --git a/scripts/spdxcheck.py b/scripts/spdxcheck.py
> index 5056fb3b897d..e559c6294c39 100755
> --- a/scripts/spdxcheck.py
> +++ b/scripts/spdxcheck.py
> @@ -168,6 +168,7 @@ class id_parser(object):
>          self.curline = 0
>          try:
>              for line in fd:
> +                line = line.decode(locale.getpreferredencoding(False), errors='ignore')
>                  self.curline += 1
>                  if self.curline > maxlines:
>                      break
> @@ -249,12 +250,13 @@ if __name__ == '__main__':
>  
>      try:
>          if len(args.path) and args.path[0] == '-':
> -            parser.parse_lines(sys.stdin, args.maxlines, '-')
> +            stdin = os.fdopen(sys.stdin.fileno(), 'rb')
> +            parser.parse_lines(stdin, args.maxlines, '-')
>          else:
>              if args.path:
>                  for p in args.path:
>                      if os.path.isfile(p):
> -                        parser.parse_lines(open(p), args.maxlines, p)
> +                        parser.parse_lines(open(p, 'rb'), args.maxlines, p)
>                      elif os.path.isdir(p):
>                          scan_git_subtree(repo.head.reference.commit.tree, p)
>                      else:
> -- 
> 2.19.1
> 

It might be worth noting this fixes commit 6f4d29df66ac
("scripts/spdxcheck.py: make python3 compliant") and also Cc this for
stable since 6f4d29df66ac got backported to v4.19. While that commit
did indeed make the script work with Python 3 for piping data, it broke
Python 2 and made its way to stable.

Reviewed-by: Jeremy Cline <jcline@redhat.com>

Regards,
Jeremy