Received: by 2002:ad5:474a:0:0:0:0:0 with SMTP id i10csp1833409imu; Wed, 12 Dec 2018 05:14:33 -0800 (PST) X-Google-Smtp-Source: AFSGD/UDO2jJysRFOnDn5wvHtRoNLQ5j0EKoO91XRXegS2dMX8vcykJbxICDwglA1OzCP5WCv1gt X-Received: by 2002:a17:902:584:: with SMTP id f4mr20453251plf.28.1544620473319; Wed, 12 Dec 2018 05:14:33 -0800 (PST) ARC-Seal: i=1; a=rsa-sha256; t=1544620473; cv=none; d=google.com; s=arc-20160816; b=ZODBUqcKTcEa8x25V0jexZMfJ3nrGhHLNwX7lkoVfmvSigxK/CP4ZreBtZa7uhYLJ5 vnelvqYfKayZ9hakemL8j68ccvrLkfHy+leJn9iqmlmuqo5RDMEWoZE2ihM79eToy28j 1vf8D5KDnNu4m1f6++bt4jGHSEBrcfz5NZZW99ioT/wUmhl6Q4foq9ldGFl5AmCrZc5o vKgEVItDyqQFc+5Ru+Ztqa+EAtFWiX50zJyWILxbSGF5Cq1Cc6sR/0o6t4tj7Nk7a7ED qrRwVZPXxb59iIAd2bhozsEv1Ouvx2BEFNaNPOuXWzWJE3b8pImvNqhkHDLmvNNo5oC9 Xi6Q== ARC-Message-Signature: i=1; a=rsa-sha256; c=relaxed/relaxed; d=google.com; s=arc-20160816; h=list-id:precedence:sender:content-transfer-encoding:mime-version :message-id:date:subject:cc:to:from:dkim-signature; bh=i6qt46qUfRiiduNIF9Ae1L7GqcRy2C24+/4DIYnGDNg=; b=JopNcTmHgwrdLU40/zbXCBtfrq3xkkMHry6G+IzcvlS5uzxJYHhsbwtTzT3aRV9W0o RUnL8TEEZUF6fD3b7Oj5WzVftBVn+alOi+ant0sRaiK1rf5swF36TqWquQssUgPWAWEo BB/D2adVpTp3/guiKWIY2kYIpeZZShrDtEJtjsgFBPT9HVeKY51WJRCNEeqpoITaGqyw 6mIZuKQb2MxEf0xOcsBKs/N2fhft0nKdHEGdgzlvo6indWVeqofqsF/wNn+b1LNahLQd p0OJdfAqMt8Xg7AJTV4af1sQ3H56f2wfbMmOOqt15X2+BKXc5VikgJTzlruvwF3bivF4 0M9g== ARC-Authentication-Results: i=1; mx.google.com; dkim=pass header.i=@gmail.com header.s=20161025 header.b="REYR/oLa"; spf=pass (google.com: best guess record for domain of linux-kernel-owner@vger.kernel.org designates 209.132.180.67 as permitted sender) smtp.mailfrom=linux-kernel-owner@vger.kernel.org; dmarc=pass (p=NONE sp=QUARANTINE dis=NONE) header.from=gmail.com Return-Path: Received: from vger.kernel.org (vger.kernel.org. [209.132.180.67]) by mx.google.com with ESMTP id c24si14328640pgk.269.2018.12.12.05.14.17; Wed, 12 Dec 2018 05:14:33 -0800 (PST) Received-SPF: pass (google.com: best guess record for domain of linux-kernel-owner@vger.kernel.org designates 209.132.180.67 as permitted sender) client-ip=209.132.180.67; Authentication-Results: mx.google.com; dkim=pass header.i=@gmail.com header.s=20161025 header.b="REYR/oLa"; spf=pass (google.com: best guess record for domain of linux-kernel-owner@vger.kernel.org designates 209.132.180.67 as permitted sender) smtp.mailfrom=linux-kernel-owner@vger.kernel.org; dmarc=pass (p=NONE sp=QUARANTINE dis=NONE) header.from=gmail.com Received: (majordomo@vger.kernel.org) by vger.kernel.org via listexpand id S1727466AbeLLNMO (ORCPT + 99 others); Wed, 12 Dec 2018 08:12:14 -0500 Received: from mail-ed1-f67.google.com ([209.85.208.67]:36460 "EHLO mail-ed1-f67.google.com" rhost-flags-OK-OK-OK-OK) by vger.kernel.org with ESMTP id S1726229AbeLLNMO (ORCPT ); Wed, 12 Dec 2018 08:12:14 -0500 Received: by mail-ed1-f67.google.com with SMTP id f23so15517253edb.3 for ; Wed, 12 Dec 2018 05:12:12 -0800 (PST) DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=gmail.com; s=20161025; h=from:to:cc:subject:date:message-id:mime-version :content-transfer-encoding; bh=i6qt46qUfRiiduNIF9Ae1L7GqcRy2C24+/4DIYnGDNg=; b=REYR/oLaVVsfbiWS0j6SwDgVRBSwy/daPGUHhyaKyldlIWv/VM2jmqg2HKTRK0oYPj 7I1qcoZ6/2g3LZqjYAGziw5i9mfyqOpIqDXFSWe0wGyY9s+1BZzkC+RibtCOF2LD+WIS 170/cZo7jcm00J4Wd/VynoWb/qmEsOFss4v4s+49xkMtSuX3xXkuyPZc/VxlndghyTsH YO4tOY9NqFodlJ4qQq8/OUTKa8ZOiotK6gjLcUesbywjdo7SHFB8NI3aMl+bxOEXf0kG UtvpVmlUdtiIEaxZmybHAkKebKKnXhCcyANqR8MdNbjPpQMgh+yYNsUF+Y+NEQsmGuLG vuQQ== X-Google-DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=1e100.net; s=20161025; h=x-gm-message-state:from:to:cc:subject:date:message-id:mime-version :content-transfer-encoding; bh=i6qt46qUfRiiduNIF9Ae1L7GqcRy2C24+/4DIYnGDNg=; b=tEZ7tUedR2oM3gptXG8zvW3gdhZLUFDcQZ1Qb9HBLydNon+J0zZd2yq/IQLzKdp8nE WR9pGIPf2/BfS39agb4d7Uon3DapCoxuB1oBaX1iYDbQRMpJh87zX30Kt1UF75p4AgDm cR6mcvfbV5Rz7ubzXNzPjR5h9uDrHrxrMMhdd0dt9Mc3LBAcPThOc+cvoywPaRY9p7eo De+DSS2no0Vx/uknd7kvsXOg2aok5EC7S6HgfotAPpINLA3lEDVdq1hi/eww8xCQD4K+ 1Q2fMW1or6k97zc0ectLbRiDD2vgowvMECTTxOyRK8HbKyCaYF7IhfVR51b/781fS/Co F2HA== X-Gm-Message-State: AA+aEWYVVQYUwOh+Qz8asFcc8Xr6dxoNpolL7Gm56m8HX2gddHywebjz iQj3JVejq6wKd+MsfD8Y//Y= X-Received: by 2002:a17:906:2496:: with SMTP id e22-v6mr15141789ejb.84.1544620332178; Wed, 12 Dec 2018 05:12:12 -0800 (PST) Received: from localhost (pD9E51040.dip0.t-ipconnect.de. [217.229.16.64]) by smtp.gmail.com with ESMTPSA id f20sm4810225edf.19.2018.12.12.05.12.11 (version=TLS1_2 cipher=ECDHE-RSA-CHACHA20-POLY1305 bits=256/256); Wed, 12 Dec 2018 05:12:11 -0800 (PST) From: Thierry Reding To: Andrew Morton , Thomas Gleixner Cc: Jonathan Corbet , Joe Perches , Jeremy Cline , =?UTF-8?q?Uwe=20Kleine-K=C3=B6nig?= , linux-kernel@vger.kernel.org Subject: [PATCH 1/2] scripts/spdxcheck.py: Always open files in binary mode Date: Wed, 12 Dec 2018 14:12:09 +0100 Message-Id: <20181212131210.28024-1-thierry.reding@gmail.com> X-Mailer: git-send-email 2.19.1 MIME-Version: 1.0 Content-Transfer-Encoding: 8bit Sender: linux-kernel-owner@vger.kernel.org Precedence: bulk List-ID: X-Mailing-List: linux-kernel@vger.kernel.org From: Thierry Reding The spdxcheck script currently falls over when confronted with a binary file (such as Documentation/logo.gif). To avoid that, always open files in binary mode and decode line-by-line, ignoring encoding errors. One tricky case is when piping data into the script and reading it from standard input. By default, standard input will be opened in text mode, so we need to reopen it in binary mode. Signed-off-by: Thierry Reding --- scripts/spdxcheck.py | 6 ++++-- 1 file changed, 4 insertions(+), 2 deletions(-) diff --git a/scripts/spdxcheck.py b/scripts/spdxcheck.py index 5056fb3b897d..e559c6294c39 100755 --- a/scripts/spdxcheck.py +++ b/scripts/spdxcheck.py @@ -168,6 +168,7 @@ class id_parser(object): self.curline = 0 try: for line in fd: + line = line.decode(locale.getpreferredencoding(False), errors='ignore') self.curline += 1 if self.curline > maxlines: break @@ -249,12 +250,13 @@ if __name__ == '__main__': try: if len(args.path) and args.path[0] == '-': - parser.parse_lines(sys.stdin, args.maxlines, '-') + stdin = os.fdopen(sys.stdin.fileno(), 'rb') + parser.parse_lines(stdin, args.maxlines, '-') else: if args.path: for p in args.path: if os.path.isfile(p): - parser.parse_lines(open(p), args.maxlines, p) + parser.parse_lines(open(p, 'rb'), args.maxlines, p) elif os.path.isdir(p): scan_git_subtree(repo.head.reference.commit.tree, p) else: -- 2.19.1