Received-SPF: pass (google.com: domain of linux-kernel-owner@vger.kernel.org designates 23.128.96.18 as permitted sender) client-ip=23.128.96.18;
From:   Nick Terrell <terrelln@fb.com>
To:     Arvind Sankar <nivedita@alum.mit.edu>
CC:     Nick Terrell <nickrterrell@gmail.com>,
        Borislav Petkov <bp@alien8.de>,
        Thomas Gleixner <tglx@linutronix.de>,
        linux-kernel <linux-kernel@vger.kernel.org>,
        Chris Mason <clm@fb.com>,
        "linux-kbuild@vger.kernel.org" <linux-kbuild@vger.kernel.org>,
        "x86@kernel.org" <x86@kernel.org>,
        "gregkh@linuxfoundation.org" <gregkh@linuxfoundation.org>,
        Petr Malat <oss@malat.biz>, Kees Cook <keescook@chromium.org>,
        Kernel Team <Kernel-team@fb.com>,
        Adam Borowski <kilobyte@angband.pl>,
        Patrick Williams <patrickw3@fb.com>,
        "Michael van der Westhuizen" <rmikey@fb.com>,
        "mingo@kernel.org" <mingo@kernel.org>,
        "Patrick Williams" <patrick@stwcx.xyz>,
        Sedat Dilek <sedat.dilek@gmail.com>,
        "Norbert Lange" <nolange79@gmail.com>,
        Andrew Morton <akpm@linux-foundation.org>
Subject: Re: [PATCH v6 4/8] init: add support for zstd compressed kernel
Thread-Topic: [PATCH v6 4/8] init: add support for zstd compressed kernel
Thread-Index: AQHWVBHah0xsJt/cakyKnqlA4hkEhaj8rv0AgABAWoA=
Date:   Wed, 8 Jul 2020 02:01:50 +0000
Message-ID: <BFD3E8B0-4BAA-4E3A-AF16-FB17E88BECFE@fb.com>
References: <20200707034604.1539157-1-nickrterrell@gmail.com>
 <20200707034604.1539157-5-nickrterrell@gmail.com>
 <20200707221130.GB1591079@rani.riverdale.lan>
In-Reply-To: <20200707221130.GB1591079@rani.riverdale.lan>
Accept-Language: en-US
Content-Language: en-US
Content-Type: text/plain; charset="us-ascii"
Content-ID: <A44DE559F25940449F9AB288F69D74BE@namprd15.prod.outlook.com>
Content-Transfer-Encoding: quoted-printable
MIME-Version: 1.0
X-MS-Exchange-CrossTenant-Network-Message-Id: 482ac2a4-4eb1-43f5-9d0e-08d822e2e131
X-MS-Exchange-CrossTenant-originalarrivaltime: 08 Jul 2020 02:01:50.7964
 (UTC)
X-MS-Exchange-CrossTenant-fromentityheader: Hosted
X-MS-Exchange-CrossTenant-id: 8ae927fe-1255-47a7-a2af-5f3a069daaa2
X-MS-Exchange-CrossTenant-mailboxtype: HOSTED
X-MS-Exchange-CrossTenant-userprincipalname: XZMPykZ50QKBhHmprTgdh6UUctdRTET6iw0EXyQJDkIxPGMojvBBb2NYSX3B64VK
X-MS-Exchange-Transport-CrossTenantHeadersStamped: CH2PR15MB3670
X-OriginatorOrg: fb.com
X-Proofpoint-Virus-Version: vendor=fsecure engine=2.50.10434:6.0.235,18.0.687
 definitions=2020-07-07_15:2020-07-07,2020-07-07 signatures=0
X-Proofpoint-Spam-Details: rule=fb_default_notspam policy=fb_default score=0 mlxlogscore=999
 mlxscore=0 suspectscore=0 clxscore=1011 impostorscore=0 adultscore=0
 malwarescore=0 cotscore=-2147483648 lowpriorityscore=0 phishscore=0
 spamscore=0 bulkscore=0 priorityscore=1501 classifier=spam adjust=0
 reason=mlx scancount=1 engine=8.12.0-2004280000
 definitions=main-2007080011
X-FB-Internal: deliver
Sender: linux-kernel-owner@vger.kernel.org
Precedence: bulk
List-ID: <linux-kernel.vger.kernel.org>
X-Mailing-List: linux-kernel@vger.kernel.org


> On Jul 7, 2020, at 6:11 PM, Arvind Sankar <nivedita@alum.mit.edu> wrote:
>=20
> On Mon, Jul 06, 2020 at 08:46:00PM -0700, Nick Terrell wrote:
>> From: Nick Terrell <terrelln@fb.com>
>>=20
>> * Adds the zstd cmd to scripts/Makefile.lib
>> * Adds the HAVE_KERNEL_ZSTD and KERNEL_ZSTD options
>>=20
>> Architecture specific support is still needed for decompression.
>>=20
>> diff --git a/scripts/Makefile.lib b/scripts/Makefile.lib
>> index 916b2f7f7098..d960f8815f87 100644
>> --- a/scripts/Makefile.lib
>> +++ b/scripts/Makefile.lib
>> @@ -413,6 +413,21 @@ quiet_cmd_xzkern =3D XZKERN  $@
>> quiet_cmd_xzmisc =3D XZMISC  $@
>>       cmd_xzmisc =3D cat $(real-prereqs) | $(XZ) --check=3Dcrc32 --lzma2=
=3Ddict=3D1MiB > $@
>>=20
>> +# ZSTD
>> +# ---------------------------------------------------------------------=
------
>> +# Appends the uncompressed size of the data using size_append. The .zst
>> +# format has the size information available at the beginning of the fil=
e too,
>> +# but it's in a more complex format and it's good to avoid changing the=
 part
>> +# of the boot code that reads the uncompressed size.
>> +# Note that the bytes added by size_append will make the zstd tool thin=
k that
>> +# the file is corrupt. This is expected.
>> +
>> +quiet_cmd_zstd =3D ZSTD    $@
>> +cmd_zstd =3D (cat $(filter-out FORCE,$^) | \
> 		   ^^ should just be $(real-prereqs)
>> +	zstd -19 && \
>> +        $(call size_append, $(filter-out FORCE,$^))) > $@ || \
> 	   ^^ size_append does not take arguments. It's used as just
> 	   $(size_append) and will always output shell code to print the
> 	   total size of $(real-prereqs) -- see other compressor command
> 	   definitions.

Yeah, when I wrote this code, this was how the rest of the compressors were=
 implemented.
I noticed that they had all been updated when I was responding to Norbert L=
ange earlier.
I will submit a v7 shortly with the updated command.

Thanks for the review,
Nick