Received: by 2002:a25:8b12:0:0:0:0:0 with SMTP id i18csp3625828ybl; Mon, 19 Aug 2019 23:07:33 -0700 (PDT) X-Google-Smtp-Source: APXvYqxBKhdS3g4vyOx0bX9haWjX6Xk+jRfqgKrxw0PM2rs4CuthFGc/2ixwtRRCYfeeTJb2CKsw X-Received: by 2002:a62:4d41:: with SMTP id a62mr9768057pfb.155.1566281253461; Mon, 19 Aug 2019 23:07:33 -0700 (PDT) ARC-Seal: i=1; a=rsa-sha256; t=1566281253; cv=none; d=google.com; s=arc-20160816; b=z6MqrevVispFMk6JYaeHhJ4mVaw8MTInLuy+V4ii2omuEcH9muq32h7VpGQTKMx+UM LqlWmo4dl/oeewScyFXNwefHbbr/K8v+dhhhgfJPyLkyXf5VfacL+fQD3k2oN+NoH5hj UdjR+T0Y/qiXDEE0HZa4GixvQoT6Opr2msP5d+YnnAy7BH5quomYtv16cXprkWBXOD2f cGx3MVSJqAjzgSNwb/XGcp3ouVrtYYlNDyBTAVbiSjcV1tP359j/JsVcOKjeSJeBl8pf pev//B79y4+oqeU/Hbaj+zahw8A+e4LOV9UOgZxTmBJAREuBYZGDYXUUamdOAah+NIt0 Vl2A== ARC-Message-Signature: i=1; a=rsa-sha256; c=relaxed/relaxed; d=google.com; s=arc-20160816; h=list-id:precedence:sender:in-reply-to:mime-version:user-agent:date :message-id:autocrypt:openpgp:from:references:cc:to:subject :dkim-signature; bh=rsOUMtqEiAJDaEleZCjSosHK8l4q5p9SY49MtGcO9/I=; b=Imjn4vdm8wC0P+NkLWHsH/2zHbuEIVIJXFhIkdmf/bD8FGAT9lTTCPIBORAMbsSCd6 nYdCNRmIKZ2kVqL/DcJ3yqTrTJUOYJAapRCr+3iTWoNliCtTddnLIOqb90bDf3jvKjsw 4M97YaRoiHDw9Zv6DeNhlwJb7VtUCYFbU+SJEMfvmeFlNm8Et8u+gEDIqqLDK3fw63El jnXsTJBgmT4NM+s4Q2WQCmgRMR/E1jaxz2HppkNPKPj0WujUhGjG2LThpkqRMsk4TEnF s1yXyufO/RPYPI6jig/xPWpH9idMAnUvqOp/eau2jjpNmCdvz5y5WLrEbnfoJPMGzsqd SR7A== ARC-Authentication-Results: i=1; mx.google.com; dkim=pass header.i=@gmx.net header.s=badeba3b8450 header.b="B/otRjJs"; spf=pass (google.com: best guess record for domain of linux-kernel-owner@vger.kernel.org designates 209.132.180.67 as permitted sender) smtp.mailfrom=linux-kernel-owner@vger.kernel.org Return-Path: Received: from vger.kernel.org (vger.kernel.org. [209.132.180.67]) by mx.google.com with ESMTP id cm7si4560156pjb.15.2019.08.19.23.07.18; Mon, 19 Aug 2019 23:07:33 -0700 (PDT) Received-SPF: pass (google.com: best guess record for domain of linux-kernel-owner@vger.kernel.org designates 209.132.180.67 as permitted sender) client-ip=209.132.180.67; Authentication-Results: mx.google.com; dkim=pass header.i=@gmx.net header.s=badeba3b8450 header.b="B/otRjJs"; spf=pass (google.com: best guess record for domain of linux-kernel-owner@vger.kernel.org designates 209.132.180.67 as permitted sender) smtp.mailfrom=linux-kernel-owner@vger.kernel.org Received: (majordomo@vger.kernel.org) by vger.kernel.org via listexpand id S1729202AbfHTGGL (ORCPT + 99 others); Tue, 20 Aug 2019 02:06:11 -0400 Received: from mout.gmx.net ([212.227.15.18]:46661 "EHLO mout.gmx.net" rhost-flags-OK-OK-OK-OK) by vger.kernel.org with ESMTP id S1727006AbfHTGGL (ORCPT ); Tue, 20 Aug 2019 02:06:11 -0400 DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/simple; d=gmx.net; s=badeba3b8450; t=1566281126; bh=IbMsxkjULl0sbI8PV+sJrFAWMn1HFEuAyJeB7Ap6EN4=; h=X-UI-Sender-Class:Subject:To:Cc:References:From:Date:In-Reply-To; b=B/otRjJsNNu1WXkg2xOb1+dXcEzwlT/KXuaC6iDJRm85tut56fvGRjttOR87+VVck tzRPwhKHtR+NH3eIeYGSkhz1SyC4Et6k2RT3BE73vo5epLihekPvgem4IEW2kgGMN0 tTw5KRl3ceoQzSjgn7Xy44DvPTqHac+OfzWURlDA= X-UI-Sender-Class: 01bb95c1-4bf8-414a-932a-4f6e2808ef9c Received: from [0.0.0.0] ([54.250.245.166]) by mail.gmx.com (mrgmx004 [212.227.17.184]) with ESMTPSA (Nemesis) id 1N3bWr-1iPUjG227v-010adS; Tue, 20 Aug 2019 08:05:26 +0200 Subject: Re: [PATCH] erofs: move erofs out of staging To: miaoxie@huawei.com, Gao Xiang , "Darrick J. Wong" Cc: Christoph Hellwig , "Theodore Y. Ts'o" , Eric Biggers , Richard Weinberger , Greg Kroah-Hartman , Jan Kara , Chao Yu , Dave Chinner , David Sterba , devel , Stephen Rothwell , Amir Goldstein , linux-erofs , Al Viro , Jaegeuk Kim , linux-kernel , Li Guifu , Fang Wei , Pavel Machek , linux-fsdevel , Andrew Morton , torvalds References: <790210571.69061.1566120073465.JavaMail.zimbra@nod.at> <20190818151154.GA32157@mit.edu> <20190818155812.GB13230@infradead.org> <20190818161638.GE1118@sol.localdomain> <20190818162201.GA16269@infradead.org> <20190818172938.GA14413@sol.localdomain> <20190818174702.GA17633@infradead.org> <20190818181654.GA1617@hsiangkao-HP-ZHAN-66-Pro-G1> <20190818201405.GA27398@hsiangkao-HP-ZHAN-66-Pro-G1> <20190819160923.GG15198@magnolia> <20190819203051.GA10075@hsiangkao-HP-ZHAN-66-Pro-G1> <698e2fa6-956b-b367-6f6a-3e6b09bfef5f@huawei.com> From: Qu Wenruo Openpgp: preference=signencrypt Autocrypt: addr=quwenruo.btrfs@gmx.com; prefer-encrypt=mutual; keydata= mQENBFnVga8BCACyhFP3ExcTIuB73jDIBA/vSoYcTyysFQzPvez64TUSCv1SgXEByR7fju3o 8RfaWuHCnkkea5luuTZMqfgTXrun2dqNVYDNOV6RIVrc4YuG20yhC1epnV55fJCThqij0MRL 1NxPKXIlEdHvN0Kov3CtWA+R1iNN0RCeVun7rmOrrjBK573aWC5sgP7YsBOLK79H3tmUtz6b 9Imuj0ZyEsa76Xg9PX9Hn2myKj1hfWGS+5og9Va4hrwQC8ipjXik6NKR5GDV+hOZkktU81G5 gkQtGB9jOAYRs86QG/b7PtIlbd3+pppT0gaS+wvwMs8cuNG+Pu6KO1oC4jgdseFLu7NpABEB AAG0IlF1IFdlbnJ1byA8cXV3ZW5ydW8uYnRyZnNAZ214LmNvbT6JAVQEEwEIAD4CGwMFCwkI BwIGFQgJCgsCBBYCAwECHgECF4AWIQQt33LlpaVbqJ2qQuHCPZHzoSX+qAUCWdWCnQUJCWYC bgAKCRDCPZHzoSX+qAR8B/94VAsSNygx1C6dhb1u1Wp1Jr/lfO7QIOK/nf1PF0VpYjTQ2au8 ihf/RApTna31sVjBx3jzlmpy+lDoPdXwbI3Czx1PwDbdhAAjdRbvBmwM6cUWyqD+zjVm4RTG rFTPi3E7828YJ71Vpda2qghOYdnC45xCcjmHh8FwReLzsV2A6FtXsvd87bq6Iw2axOHVUax2 FGSbardMsHrya1dC2jF2R6n0uxaIc1bWGweYsq0LXvLcvjWH+zDgzYCUB0cfb+6Ib/ipSCYp 3i8BevMsTs62MOBmKz7til6Zdz0kkqDdSNOq8LgWGLOwUTqBh71+lqN2XBpTDu1eLZaNbxSI ilaVuQENBFnVga8BCACqU+th4Esy/c8BnvliFAjAfpzhI1wH76FD1MJPmAhA3DnX5JDORcga CbPEwhLj1xlwTgpeT+QfDmGJ5B5BlrrQFZVE1fChEjiJvyiSAO4yQPkrPVYTI7Xj34FnscPj /IrRUUka68MlHxPtFnAHr25VIuOS41lmYKYNwPNLRz9Ik6DmeTG3WJO2BQRNvXA0pXrJH1fN GSsRb+pKEKHKtL1803x71zQxCwLh+zLP1iXHVM5j8gX9zqupigQR/Cel2XPS44zWcDW8r7B0 q1eW4Jrv0x19p4P923voqn+joIAostyNTUjCeSrUdKth9jcdlam9X2DziA/DHDFfS5eq4fEv ABEBAAGJATwEGAEIACYWIQQt33LlpaVbqJ2qQuHCPZHzoSX+qAUCWdWBrwIbDAUJA8JnAAAK CRDCPZHzoSX+qA3xB/4zS8zYh3Cbm3FllKz7+RKBw/ETBibFSKedQkbJzRlZhBc+XRwF61mi f0SXSdqKMbM1a98fEg8H5kV6GTo62BzvynVrf/FyT+zWbIVEuuZttMk2gWLIvbmWNyrQnzPl mnjK4AEvZGIt1pk+3+N/CMEfAZH5Aqnp0PaoytRZ/1vtMXNgMxlfNnb96giC3KMR6U0E+siA 4V7biIoyNoaN33t8m5FwEwd2FQDG9dAXWhG13zcm9gnk63BN3wyCQR+X5+jsfBaS4dvNzvQv h8Uq/YGjCoV1ofKYh3WKMY8avjq25nlrhzD/Nto9jHp8niwr21K//pXVA81R2qaXqGbql+zo Message-ID: <301ccbea-4140-3816-a1b3-5018ffb4036c@gmx.com> Date: Tue, 20 Aug 2019 14:04:46 +0800 User-Agent: Mozilla/5.0 (X11; Linux x86_64; rv:60.0) Gecko/20100101 Thunderbird/60.8.0 MIME-Version: 1.0 In-Reply-To: <698e2fa6-956b-b367-6f6a-3e6b09bfef5f@huawei.com> Content-Type: multipart/signed; micalg=pgp-sha256; protocol="application/pgp-signature"; boundary="3bJxyv1PvPqOyLfzZcRWdqjlgomhmdc78" X-Provags-ID: V03:K1:oX7mHmiMsoFpvgiuoR/eWsBsLEEhaKf3hVRHMoPOsKwmopiQdZx rHjkE92dh7RDyRL6UdcW3WoefWsONT78p0UW33wTIary/kAgbR8mtj3+6saZsIXvncyle8f +4rUZ9C8kkO8pltGJVPaG962QBrktB+64wq8OH6oSIlDPTqOx4mEaIFB5tSO9QTbtTSM0EH g4pAtxYh7hcdR8YTiHF8g== X-Spam-Flag: NO X-UI-Out-Filterresults: notjunk:1;V03:K0:L8Dhug8qyvA=:NYnppo2JgU4b4Vt7CBjMhs YQnbQUIv4hD8BlNRUK9FohaJew7aaMZ53Ek615FLdqI+OjNA89p+Ra+lSxIL2l7qliHZe+RTI y6nmjMY0ARDe9EETSz1fXiu19JoqYjBGHvjmX2YwfTYaBQJakF5Ia10J8YtIdY9PHzmsRQ4fi GhB3EgyOheyl4H8nejoBDCYaTZW6QlwToAaSlVd0KOeS9VA2JAElRBE6OcN+VvQZHz7YOBT40 Af3fJtaKOopRQW8WU8w30w3Pou0wVVw0qOg3dZH7TTGMXXEeGZ8xyzdM6qlhbLt94P4/9DVTM WHOkc9aTUovYIrWB3dDEd2MAEuMIp8yzImjM5UWTwb+x6EmcxWp+ri4qnyJQ9DmJOutk/0jVI /W+xIbrD461dV7F2bapP6OCpUwPTL5ihOsD0h9npx12x8FDcT65qnU5oyx2QFNh/LDvWoyz1l ZuIQtOU0C4HENpdCYeGILCBSFpXRPRgZI4/j66V8VJ3e/3ghNTEaeu/6rlpJW4vZ6AldPhWPb MwX7oUubEVj6R8GtrvDpv4h0q6Z8BmM22gOlLN9kxxY3aYE/yPbF4WSF1DUeu5Ub3KMm3wCh8 6OkvYCyh7xUmU7/qiqPXb+NT00DL9nwiLcsjREnJAj+kvOnfxDlHpyB7eH254mPaI3rtAGiaI Pn7GPDWonmBhJ+GEVf/k1XsWuK37sD1MkB7H/dqBCJUOsYYesGXWEy0tLB9ld5Fsxhg6t3D25 3K/bdT8m9nBKOqf55fpNU3WD+7YP6z0oREDNEiTvoa4S6fKCO5yHyvwE7ZSKrGLMGWRllF8B+ R60SdGTjFPmFlnIunJPF6kaDbNGtyd68MqL1uuQRfswQTAg+qYqeoASJ5TVGL4Bet4cxCQpCh aizUybf98Fcm2JuniuwxwcdLfTDfR5O+ZmdVBTy+Krs2S3kMjv/uk0rNAixBMzehgUrMfbl7F 9yoHFUxNc2xARXZCyO2+yT5G4RBfA6xnsSNJ32d6a21rkL2kTxUqg6g1zAZByGLx5IjzUHKNO 2GnBAseSiUKRCVJ+vCsWcODW35+gL1Jn08oiH8XF83NG4rYScW2H5eHjhnECkrHD4XCValEa9 IuWObXnnI1stKRHWlPEk/oU0eWkkKBQ5oueTEotfEKc8o60TDTekWjPMtjcfN1k1KBq2ww7wQ DgDMUtDkMzg0Wzgcaf6ecpxzOmuHcJT6D4HMUA5lyKfiwLcQ== Sender: linux-kernel-owner@vger.kernel.org Precedence: bulk List-ID: X-Mailing-List: linux-kernel@vger.kernel.org This is an OpenPGP/MIME signed message (RFC 4880 and 3156) --3bJxyv1PvPqOyLfzZcRWdqjlgomhmdc78 Content-Type: multipart/mixed; boundary="Olm72Ypg8nkRzM1mIhNGvCKobZrRVB0Ah"; protected-headers="v1" From: Qu Wenruo To: miaoxie@huawei.com, Gao Xiang , "Darrick J. Wong" Cc: Christoph Hellwig , "Theodore Y. Ts'o" , Eric Biggers , Richard Weinberger , Greg Kroah-Hartman , Jan Kara , Chao Yu , Dave Chinner , David Sterba , devel , Stephen Rothwell , Amir Goldstein , linux-erofs , Al Viro , Jaegeuk Kim , linux-kernel , Li Guifu , Fang Wei , Pavel Machek , linux-fsdevel , Andrew Morton , torvalds Message-ID: <301ccbea-4140-3816-a1b3-5018ffb4036c@gmx.com> Subject: Re: [PATCH] erofs: move erofs out of staging References: <790210571.69061.1566120073465.JavaMail.zimbra@nod.at> <20190818151154.GA32157@mit.edu> <20190818155812.GB13230@infradead.org> <20190818161638.GE1118@sol.localdomain> <20190818162201.GA16269@infradead.org> <20190818172938.GA14413@sol.localdomain> <20190818174702.GA17633@infradead.org> <20190818181654.GA1617@hsiangkao-HP-ZHAN-66-Pro-G1> <20190818201405.GA27398@hsiangkao-HP-ZHAN-66-Pro-G1> <20190819160923.GG15198@magnolia> <20190819203051.GA10075@hsiangkao-HP-ZHAN-66-Pro-G1> <698e2fa6-956b-b367-6f6a-3e6b09bfef5f@huawei.com> In-Reply-To: <698e2fa6-956b-b367-6f6a-3e6b09bfef5f@huawei.com> --Olm72Ypg8nkRzM1mIhNGvCKobZrRVB0Ah Content-Type: text/plain; charset=utf-8 Content-Language: en-US Content-Transfer-Encoding: quoted-printable [...] >> The same tool exists for btrfs, although lacks the write ability, but >> that dump is more comprehensive and a great tool to learn the on-disk >> format. >> >> >> And for the fuzzing defending part, just a few kernel releases ago, >> there is none for btrfs, and now we have a full static verification >> layer to cover (almost) all on-disk data at read and write time. >> (Along with enhanced runtime check) >> >> We have covered from vague values inside tree blocks and invalid/missi= ng >> cross-ref find at runtime. >> >> Currently the two layered check works pretty fine (well, sometimes too= >> good to detect older, improper behaved kernel). >> - Tree blocks with vague data just get rejected by verification layer >> So that all members should fit on-disk format, from alignment to >> generation to inode mode. >> >> The error will trigger a good enough (TM) error message for develope= r >> to read, and if we have other copies, we retry other copies just as >> we hit a bad copy. >> >> - At runtime, we have much less to check >> Only cross-ref related things can be wrong now. since everything >> inside a single tree block has already be checked. >> >> In fact, from my respect of view, such read time check should be there= >> from the very beginning. >> It acts kinda of a on-disk format spec. (In fact, by implementing the >> verification layer itself, it already exposes a lot of btrfs design >> trade-offs) >> >> Even for a fs as complex (buggy) as btrfs, we only take 1K lines to >> implement the verification layer. >> So I'd like to see every new mainlined fs to have such ability. >=20 > It is a good idea. In fact, we already have a verification layer which = was implemented > as a device mapper sub-module. I think it is enough for a read-only fil= esystem because > it is simple, flexible and independent(we can modify the filesystem lay= out without > verification module modification). If you're talking about dm-verity, then IMHO they are with completely different objective. For dm-verity it's more like authentication. Without proper key (authentication), no one can modify the data without being caught. That's why I hate such thing, it's not open at all, *as bad as locked bootloader*. While the tree-checker (the layer in btrfs) is more like a sensitive and sometimes overreacting detector, find anything wrong, then reject the offending tree block. The original objective of tree-checker is to free coder from defensive coding, providing a centralized verification service, thus we don't need to verify tree blocks randomly using ugly BUG_ON()s. (But unfortunately, a lot of BUG_ON()s are still kept as is, as it takes more time to persuade reviewers that those BUG_ON()s are impossible to hit anymore) Tree-checker can not only detect suspicious metadata (either caused by mem bit flip or poorly crafted image), but also bad *kernel* behavior or even runtime bitflip. (Well, it only works for RW fs, so not really helpful for a RO fs). And performance is another point. That tree-checker in btrfs is as fast/slow as CRC32. Not sure how it would be for dm-verity, but I guess it's slower than CRC32 if using any strong hash. Anyway, for a RO fs, if it's relying on dm-verify then that's OK for real-world usage. But as a standalone fs, even it's RO, a verification layer would be a great plus. At least when new student developers try fuzzed images on the fs, it would be a good surprise other than tons of new bug reports. >=20 > =20 >>> [...] >>> >>> Yes, we will do such a debugging tool of course. Actually Li Guifu is= now >>> developping a erofs-fuse to support old linux versions or other OSes = for >>> archiveing only use, we will base on that code to develop a better fu= zzer >>> tool as well. >> >> Personally speaking, debugging tool is way more important than a runni= ng >> kernel module/fuse. >> It's human trying to write the code, most of time is spent educating >> code readers, thus debugging tool is way more important than dead cold= code. >=20 > Agree, Xiang and I have no time to developing this feature now, we are = glad very much if you could help > us to do it ;) In fact, since the fs is a RO fs, it could be pretty good educational example for any fs newbies. Thus a debug tool which can show the full metadata of the fs can really be helpful. In fact, btrfs-debug-tree (now "btrfs ins dump-tree") leads my way to btrfs, and still one of my favourite tool to debug. Thanks, Qu --Olm72Ypg8nkRzM1mIhNGvCKobZrRVB0Ah-- --3bJxyv1PvPqOyLfzZcRWdqjlgomhmdc78 Content-Type: application/pgp-signature; name="signature.asc" Content-Description: OpenPGP digital signature Content-Disposition: attachment; filename="signature.asc" -----BEGIN PGP SIGNATURE----- iQEzBAEBCAAdFiEELd9y5aWlW6idqkLhwj2R86El/qgFAl1bjX4ACgkQwj2R86El /qhLkwf/QvNM5Jdv2JgYzivaLSTXBKPyCLWpUqZ+dIfb+1eG+ZdCEl3yn0n1oqQA 6yaSa43H9AHVAU4vvB2JSsF9+O5twW163Wt1xv9hnDmI13N15jkPyyzh204XsTCc 9VwqcUOjEJO0IjO+y711kR0nai8SwRhXDvlNx0PpINrO1XDZtz1rnEBD/I5OgDiL 2CaGcdiILui3WW+5GWBeU1lGAaZZOEz8PYas2VeUTP4Zpidps9ClmquyPo7WH19C QpKDnYayXRF9xB45qCXD+2hcm2PN5D0U+rGe1bCQmOroJjJ1bW0tFlh5kEkyZXNi mNmvtLVY/lVtvOt3xw9S3UXreStBhQ== =G385 -----END PGP SIGNATURE----- --3bJxyv1PvPqOyLfzZcRWdqjlgomhmdc78--