Return-Path: Received: (majordomo@vger.kernel.org) by vger.kernel.org via listexpand id S1754298AbdHYBgE (ORCPT ); Thu, 24 Aug 2017 21:36:04 -0400 Received: from mx0a-00082601.pphosted.com ([67.231.145.42]:45777 "EHLO mx0a-00082601.pphosted.com" rhost-flags-OK-OK-OK-OK) by vger.kernel.org with ESMTP id S1754170AbdHYBgD (ORCPT ); Thu, 24 Aug 2017 21:36:03 -0400 From: Nick Terrell To: Joonsoo Kim CC: "sergey.senozhatsky.work@gmail.com" , "linux-kernel@vger.kernel.org" , "minchan@kernel.org" , Yann Collet Subject: Re: [PATCH] zram: add zstd to the supported algorithms list Thread-Topic: [PATCH] zram: add zstd to the supported algorithms list Thread-Index: AQHTHSCvXWqWlMVfbUKYI//VGek9A6KUPdeA//+XcYA= Date: Fri, 25 Aug 2017 01:35:35 +0000 Message-ID: <27EDD68A-61DC-42F1-8422-16B9AB9F0EB3@fb.com> References: <20170824014936.4738-1-sergey.senozhatsky@gmail.com> <20170825004947.GE29701@js1304-P5Q-DELUXE> In-Reply-To: <20170825004947.GE29701@js1304-P5Q-DELUXE> Accept-Language: en-US Content-Language: en-US X-MS-Has-Attach: X-MS-TNEF-Correlator: x-originating-ip: [2620:10d:c090:200::7:5460] x-ms-publictraffictype: Email x-microsoft-exchange-diagnostics: 1;DM5PR15MB1289;20:aXPl4iVFdcsZvBsGmWBgURdKB6sq1A9vo1E2zlBTdhorvORxt+9eq2BTYPaMDmw3Gue+dsn4Mv57MZzAvBy5fGq7TiGmEO0IzKVNZLPOYpps9YjrknGqxqCMQ7X10Bj+bTUdSzGjZm1p8nqBB3SH/TFYlrA52h5pwy4Bh66PSAI= x-ms-exchange-antispam-srfa-diagnostics: SSOS;SSOR; x-forefront-antispam-report: SFV:SKI;SCL:-1;SFV:NSPM;SFS:(10019020)(6009001)(199003)(189002)(24454002)(377454003)(5660300001)(189998001)(76176999)(50986999)(54356999)(106356001)(83716003)(105586002)(82746002)(86362001)(25786009)(36756003)(101416001)(97736004)(6116002)(102836003)(4326008)(39060400002)(8676002)(110136004)(6246003)(77096006)(6486002)(81156014)(81166006)(54906002)(8936002)(229853002)(99286003)(7736002)(3660700001)(3280700002)(305945005)(2950100002)(53936002)(6512007)(2906002)(53546010)(478600001)(6916009)(2900100001)(14454004)(6436002)(33656002)(6506006)(68736007);DIR:OUT;SFP:1102;SCL:1;SRVR:DM5PR15MB1289;H:DM5PR15MB1753.namprd15.prod.outlook.com;FPR:;SPF:None;PTR:InfoNoRecords;A:1;MX:1;LANG:en; x-ms-office365-filtering-correlation-id: 49a30d68-b3c6-475c-5e65-08d4eb59953a x-microsoft-antispam: UriScan:;BCL:0;PCL:0;RULEID:(300000500095)(300135000095)(300000501095)(300135300095)(22001)(300000502095)(300135100095)(2017030254152)(300000503095)(300135400095)(2017052603199)(201703131423075)(201703031133081)(201702281549075)(300000504095)(300135200095)(300000505095)(300135600095)(300000506095)(300135500095);SRVR:DM5PR15MB1289; x-ms-traffictypediagnostic: DM5PR15MB1289: x-exchange-antispam-report-test: UriScan:; x-microsoft-antispam-prvs: x-exchange-antispam-report-cfa-test: BCL:0;PCL:0;RULEID:(100000700101)(100105000095)(100000701101)(100105300095)(100000702101)(100105100095)(6040450)(601004)(2401047)(8121501046)(5005006)(100000703101)(100105400095)(10201501046)(3002001)(93006095)(93001095)(6041248)(20161123558100)(20161123555025)(201703131423075)(201702281528075)(201703061421075)(201703061406153)(20161123562025)(20161123564025)(20161123560025)(6072148)(201708071742011)(100000704101)(100105200095)(100000705101)(100105500095);SRVR:DM5PR15MB1289;BCL:0;PCL:0;RULEID:(100000800101)(100110000095)(100000801101)(100110300095)(100000802101)(100110100095)(100000803101)(100110400095)(100000804101)(100110200095)(100000805101)(100110500095);SRVR:DM5PR15MB1289; x-forefront-prvs: 041032FF37 spamdiagnosticoutput: 1:99 spamdiagnosticmetadata: NSPM Content-Type: text/plain; charset="utf-8" Content-ID: MIME-Version: 1.0 X-MS-Exchange-CrossTenant-originalarrivaltime: 25 Aug 2017 01:35:35.3548 (UTC) X-MS-Exchange-CrossTenant-fromentityheader: Hosted X-MS-Exchange-CrossTenant-id: 8ae927fe-1255-47a7-a2af-5f3a069daaa2 X-MS-Exchange-Transport-CrossTenantHeadersStamped: DM5PR15MB1289 X-OriginatorOrg: fb.com X-Proofpoint-Spam-Reason: safe X-FB-Internal: Safe X-Proofpoint-Virus-Version: vendor=fsecure engine=2.50.10432:,, definitions=2017-08-25_01:,, signatures=0 Sender: linux-kernel-owner@vger.kernel.org List-ID: X-Mailing-List: linux-kernel@vger.kernel.org Content-Transfer-Encoding: 8bit X-MIME-Autoconverted: from base64 to 8bit by nfs id v7P1aA3q017292 Content-Length: 1350 Lines: 36 On 8/24/17, 5:49 PM, "Joonsoo Kim" wrote: > On Thu, Aug 24, 2017 at 09:33:54PM +0000, Nick Terrell wrote: > > On Thu, Aug 24, 2017 at 10:49:36AM +0900, Sergey Senozhatsky wrote: > > > Add ZSTD to the list of supported compression algorithms. > > > > > > Official benchmarks [1]: > > > > Awesome! Let me know if you need anything from me. > > > Hello, Nick. > > Awesome work!!! > > Let me ask a question. > Zram compress and decompress a small data (a page) and your github > site says that using predefined dictionary would be helpful in this > situation. However, it seems that compression crypto API for zstd > doesn't use ZSTD_compress_usingDict(). Is there any plan to support > it? I think using dictionaries in zram could be very interesting. We could for example, take a random sample of the RAM and use that as the dictionary for compression. E.g. take 32 512B samples from RAM and build a 16 KB dictionary (sizes may vary). I'm not sure how you would pass a dictionary into the crypto compression API, but I'm sure we can make something work if dictionary compression proves to be beneficial enough. What data have you, or anyone, used for benchmarking compression ratio and speed for RAM? Since it is such a specialized application, the standard compression benchmarks aren't very applicable. > Thanks.