Received: by 2002:a25:31c3:0:0:0:0:0 with SMTP id x186csp596709ybx; Wed, 30 Oct 2019 01:47:07 -0700 (PDT) X-Google-Smtp-Source: APXvYqw6wsj+CcnlpwmJOzQz/5GUmGWZ+BQgKlZLk3/C4UtHIotPX075i4X0bc3enfYu4LM1JD3c X-Received: by 2002:a50:9269:: with SMTP id j38mr30129923eda.5.1572425227273; Wed, 30 Oct 2019 01:47:07 -0700 (PDT) ARC-Seal: i=1; a=rsa-sha256; t=1572425227; cv=none; d=google.com; s=arc-20160816; b=OWfn0gcjOIuXSTrX3bFIxRDJ/NdZ0E4myIow+4SoOXk/MYOHWQ9Q1qaPCljZ3qYDNp Q4aTOuglBhLP3I+QwRxQtJFxq+wYkhCUG79W0pkmJQx0oj6ev3uX2nDNUdW5QtbrMxFC +6t9lHHweh6elIe5L+N+JEZIXG5z0gslvT5GXXeIL9vHtKvzv9NhXGsIV6pyejHH0AUL QZDkVSpBnJunGymVBvunqAUhRXDUAO6y/QxQwKowPNvDX85ZDvdE9sfNxjkWUvXbsNtC ckGCfucontaxIGemtA9dCGdi8gKUO/bhTZIiarCKX8hlIddl/fttNHO516IcKOwI4Bo5 6YWg== ARC-Message-Signature: i=1; a=rsa-sha256; c=relaxed/relaxed; d=google.com; s=arc-20160816; h=list-id:precedence:sender:content-transfer-encoding :content-language:in-reply-to:mime-version:user-agent:date :message-id:cc:from:references:to:subject; bh=hdtUcNKdiNNDsG0k0IvYJ8YI9WM+CvmooSdrnb9avZ4=; b=LkqlHswjJQweTMqxlcXRegwYP90XlnKjUtJ3w8qMHalJ45/tDvCQb4Bu8xeNphMF+F RJmbhpzXBdyV6pqzB7YRiwoi6JSMS1bdrMQwsJtzrDPrW/NAwaDcDBgKmO9A67OO7C8k mdLhOLlMClPQHYMMfNV46jlBoYvjYQ+CDtoI5OCFk8xxIK1q8QtFLRghjcR+082W9aWL 35VqYGpROB64lA0qQTejiRvZdOKI7Y8b1eqPIoWj+wQy86WAW4p+FAiU5T/pqGVYREjC ZX1g7Cfa1UYRE14ieg+n6QBym6S+N2Xf1m6uk4aqWh7nbqNPDnb2S/0WD3teB9EqX4zW uuxg== ARC-Authentication-Results: i=1; mx.google.com; spf=pass (google.com: best guess record for domain of linux-kernel-owner@vger.kernel.org designates 209.132.180.67 as permitted sender) smtp.mailfrom=linux-kernel-owner@vger.kernel.org Return-Path: Received: from vger.kernel.org (vger.kernel.org. [209.132.180.67]) by mx.google.com with ESMTP id og20si843979ejb.29.2019.10.30.01.46.43; Wed, 30 Oct 2019 01:47:07 -0700 (PDT) Received-SPF: pass (google.com: best guess record for domain of linux-kernel-owner@vger.kernel.org designates 209.132.180.67 as permitted sender) client-ip=209.132.180.67; Authentication-Results: mx.google.com; spf=pass (google.com: best guess record for domain of linux-kernel-owner@vger.kernel.org designates 209.132.180.67 as permitted sender) smtp.mailfrom=linux-kernel-owner@vger.kernel.org Received: (majordomo@vger.kernel.org) by vger.kernel.org via listexpand id S1726209AbfJ3IoC (ORCPT + 99 others); Wed, 30 Oct 2019 04:44:02 -0400 Received: from szxga06-in.huawei.com ([45.249.212.32]:42684 "EHLO huawei.com" rhost-flags-OK-OK-OK-FAIL) by vger.kernel.org with ESMTP id S1726028AbfJ3IoB (ORCPT ); Wed, 30 Oct 2019 04:44:01 -0400 Received: from DGGEMS402-HUB.china.huawei.com (unknown [172.30.72.60]) by Forcepoint Email with ESMTP id 15ADDE56AFBAB5BA9233; Wed, 30 Oct 2019 16:43:58 +0800 (CST) Received: from [10.134.22.195] (10.134.22.195) by smtp.huawei.com (10.3.19.202) with Microsoft SMTP Server (TLS) id 14.3.439.0; Wed, 30 Oct 2019 16:43:52 +0800 Subject: Re: [PATCH 2/2] f2fs: support data compression To: Eric Biggers References: <20191022171602.93637-1-jaegeuk@kernel.org> <20191022171602.93637-2-jaegeuk@kernel.org> <20191027225006.GA321938@sol.localdomain> <20191030025512.GA4791@sol.localdomain> From: Chao Yu CC: Jaegeuk Kim , , Message-ID: <97c33fa1-15af-b319-29a1-22f254a26c0a@huawei.com> Date: Wed, 30 Oct 2019 16:43:52 +0800 User-Agent: Mozilla/5.0 (Windows NT 6.1; WOW64; rv:52.0) Gecko/20100101 Thunderbird/52.9.1 MIME-Version: 1.0 In-Reply-To: <20191030025512.GA4791@sol.localdomain> Content-Type: text/plain; charset="windows-1252" Content-Language: en-US Content-Transfer-Encoding: 7bit X-Originating-IP: [10.134.22.195] X-CFilter-Loop: Reflected Sender: linux-kernel-owner@vger.kernel.org Precedence: bulk List-ID: X-Mailing-List: linux-kernel@vger.kernel.org On 2019/10/30 10:55, Eric Biggers wrote: > On Tue, Oct 29, 2019 at 04:33:36PM +0800, Chao Yu wrote: >> On 2019/10/28 6:50, Eric Biggers wrote: >>>> +bool f2fs_is_compressed_page(struct page *page) >>>> +{ >>>> + if (!page_private(page)) >>>> + return false; >>>> + if (IS_ATOMIC_WRITTEN_PAGE(page) || IS_DUMMY_WRITTEN_PAGE(page)) >>>> + return false; >>>> + return *((u32 *)page_private(page)) == F2FS_COMPRESSED_PAGE_MAGIC; >>>> +} >>> >>> This code implies that there can be multiple page private structures each of >>> which has a different magic number. But I only see F2FS_COMPRESSED_PAGE_MAGIC. >>> Where in the code is the other one(s)? >> >> I'm not sure I understood you correctly, did you mean it needs to introduce >> f2fs_is_atomic_written_page() and f2fs_is_dummy_written_page() like >> f2fs_is_compressed_page()? >> > > No, I'm asking what is the case where the line > > *((u32 *)page_private(page)) == F2FS_COMPRESSED_PAGE_MAGIC > > returns false? Should be this? if (!page_private(page)) return false; f2fs_bug_on(*((u32 *)page_private(page)) != F2FS_COMPRESSED_PAGE_MAGIC) return true; > >>> >>>> + >>>> +static void f2fs_set_compressed_page(struct page *page, >>>> + struct inode *inode, pgoff_t index, void *data, refcount_t *r) >>>> +{ >>>> + SetPagePrivate(page); >>>> + set_page_private(page, (unsigned long)data); >>>> + >>>> + /* i_crypto_info and iv index */ >>>> + page->index = index; >>>> + page->mapping = inode->i_mapping; >>>> + if (r) >>>> + refcount_inc(r); >>>> +} >>> >>> It isn't really appropriate to create fake pagecache pages like this. Did you >>> consider changing f2fs to use fscrypt_decrypt_block_inplace() instead? >> >> We need to store i_crypto_info and iv index somewhere, in order to pass them to >> fscrypt_decrypt_block_inplace(), where did you suggest to store them? >> > > The same place where the pages are stored. Still we need allocate space for those fields, any strong reason to do so? > >>>> + >>>> +void f2fs_destroy_compress_ctx(struct compress_ctx *cc) >>>> +{ >>>> + kvfree(cc->rpages); >>>> +} >>> >>> The memory is allocated with kzalloc(), so why is it freed with kvfree() and not >>> just kfree()? >> >> It was allocated by f2fs_*alloc() which will fallback to kvmalloc() once >> kmalloc() failed. > > This seems to be a bug in f2fs_kmalloc() -- it inappropriately falls back to > kvmalloc(). As per its name, it should only use kmalloc(). f2fs_kvmalloc() > already exists, so it can be used when the fallback is wanted. We can introduce f2fs_memalloc() to wrap f2fs_kmalloc() and f2fs_kvmalloc() as below: f2fs_memalloc() { mem = f2fs_kmalloc(); if (mem) return mem; return f2fs_kvmalloc(); } It can be used in specified place where we really need it, like the place descirbied in 5222595d093e ("f2fs: use kvmalloc, if kmalloc is failed") in where we introduced original logic. > >> >>>> +static int lzo_compress_pages(struct compress_ctx *cc) >>>> +{ >>>> + int ret; >>>> + >>>> + ret = lzo1x_1_compress(cc->rbuf, cc->rlen, cc->cbuf->cdata, >>>> + &cc->clen, cc->private); >>>> + if (ret != LZO_E_OK) { >>>> + printk_ratelimited("%sF2FS-fs: lzo compress failed, ret:%d\n", >>>> + KERN_ERR, ret); >>>> + return -EIO; >>>> + } >>>> + return 0; >>>> +} >>> >>> Why not using f2fs_err()? Same in lots of other places. >> >> We use printk_ratelimited at some points where we can afford to lose logs, >> otherwise we use f2fs_{err,warn...} to record info as much as possible for >> troubleshoot. >> > > It used to be the case that f2fs_msg() was ratelimited. What stops it from > spamming the logs now? https://lore.kernel.org/patchwork/patch/973837/ > > The problem with a bare printk is that it doesn't show which filesystem instance > the message is coming from. We can add to print sbi->sb->s_id like f2fs_printk(). > >>>> + >>>> + ret = cops->compress_pages(cc); >>>> + if (ret) >>>> + goto out_vunmap_cbuf; >>>> + >>>> + max_len = PAGE_SIZE * (cc->cluster_size - 1) - COMPRESS_HEADER_SIZE; >>>> + >>>> + if (cc->clen > max_len) { >>>> + ret = -EAGAIN; >>>> + goto out_vunmap_cbuf; >>>> + } >>> >>> Since we already know the max length we're willing to compress to (the max >>> length for any space to be saved), why is more space than that being allocated? >>> LZ4_compress_default() will return an error if there isn't enough space, so that >>> error could just be used as the indication to store the data uncompressed. >> >> AFAIK, there is no such common error code returned from all compression >> algorithms indicating there is no room for limited target size, however we need >> that information to fallback to write raw pages. Any better idea? >> > > "Not enough room" is the only reasonable way for compression to fail, so all At a glance, compression comments did say only fail due to out-of-space of dst_buf, and it will fail due to other reasons as I checked few codes. a) dst_buf is too small b) src_buf is too large/small c) wrong step maybe missed other cases... Yeah, we can get rid of condition b)/c) during implementation, however, what I'm concern is the implementation is too tight to all error handling of all compression algorithms, as we're not always aware of compression error handling changes. > that's needed is the ability for compression to report errors at all. What > specifically prevents this approach from working? > >>>> static void bio_post_read_processing(struct bio_post_read_ctx *ctx) >>>> { >>>> - /* >>>> - * We use different work queues for decryption and for verity because >>>> - * verity may require reading metadata pages that need decryption, and >>>> - * we shouldn't recurse to the same workqueue. >>>> - */ >>> >>> Why is it okay (i.e., no deadlocks) to no longer use different work queues for >>> decryption and for verity? See the comment above which is being deleted. >> >> Could you explain more about how deadlock happen? or share me a link address if >> you have described that case somewhere? >> > > The verity work can read pages from the file which require decryption. I'm > concerned that it could deadlock if the work is scheduled on the same workqueue. I assume you've tried one workqueue, and suffered deadlock.. > Granted, I'm not an expert in Linux workqueues, so if you've investigated this > and determined that it's safe, can you explain why? I'm not familiar with workqueue... I guess it may not safe that if the work is scheduled to the same cpu in where verity was waiting for data? if the work is scheduled to other cpu, it may be safe. I can check that before splitting the workqueue for verity and decrypt/decompress. Thanks, > > - Eric > . >