Received-SPF: pass (google.com: best guess record for domain of linux-kernel-owner@vger.kernel.org designates 209.132.180.67 as permitted sender) client-ip=209.132.180.67;
Subject: Re: Creating compressed backing_store as swapfile
To:     valdis.kletnieks@vt.edu
Cc:     Adam Borowski <kilobyte@angband.pl>,
        Pintu Agarwal <pintu.ping@gmail.com>, linux-mm@kvack.org,
        open list <linux-kernel@vger.kernel.org>,
        kernelnewbies@kernelnewbies.org
References: <CAOuPNLjuM5qq3go9ZFZcK0G5pQxTQb0DY36xu+8SL4vC4zJntw@mail.gmail.com>
 <20181105155815.i654i5ctmfpqhggj@angband.pl>
 <79d0c96a-a0a2-63ec-db91-42fd349d50c1@gmail.com>
 <42594.1541434463@turing-police.cc.vt.edu>
 <6a1f57b6-503c-48a2-689b-3c321cd6d29f@gmail.com>
 <83467.1541436836@turing-police.cc.vt.edu>
From:   "Austin S. Hemmelgarn" <ahferroin7@gmail.com>
Message-ID: <20fe7145-8426-c67d-2ab2-258ec5717966@gmail.com>
Date:   Mon, 5 Nov 2018 11:55:58 -0500
User-Agent: Mozilla/5.0 (Windows NT 10.0; WOW64; rv:60.0) Gecko/20100101
 Thunderbird/60.3.0
MIME-Version: 1.0
In-Reply-To: <83467.1541436836@turing-police.cc.vt.edu>
Content-Type: text/plain; charset=windows-1252; format=flowed
Content-Language: en-US
Content-Transfer-Encoding: 7bit
Sender: linux-kernel-owner@vger.kernel.org
Precedence: bulk

On 11/5/2018 11:53 AM, valdis.kletnieks@vt.edu wrote:
> On Mon, 05 Nov 2018 11:28:49 -0500, "Austin S. Hemmelgarn" said:
> 
>> Also, it's probably worth noting that BTRFS doesn't need to decompress
>> the entire file to read or write blocks in the middle, it splits the
>> file into 128k blocks and compresses each of those independent of the
>> others, so it can just decompress the 128k block that holds the actual
>> block that's needed.
> 
> Presumably it does something sane with block allocation for the now-compressed
> 128K that's presumably much smaller.  Also, that limits the damage from writing to
> the middle of a compression unit....
> 
> That *does* however increase the memory requirement - you can OOM or
> deadlock if your read/write from the swap needs an additional 128K for the
> compression buffer at an inconvenient time...
> 
Indeed, and I can't really comment on how it might behave under those 
circumstances (the systems I did the testing on never saw memory 
pressure quite _that_ bad, and I had them set up to swap things out 
pretty early and really aggressively).