Received-SPF: pass (google.com: domain of linux-kernel-owner@vger.kernel.org designates 2620:137:e000::1:20 as permitted sender) client-ip=2620:137:e000::1:20;
Date:   Tue, 30 Aug 2022 21:59:30 +0800
From:   Jisheng Zhang <jszhang@kernel.org>
To:     Conor.Dooley@microchip.com
Cc:     paul.walmsley@sifive.com, palmer@dabbelt.com,
        aou@eecs.berkeley.edu, linux-riscv@lists.infradead.org,
        linux-kernel@vger.kernel.org, ajones@ventanamicro.com
Subject: Re: [PATCH v2] riscv: enable THP_SWAP for RV64
Message-ID: <Yw4XwsVQBB1HRw04@xhacker>
References: <20220827095815.698-1-jszhang@kernel.org>
 <0256a458-440c-171c-2a6f-e88a50c16f82@microchip.com>
 <YwzI7VmfFnOvYWgf@xhacker>
 <4037b310-47c4-ca30-d4b6-a284e87da437@microchip.com>
MIME-Version: 1.0
Content-Type: text/plain; charset=utf-8
Content-Disposition: inline
In-Reply-To: <4037b310-47c4-ca30-d4b6-a284e87da437@microchip.com>
Precedence: bulk

On Mon, Aug 29, 2022 at 05:27:48PM +0000, Conor.Dooley@microchip.com wrote:
> On 29/08/2022 15:10, Jisheng Zhang wrote:
> > EXTERNAL EMAIL: Do not click links or open attachments unless you know the content is safe
> > 
> > On Sat, Aug 27, 2022 at 09:13:03PM +0000, Conor.Dooley@microchip.com wrote:
> >> Hey Jisheng,
> > 
> > Hi Conor,
> > 
> >> On 27/08/2022 10:58, Jisheng Zhang wrote:
> >>> I have a Sipeed Lichee RV dock board which only has 512MB DDR, so
> >>> memory optimizations such as swap on zram are helpful. As is seen
> >>> in commit d0637c505f8a ("arm64: enable THP_SWAP for arm64") and
> >>> commit bd4c82c22c367e ("mm, THP, swap: delay splitting THP after
> >>> swapped out"), THP_SWAP can improve the swap throughput significantly.
> >>>
> >>> Enable THP_SWAP for RV64, testing the micro-benchmark which is
> >>> introduced by commit d0637c505f8a ("arm64: enable THP_SWAP for arm64")
> >>> shows below numbers on the Lichee RV dock board:
> >>>
> >>> thp swp throughput w/o patch: 66908 bytes/ms (mean of 10 tests)
> >>> thp swp throughput w/ patch: 322638 bytes/ms (mean of 10 tests)
> >>
> >> I know the original commit message contains this, but it's a little
> >> odd. If the patch /enables/ THP then how would there be THP swap
> >> prior to the patch?
> > 
> > hmm, it's swap I'll send a v3 to correct the description.
> > 
> >>
> >>>
> >>> Improved by 382%!
> >>
> >> I could not replicate the after numbers on my nezha, so I suspect
> >> I am missing something in my config/setup. zswap is enabled and is
> > 
> > swap on zram rather than zswap ;)
> 
> I think I tried about 30 different config variations, initially not
> using zswap and later using it.
> My zramctl looks like so (although I did try zstd too) after running
> the demo application from that commit:
> 
> NAME       ALGORITHM DISKSIZE DATA COMPR TOTAL STREAMS MOUNTPOINT
> /dev/zram0 lzo-rle       241M  22M  8.4M  9.1M       1 [SWAP]
> 
> I am using the default riscv defconfig + the following:
> CONFIG_ZRAM=y
> CONFIG_CRYPTO_DEFLATE=y
> CONFIG_CRYPTO_LZO=y
> CONFIG_CRYPTO_ZSTD=y
> CONFIG_ZRAM_MEMORY_TRACKING=y
> CONFIG_TRANSPARENT_HUGEPAGE=y
> CONFIG_TRANSPARENT_HUGEPAGE_MADVISE=y
> CONFIG_THP_SWAP=y
> 
> Am I just missing something obvious here?

similar config options here. what's your rootfs? Is your board busy
with something? I used a minimal rootfs built from buildroot.
can you plz show your numbers w/ and w/o the patch?

I also tried the simple benchmark on qemu(just for reference, since
I have no other riscv boards except the lichee RV dock board):
swp out w/o patch: 30066 bytes/ms (mean of 10 tests)
swp out w/ patch: 130055 bytes/ms (mean of 10 tests)
so improved by 332.7%