Received: by 2002:a25:ab43:0:0:0:0:0 with SMTP id u61csp7193495ybi; Wed, 5 Jun 2019 12:55:26 -0700 (PDT) X-Google-Smtp-Source: APXvYqxRSBli+5l5hul2xM0Y+mQlaOA01wQAKNBw9GFRnmN3jG4Z/droW9PEsJm6lKv+PYqTmji6 X-Received: by 2002:a62:1787:: with SMTP id 129mr17760151pfx.60.1559764526878; Wed, 05 Jun 2019 12:55:26 -0700 (PDT) ARC-Seal: i=1; a=rsa-sha256; t=1559764526; cv=none; d=google.com; s=arc-20160816; b=mKS9gar+SSCrp0+CFAqd0Bni/TGS3iz32EW1recghkfms+XiyvkvWCI7hC3IPeJ1DU 3KKi12SXsaZi6OfppeNSNQf9yQ/B0qKi6hpZoN90uRdD2j9T32nDp32ThXmY/rkp1V2t H0QNn7XdiEtmAg8wvO4Vk39C5q6ucuuhnzKPSJYaCiDhLdjBOO9fQYL7dYqI/JtqOob9 V+ORAIwQYJr5E4Kh2TJ9JbblwDHOy/1dYiwLXyuyrmwjpDhjT/+Vyd2r1ASL/37FheqO 5iLVbLcoIsV5xx0bYbZ/cGjDOKZl25Bwtx0hqiosC+Z9WvBIYeLPp9OcHJ1NnB0Z1pAl aR1g== ARC-Message-Signature: i=1; a=rsa-sha256; c=relaxed/relaxed; d=google.com; s=arc-20160816; h=list-id:precedence:sender:cc:to:subject:message-id:date:from :in-reply-to:references:mime-version:dkim-signature; bh=pq+ABjtf9saoF6dUdDDxsada/8divT9WxU6TICP/hS0=; b=BAXJcLWklgM119TpPRJ1TygTNev5HQXiYDVMtyU/2kwtLKBews3Q2WmdPSNBwbf6EL xSgfXuIZm0tzcl1sOuo51bTiMfSFBbVpkhcwG4aHdBXuk5HIEKDr9BeGLp4k8gjFSON6 W3BYrDIvaWxOMeTxhoI4V3kX9VYNdaaYryisCthZ0yyzzR3fN9b0rrlL7ZfO8labooOr oVL9Wa6wKsSYf+VdE9VWAIFe6NVJ+SfzBzgbKs3ygcIECnefZOzO5Wq/QiRg074wuDnk r6DAL5EeRC9JvBHpRcs2jsw2X08iv5aOXoMhrl/r9YL89fO75A21gh117NuHOR6RL9a7 Ib+w== ARC-Authentication-Results: i=1; mx.google.com; dkim=pass header.i=@google.com header.s=20161025 header.b=aDihaTMg; spf=pass (google.com: best guess record for domain of linux-kernel-owner@vger.kernel.org designates 209.132.180.67 as permitted sender) smtp.mailfrom=linux-kernel-owner@vger.kernel.org; dmarc=pass (p=REJECT sp=REJECT dis=NONE) header.from=google.com Return-Path: Received: from vger.kernel.org (vger.kernel.org. [209.132.180.67]) by mx.google.com with ESMTP id y1si33859910pfb.264.2019.06.05.12.55.09; Wed, 05 Jun 2019 12:55:26 -0700 (PDT) Received-SPF: pass (google.com: best guess record for domain of linux-kernel-owner@vger.kernel.org designates 209.132.180.67 as permitted sender) client-ip=209.132.180.67; Authentication-Results: mx.google.com; dkim=pass header.i=@google.com header.s=20161025 header.b=aDihaTMg; spf=pass (google.com: best guess record for domain of linux-kernel-owner@vger.kernel.org designates 209.132.180.67 as permitted sender) smtp.mailfrom=linux-kernel-owner@vger.kernel.org; dmarc=pass (p=REJECT sp=REJECT dis=NONE) header.from=google.com Received: (majordomo@vger.kernel.org) by vger.kernel.org via listexpand id S1726556AbfFETxR (ORCPT + 99 others); Wed, 5 Jun 2019 15:53:17 -0400 Received: from mail-yw1-f67.google.com ([209.85.161.67]:46382 "EHLO mail-yw1-f67.google.com" rhost-flags-OK-OK-OK-OK) by vger.kernel.org with ESMTP id S1726305AbfFETxR (ORCPT ); Wed, 5 Jun 2019 15:53:17 -0400 Received: by mail-yw1-f67.google.com with SMTP id v188so1121598ywb.13 for ; Wed, 05 Jun 2019 12:53:16 -0700 (PDT) DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=google.com; s=20161025; h=mime-version:references:in-reply-to:from:date:message-id:subject:to :cc; bh=pq+ABjtf9saoF6dUdDDxsada/8divT9WxU6TICP/hS0=; b=aDihaTMgou/t97G57d7rSXoCcTQDt7cJhZO/VzPbMeLi4sGgyQE/L+c/Xb2mFlIHNc KaML8q353hRhpyMrBy6fUDg55Gbs7i6N+EZcB6tOI30QgIX+8REuTUsVHEh/37+Lr+Ng jNb4ZdmEeXJqM9mhi2gQO2UwqZCUeQx7LvIWYvy9CteXIaZEtI+6ecTcJB/NmyXLcQe0 SvWHAspDlFvaNOq04iKRcDFUKaib9HUW1PYlGomdLdDVMiBpKMV/VZMhOW9XL0fTWOt0 lK290QFMMxxqOu5DnJfLakjc8YpLtXySQhJ9HWoKAbxNvNX5Hqh7LQbTqWFkyzTPFX2D VT0Q== X-Google-DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=1e100.net; s=20161025; h=x-gm-message-state:mime-version:references:in-reply-to:from:date :message-id:subject:to:cc; bh=pq+ABjtf9saoF6dUdDDxsada/8divT9WxU6TICP/hS0=; b=Nb7k3PPxzrZgLnb+H7Ey0Ft/OOLjdbJ1xBJ0ShtwHTjfdaJMaGGnjZumKK+AOc0zln D2fT35D0oFYiYPUPwAh0Ep8rpd/Xv0cQS+E3GCVnt5wZzHAQdyaysnbfD7tLU09jqR+R B9VLiG3AwdOvj/XyQvrm5kkMAqob8lNDhl08ruP7Ax+gPsMiR6rEILO1aRuNSDnmuQVW tk+iaKHwITo6miyG3HqjNgjoy//x+FKj3KrIydkuFQQ+Lb7SZpoWYt5tkiTcHaGZ9PBG h0Fnjno5bGKTOm3/OtVpJ/1I1P5wxF4b7qoUEQDBb++Rnn0yGqWixu89JMsRtY9SZ0E9 ocJw== X-Gm-Message-State: APjAAAWD9hmbfW+Wde2XpgDiiFdmt9cP/WKy4ZYjhs25rJ1gfH8wuSz4 vAdpnnCRKQKfEAvCgLRZTh8odC5xujriAv7YXoEfWg== X-Received: by 2002:a81:5741:: with SMTP id l62mr21598650ywb.4.1559764395811; Wed, 05 Jun 2019 12:53:15 -0700 (PDT) MIME-Version: 1.0 References: <20190605100630.13293-1-teawaterz@linux.alibaba.com> <20190605100630.13293-2-teawaterz@linux.alibaba.com> In-Reply-To: <20190605100630.13293-2-teawaterz@linux.alibaba.com> From: Shakeel Butt Date: Wed, 5 Jun 2019 12:53:04 -0700 Message-ID: Subject: Re: [PATCH V3 2/2] zswap: Use movable memory if zpool support allocate movable memory To: Hui Zhu Cc: Dan Streetman , Minchan Kim , ngupta@vflare.org, sergey.senozhatsky.work@gmail.com, Seth Jennings , Linux MM , LKML Content-Type: text/plain; charset="UTF-8" Sender: linux-kernel-owner@vger.kernel.org Precedence: bulk List-ID: X-Mailing-List: linux-kernel@vger.kernel.org On Wed, Jun 5, 2019 at 3:06 AM Hui Zhu wrote: > > This is the third version that was updated according to the comments > from Sergey Senozhatsky https://lkml.org/lkml/2019/5/29/73 and > Shakeel Butt https://lkml.org/lkml/2019/6/4/973 > > zswap compresses swap pages into a dynamically allocated RAM-based > memory pool. The memory pool should be zbud, z3fold or zsmalloc. > All of them will allocate unmovable pages. It will increase the > number of unmovable page blocks that will bad for anti-fragment. > > zsmalloc support page migration if request movable page: > handle = zs_malloc(zram->mem_pool, comp_len, > GFP_NOIO | __GFP_HIGHMEM | > __GFP_MOVABLE); > > And commit "zpool: Add malloc_support_movable to zpool_driver" add > zpool_malloc_support_movable check malloc_support_movable to make > sure if a zpool support allocate movable memory. > > This commit let zswap allocate block with gfp > __GFP_HIGHMEM | __GFP_MOVABLE if zpool support allocate movable memory. > > Following part is test log in a pc that has 8G memory and 2G swap. > > Without this commit: > ~# echo lz4 > /sys/module/zswap/parameters/compressor > ~# echo zsmalloc > /sys/module/zswap/parameters/zpool > ~# echo 1 > /sys/module/zswap/parameters/enabled > ~# swapon /swapfile > ~# cd /home/teawater/kernel/vm-scalability/ > /home/teawater/kernel/vm-scalability# export unit_size=$((9 * 1024 * 1024 * 1024)) > /home/teawater/kernel/vm-scalability# ./case-anon-w-seq > 2717908992 bytes / 4826062 usecs = 549973 KB/s > 2717908992 bytes / 4864201 usecs = 545661 KB/s > 2717908992 bytes / 4867015 usecs = 545346 KB/s > 2717908992 bytes / 4915485 usecs = 539968 KB/s > 397853 usecs to free memory > 357820 usecs to free memory > 421333 usecs to free memory > 420454 usecs to free memory > /home/teawater/kernel/vm-scalability# cat /proc/pagetypeinfo > Page block order: 9 > Pages per block: 512 > > Free pages count per migrate type at order 0 1 2 3 4 5 6 7 8 9 10 > Node 0, zone DMA, type Unmovable 1 1 1 0 2 1 1 0 1 0 0 > Node 0, zone DMA, type Movable 0 0 0 0 0 0 0 0 0 1 3 > Node 0, zone DMA, type Reclaimable 0 0 0 0 0 0 0 0 0 0 0 > Node 0, zone DMA, type HighAtomic 0 0 0 0 0 0 0 0 0 0 0 > Node 0, zone DMA, type CMA 0 0 0 0 0 0 0 0 0 0 0 > Node 0, zone DMA, type Isolate 0 0 0 0 0 0 0 0 0 0 0 > Node 0, zone DMA32, type Unmovable 6 5 8 6 6 5 4 1 1 1 0 > Node 0, zone DMA32, type Movable 25 20 20 19 22 15 14 11 11 5 767 > Node 0, zone DMA32, type Reclaimable 0 0 0 0 0 0 0 0 0 0 0 > Node 0, zone DMA32, type HighAtomic 0 0 0 0 0 0 0 0 0 0 0 > Node 0, zone DMA32, type CMA 0 0 0 0 0 0 0 0 0 0 0 > Node 0, zone DMA32, type Isolate 0 0 0 0 0 0 0 0 0 0 0 > Node 0, zone Normal, type Unmovable 4753 5588 5159 4613 3712 2520 1448 594 188 11 0 > Node 0, zone Normal, type Movable 16 3 457 2648 2143 1435 860 459 223 224 296 > Node 0, zone Normal, type Reclaimable 0 0 44 38 11 2 0 0 0 0 0 > Node 0, zone Normal, type HighAtomic 0 0 0 0 0 0 0 0 0 0 0 > Node 0, zone Normal, type CMA 0 0 0 0 0 0 0 0 0 0 0 > Node 0, zone Normal, type Isolate 0 0 0 0 0 0 0 0 0 0 0 > > Number of blocks type Unmovable Movable Reclaimable HighAtomic CMA Isolate > Node 0, zone DMA 1 7 0 0 0 0 > Node 0, zone DMA32 4 1652 0 0 0 0 > Node 0, zone Normal 931 1485 15 0 0 0 > > With this commit: > ~# echo lz4 > /sys/module/zswap/parameters/compressor > ~# echo zsmalloc > /sys/module/zswap/parameters/zpool > ~# echo 1 > /sys/module/zswap/parameters/enabled > ~# swapon /swapfile > ~# cd /home/teawater/kernel/vm-scalability/ > /home/teawater/kernel/vm-scalability# export unit_size=$((9 * 1024 * 1024 * 1024)) > /home/teawater/kernel/vm-scalability# ./case-anon-w-seq > 2717908992 bytes / 4689240 usecs = 566020 KB/s > 2717908992 bytes / 4760605 usecs = 557535 KB/s > 2717908992 bytes / 4803621 usecs = 552543 KB/s > 2717908992 bytes / 5069828 usecs = 523530 KB/s > 431546 usecs to free memory > 383397 usecs to free memory > 456454 usecs to free memory > 224487 usecs to free memory > /home/teawater/kernel/vm-scalability# cat /proc/pagetypeinfo > Page block order: 9 > Pages per block: 512 > > Free pages count per migrate type at order 0 1 2 3 4 5 6 7 8 9 10 > Node 0, zone DMA, type Unmovable 1 1 1 0 2 1 1 0 1 0 0 > Node 0, zone DMA, type Movable 0 0 0 0 0 0 0 0 0 1 3 > Node 0, zone DMA, type Reclaimable 0 0 0 0 0 0 0 0 0 0 0 > Node 0, zone DMA, type HighAtomic 0 0 0 0 0 0 0 0 0 0 0 > Node 0, zone DMA, type CMA 0 0 0 0 0 0 0 0 0 0 0 > Node 0, zone DMA, type Isolate 0 0 0 0 0 0 0 0 0 0 0 > Node 0, zone DMA32, type Unmovable 10 8 10 9 10 4 3 2 3 0 0 > Node 0, zone DMA32, type Movable 18 12 14 16 16 11 9 5 5 6 775 > Node 0, zone DMA32, type Reclaimable 0 0 0 0 0 0 0 0 0 0 1 > Node 0, zone DMA32, type HighAtomic 0 0 0 0 0 0 0 0 0 0 0 > Node 0, zone DMA32, type CMA 0 0 0 0 0 0 0 0 0 0 0 > Node 0, zone DMA32, type Isolate 0 0 0 0 0 0 0 0 0 0 0 > Node 0, zone Normal, type Unmovable 2669 1236 452 118 37 14 4 1 2 3 0 > Node 0, zone Normal, type Movable 3850 6086 5274 4327 3510 2494 1520 934 438 220 470 > Node 0, zone Normal, type Reclaimable 56 93 155 124 47 31 17 7 3 0 0 > Node 0, zone Normal, type HighAtomic 0 0 0 0 0 0 0 0 0 0 0 > Node 0, zone Normal, type CMA 0 0 0 0 0 0 0 0 0 0 0 > Node 0, zone Normal, type Isolate 0 0 0 0 0 0 0 0 0 0 0 > > Number of blocks type Unmovable Movable Reclaimable HighAtomic CMA Isolate > Node 0, zone DMA 1 7 0 0 0 0 > Node 0, zone DMA32 4 1650 2 0 0 0 > Node 0, zone Normal 79 2326 26 0 0 0 > > You can see that the number of unmovable page blocks is decreased > when the kernel has this commit. > > Signed-off-by: Hui Zhu Reviewed-by: Shakeel Butt