Received-SPF: softfail (google.com: domain of transitioning linux-kernel-owner@vger.kernel.org does not designate 23.128.96.19 as permitted sender) client-ip=23.128.96.19;
Date:   Fri, 22 Apr 2022 14:43:03 +0800
From:   Aaron Lu <aaron.lu@intel.com>
To:     "ying.huang@intel.com" <ying.huang@intel.com>
CC:     Yang Shi <shy828301@gmail.com>, Michal Hocko <mhocko@suse.com>,
        "Andrew Morton" <akpm@linux-foundation.org>,
        Linux MM <linux-mm@kvack.org>,
        "Linux Kernel Mailing List" <linux-kernel@vger.kernel.org>
Subject: Re: [PATCH] mm: swap: determine swap device by using page nid
Message-ID: <YmJOd5BlyNvB9fX4@ziqianlu-desk1>
References: <20220407020953.475626-1-shy828301@gmail.com>
 <Yk6cutNf5sOuYbDl@ziqianlu-nuc9qn>
 <CAHbLzkq+eKcKCsxXDTiOcBxk8FjMdWBqOxwi4N_NG7PZWbAAkA@mail.gmail.com>
 <Yl/FS9enAD4V8jG3@ziqianlu-nuc9qn>
 <f27ec36beb3cf1dbbfc3b8835e586d5d6fe7f561.camel@intel.com>
 <YmETEHnK/FiNbLO7@ziqianlu-desk1>
 <6f7210be7353d1c01dc9f872b2692b83f87f5452.camel@intel.com>
 <YmJKBaq1yj6/iBJ3@ziqianlu-desk1>
 <4f1bc4dc65117a185833555ff8df30a944453499.camel@intel.com>
Content-Type: text/plain; charset="iso-8859-1"
Content-Disposition: inline
Content-Transfer-Encoding: 8bit
In-Reply-To: <4f1bc4dc65117a185833555ff8df30a944453499.camel@intel.com>
MIME-Version: 1.0
X-MS-Exchange-AntiSpam-MessageData-ChunkCount: 1
X-MS-Exchange-AntiSpam-MessageData-0: =?iso-8859-1?Q?RepuYTYSo8O+lFoaPVviYJ5HxxYWct1yeJDNDGFN/PpQxC6R28gvPC/zbE?=
 =?iso-8859-1?Q?Ukq48I+0p1StFU9mTJT/ERmS0xcproTIQXqGrWrIKYW75/r+62buHNI7A0?=
 =?iso-8859-1?Q?pPp2Tp5hhjDmxjBse/aCfTYkWi9zXuStzzf2Fp1AFSy9RvpTE3Mk6fODI9?=
 =?iso-8859-1?Q?sT0xhFtwtTqr54Sfi4k+A9iIQ/zFSNTPXNAhe5UhHCnTZcHFs3Yd2KWDzi?=
 =?iso-8859-1?Q?i7qVrlk4z0+2nEoHKwsHpcg4NAr6TIUmtiu92jwGD4Bo78PG/KWUOu9A/B?=
 =?iso-8859-1?Q?sfMppwD+lrTffQcisyEseXVtGc/qmK9Z+4HbFp0PIgLhyifjYMF/fGe0Ba?=
 =?iso-8859-1?Q?hoU7+gsc4Y9UpXhGBIdQpUBJZiIvIl/1YPHLHrY3R/LWPUlr5gBartNZqI?=
 =?iso-8859-1?Q?xA0he/uLbrE0pKJb4Bxb2abOpvUd6hWNjen8yypGXNnHC2KW0EfS0CitRc?=
 =?iso-8859-1?Q?M0alM+P6F+7kWXtHlQw1u4DcgMpuEaskHJk6Vrt28B6xnrz5koXhyX+PrJ?=
 =?iso-8859-1?Q?COlnIAWpgw0V/QN3eoVH6ZHQyfpS0LgL/5wyynazxSyXsqaKd27u6w1tRj?=
 =?iso-8859-1?Q?aLca0KQ9cp8p3JqSnmh4ci9hH01MHoC/NkbBAM+uJAblRa6bgR0SMi6ZDh?=
 =?iso-8859-1?Q?mcmp3Bz3hHbcCJZfKogwoCXqL9yZ5ZLP1N5XrFhugqytyPhB2vsigUXhNd?=
 =?iso-8859-1?Q?07vlrkTgLysy0At1d2o6sFf4y7pkxu2XGLI/KwajUYqnu+qgyZG3Hhv8GQ?=
 =?iso-8859-1?Q?8CWW50UIYzVMhgvg2xe7xE3F8SX0PVt0KOZrRHHd8lHcezConud72eUQB9?=
 =?iso-8859-1?Q?rZ7hIXp93niVSfphLd9LY28U50MYqC0d1S/+4HEkHGHQcacFY6zz86aV0y?=
 =?iso-8859-1?Q?nIvJN0Wx+zQcClBQfRrxSwMUMaSgZs+wiyR9F2kaniCIcMa/nF2Cfde1rI?=
 =?iso-8859-1?Q?bDZrhZI2uk0q4A+KUI0VwWnxE509rJOZ8LBDIYm5d6pf4tWsYcx91q8SoB?=
 =?iso-8859-1?Q?PEj3mfLFm+9NyFeIHiFcuoTubTElT90hcmSCvWdEfguYmTbTFSkqvCLhU6?=
 =?iso-8859-1?Q?Rccl1Td6OQa/yaFU8BT6M1UCFsKbau5lJ5jSdgfo1f0a3LMdvAUe50iPLt?=
 =?iso-8859-1?Q?BLyICNn8wkaqtSqwuF0aQ7C+ypMBSbA0R6b1N3klkLEMGDzckdXwCoiFUb?=
 =?iso-8859-1?Q?JAeeiQcYFvB2ggpI9hsxQl9NPh525/CgyH3P1+eputpChoW3tMkG5JLi5D?=
 =?iso-8859-1?Q?DS2JBLxtnUY5ogFllcFg8btcN64x1l4vR4/jgrG5pONpHWaC+wsG6DcTfq?=
 =?iso-8859-1?Q?syym7mwElwxOGT6EdEAJEzIZETwnHhuuekky3YLI2AZzOxH1N9Tqv20o8I?=
 =?iso-8859-1?Q?9Ckd/Se3PDSVFQvKW8TGVNt/7/fJuozKFzhaOx3QDwCf2nqdXSH6BqRHRW?=
 =?iso-8859-1?Q?vUSfSj35BuJmPN6IthWQaDMo53fa+dOIiugCvcYmkVM3RO9BQiv/ADo37H?=
 =?iso-8859-1?Q?E/btuZNiBDSWszHehqN/zLwEi4xFhuvKZOZV263LpXUzBOLfCXzgLtf+oa?=
 =?iso-8859-1?Q?/g2mKqETPoqK//jItZ3UdpFm918WbdtVbX537GAH3E5nl5FkwO6Jpw9fb/?=
 =?iso-8859-1?Q?ejVvlkQBFOVP7U48iW2p6wgaQsNS6R59rUKbzHwUtKF1UqXxhm+ewhu3i+?=
 =?iso-8859-1?Q?BBJPeEQ05PW6GNucL4BrwkgH0iwhe8v8oUsUgcFox3u9kcUu+oDtLokmtT?=
 =?iso-8859-1?Q?7DHJDZQa6XUwJrLXvsYohQPeZCSsNwns4KYV3jD/0wZzaKsv6vvCIBGiOF?=
 =?iso-8859-1?Q?5ie7+Z0gdg=3D=3D?=
X-MS-Exchange-CrossTenant-Network-Message-Id: 995b9b2f-ad8d-45d1-f9d0-08da242b5e08
X-MS-Exchange-CrossTenant-AuthSource: BYAPR11MB3062.namprd11.prod.outlook.com
X-MS-Exchange-CrossTenant-AuthAs: Internal
X-MS-Exchange-CrossTenant-OriginalArrivalTime: 22 Apr 2022 06:43:10.7436
 (UTC)
X-MS-Exchange-CrossTenant-FromEntityHeader: Hosted
X-MS-Exchange-CrossTenant-Id: 46c98d88-e344-4ed4-8496-4ed7712e255d
X-MS-Exchange-CrossTenant-MailboxType: HOSTED
X-MS-Exchange-CrossTenant-UserPrincipalName: exTLAip5B+TPt2UXCRWaNv/Rw0FRj+Iwxel7C7H9JN9lwSmgtVomPnDy759DcvJ8+dZZnJzhrTDQdgYuVkMasA==
X-MS-Exchange-Transport-CrossTenantHeadersStamped: CY4PR1101MB2088
X-OriginatorOrg: intel.com
X-Spam-Status: No, score=-2.5 required=5.0 tests=BAYES_00,DKIMWL_WL_HIGH,
	DKIM_SIGNED,DKIM_VALID,DKIM_VALID_AU,HEADER_FROM_DIFFERENT_DOMAINS,
	MAILING_LIST_MULTI,RDNS_NONE,SPF_HELO_NONE autolearn=unavailable
	autolearn_force=no version=3.4.6
X-Spam-Checker-Version: SpamAssassin 3.4.6 (2021-04-09) on
	lindbergh.monkeyblade.net
Precedence: bulk
List-ID: <linux-kernel.vger.kernel.org>
X-Mailing-List: linux-kernel@vger.kernel.org

On Fri, Apr 22, 2022 at 02:27:45PM +0800, ying.huang@intel.com wrote:
> On Fri, 2022-04-22 at 14:24 +0800, Aaron Lu wrote:
> > On Thu, Apr 21, 2022 at 04:34:09PM +0800, ying.huang@intel.com wrote:
> > > On Thu, 2022-04-21 at 16:17 +0800, Aaron Lu wrote:
> > > > On Thu, Apr 21, 2022 at 03:49:21PM +0800, ying.huang@intel.com wrote:
> > 
> > ... ...
> > 
> > > > > For swap-in latency, we can use pmbench, which can output latency
> > > > > information.
> > > > > 
> > > > 
> > > > OK, I'll give pmbench a run, thanks for the suggestion.
> > > 
> > > Better to construct a senario with more swapin than swapout.  For
> > > example, start a memory eater, then kill it later.
> > 
> > What about vm-scalability/case-swapin?
> > https://git.kernel.org/pub/scm/linux/kernel/git/wfg/vm-scalability.git/tree/case-swapin
> > 
> > I think you are pretty familiar with it but still:
> > 1) it starts $nr_task processes and each mmaps $size/$nr_task area and
> > ???then consumes the memory, after this, it waits for a signal;
> > 2) start another process to consume $size memory to push the memory in
> > ???step 1) to swap device;
> > 3) kick processes in step 1) to start accessing their memory, thus
> > ???trigger swapins. The metric of this testcase is the swapin throughput.
> > 
> > I plan to restrict the cgroup's limit to $size.
> > 
> > Considering there is only one NVMe drive attached to node 0, I will run
> > the test as described before:
> > 1) bind processes to run on node 0, allocate on node 1 to test the
> > ???performance when reclaimer's node id is the same as swap device's.
> > 2) bind processes to run on node 1, allocate on node 0 to test the
> > ???performance when page's node id is the same as swap device's.
> > 
> > Ying and Yang,
> > 
> > Let me know what you think about the case used and the way the test is
> > conducted.
> 
> The test case looks good to me.  And, do you have a way to measure swap
> in latency?  Better to compare between enabling and disabling per-node

By swap in latency, do you mean the time it takes for a fault that is
served by swap in?

Since the test is swap in only, I think throughput can tell us the
average swap in latency?

> swap device support too to make sure per-node support has performance
> impact on this system.

I think we can tell by conducting two more tests:
1) bind processes to run on node 0, allocate on node 0;
2) bind processes to run on node 1, allocate on node 1.
If case 1) is faster, we can say per-node support has performance impact
on this system.