Received: by 2002:ac0:bc90:0:0:0:0:0 with SMTP id a16csp120295img; Thu, 21 Mar 2019 15:38:30 -0700 (PDT) X-Google-Smtp-Source: APXvYqyk0FvnobLzL59o0DvEjQ1DEpvLdOAau7vJ7w3RrwtD962rznkqtpWjFJwht198Mi+vAAO6 X-Received: by 2002:a63:f212:: with SMTP id v18mr5758175pgh.261.1553207910862; Thu, 21 Mar 2019 15:38:30 -0700 (PDT) ARC-Seal: i=1; a=rsa-sha256; t=1553207910; cv=none; d=google.com; s=arc-20160816; b=PbJ84jhfw+tG3tKWrEjXyJTLuXqoXJPnMCV90VlyXQeUQ7gUAn+DIZBnQiWtcV/qhI lFQMMwk7lEH8TlWLKpHT9QIi7IJZWre9kV1ufCjibGh+D9wn8Bq68GRSZNoD59DqA3EI 2vnhHJCq//s2EWlZDvpOPc3hENn19ydHtQjnpJMSQ6qxSWJpYU8kIhaJKyOCF1PY+ZR1 JQfgl95atWEGJqXy9PhBOG1VViRiQvE91clOEzzTdARAjCMq7DcVsJ/cDjfi2zssHGKj ScQfntQVuytGRTrhF++cqvA8TLd5e2s66MGs9wi5b84M4SNILr915xKkqwOYbDuNO5v9 ghDg== ARC-Message-Signature: i=1; a=rsa-sha256; c=relaxed/relaxed; d=google.com; s=arc-20160816; h=list-id:precedence:sender:user-agent:in-reply-to :content-transfer-encoding:content-disposition:mime-version :references:message-id:subject:cc:to:from:date; bh=/J9nv2hX6UWKMr2II9vzpM3SerBuBExGQmcOvxECRaM=; b=lqCIJ9YBuQZshO0lqcwYZ8+ZLxwPL5h1hZL04bedtEiC16E6TmG5hmLsc2Pyae3IOu RUjGSQ1C+pxa7p9RV2D2d4q+sCsSVlcJfM2oNxW7WFF0wReBW9vQcFC87f2wUqRuzJeV LMdTnWsyLt2i4ZlCc9+cArFrHDLpiVELt14QxJEnuHGP2I8BBBPO1vBjrhkfEwx1Uavh M8HwZjNWsXokJkmDQ2h0NyUvEAXKewOqoRb3kNJBZJvFekYc9zLcWyYFGs/mknCQixZE YHhT1J8BBnPjoi80Yj3MuByXrBLAQbMFQuz3CxqS8BxEQM4MVM/sR/79EKsHTgUx+t5P o/1A== ARC-Authentication-Results: i=1; mx.google.com; spf=pass (google.com: best guess record for domain of linux-kernel-owner@vger.kernel.org designates 209.132.180.67 as permitted sender) smtp.mailfrom=linux-kernel-owner@vger.kernel.org; dmarc=fail (p=NONE sp=NONE dis=NONE) header.from=intel.com Return-Path: Received: from vger.kernel.org (vger.kernel.org. [209.132.180.67]) by mx.google.com with ESMTP id f131si5291175pfc.92.2019.03.21.15.38.15; Thu, 21 Mar 2019 15:38:30 -0700 (PDT) Received-SPF: pass (google.com: best guess record for domain of linux-kernel-owner@vger.kernel.org designates 209.132.180.67 as permitted sender) client-ip=209.132.180.67; Authentication-Results: mx.google.com; spf=pass (google.com: best guess record for domain of linux-kernel-owner@vger.kernel.org designates 209.132.180.67 as permitted sender) smtp.mailfrom=linux-kernel-owner@vger.kernel.org; dmarc=fail (p=NONE sp=NONE dis=NONE) header.from=intel.com Received: (majordomo@vger.kernel.org) by vger.kernel.org via listexpand id S1726897AbfCUWgJ (ORCPT + 99 others); Thu, 21 Mar 2019 18:36:09 -0400 Received: from mga05.intel.com ([192.55.52.43]:22607 "EHLO mga05.intel.com" rhost-flags-OK-OK-OK-OK) by vger.kernel.org with ESMTP id S1726068AbfCUWgI (ORCPT ); Thu, 21 Mar 2019 18:36:08 -0400 X-Amp-Result: UNKNOWN X-Amp-Original-Verdict: FILE UNKNOWN X-Amp-File-Uploaded: False Received: from fmsmga004.fm.intel.com ([10.253.24.48]) by fmsmga105.fm.intel.com with ESMTP/TLS/DHE-RSA-AES256-GCM-SHA384; 21 Mar 2019 15:36:08 -0700 X-ExtLoop1: 1 X-IronPort-AV: E=Sophos;i="5.60,254,1549958400"; d="scan'208";a="154574896" Received: from unknown (HELO localhost.localdomain) ([10.232.112.69]) by fmsmga004.fm.intel.com with ESMTP; 21 Mar 2019 15:36:08 -0700 Date: Thu, 21 Mar 2019 16:37:08 -0600 From: Keith Busch To: Zi Yan Cc: linux-kernel@vger.kernel.org, linux-mm@kvack.org, linux-nvdimm@lists.01.org, Dave Hansen , Dan Williams , "Kirill A. Shutemov" , John Hubbard , Michal Hocko , David Nellans Subject: Re: [PATCH 0/5] Page demotion for memory reclaim Message-ID: <20190321223706.GA29817@localhost.localdomain> References: <20190321200157.29678-1-keith.busch@intel.com> <5B5EFBC2-2979-4B9F-A43A-1A14F16ACCE1@nvidia.com> MIME-Version: 1.0 Content-Type: text/plain; charset=utf-8 Content-Disposition: inline Content-Transfer-Encoding: 8bit In-Reply-To: <5B5EFBC2-2979-4B9F-A43A-1A14F16ACCE1@nvidia.com> User-Agent: Mutt/1.9.1 (2017-09-22) Sender: linux-kernel-owner@vger.kernel.org Precedence: bulk List-ID: X-Mailing-List: linux-kernel@vger.kernel.org On Thu, Mar 21, 2019 at 02:20:51PM -0700, Zi Yan wrote: > 1. The name of “page demotion” seems confusing to me, since I thought it was about large pages > demote to small pages as opposite to promoting small pages to THPs. Am I the only > one here? If you have a THP, we'll skip the page migration and fall through to split_huge_page_to_list(), then the smaller pages can be considered, migrated and reclaimed individually. Not that we couldn't try to migrate a THP directly. It was just simpler implementation for this first attempt. > 2. For the demotion path, a common case would be from high-performance memory, like HBM > or Multi-Channel DRAM, to DRAM, then to PMEM, and finally to disks, right? More general > case for demotion path would be derived from the memory performance description from HMAT[1], > right? Do you have any algorithm to form such a path from HMAT? Yes, I have a PoC for the kernel setting up a demotion path based on HMAT properties here: https://git.kernel.org/pub/scm/linux/kernel/git/kbusch/linux.git/commit/?h=mm-migrate&id=4d007659e1dd1b0dad49514348be4441fbe7cadb The above is just from an experimental branch. > 3. Do you have a plan for promoting pages from lower-level memory to higher-level memory, > like from PMEM to DRAM? Will this one-way demotion make all pages sink to PMEM and disk? Promoting previously demoted pages would require the application do something to make that happen if you turn demotion on with this series. Kernel auto-promotion is still being investigated, and it's a little trickier than reclaim. If it sinks to disk, though, the next access behavior is the same as before, without this series. > 4. In your patch 3, you created a new method migrate_demote_mapping() to migrate pages to > other memory node, is there any problem of reusing existing migrate_pages() interface? Yes, we may not want to migrate everything in the shrink_page_list() pages. We might want to keep a page, so we have to do those checks first. At the point we know we want to attempt migration, the page is already locked and not in a list, so it is just easier to directly invoke the new __unmap_and_move_locked() that migrate_pages() eventually also calls. > 5. In addition, you only migrate base pages, is there any performance concern on migrating THPs? > Is it too costly to migrate THPs? It was just easier to consider single pages first, so we let a THP split if possible. I'm not sure of the cost in migrating THPs directly.