Received: by 2002:a25:c205:0:0:0:0:0 with SMTP id s5csp444383ybf; Fri, 28 Feb 2020 00:56:23 -0800 (PST) X-Google-Smtp-Source: APXvYqyATIZGHvVAfCFM+wP62D+W5Hwqyvn3w8CLE/GSQ9jGj2Kr4HIWpYtEpiO7wN8gki2MUSA4 X-Received: by 2002:a05:6830:1d59:: with SMTP id p25mr2486456oth.308.1582880183409; Fri, 28 Feb 2020 00:56:23 -0800 (PST) ARC-Seal: i=1; a=rsa-sha256; t=1582880183; cv=none; d=google.com; s=arc-20160816; b=RlyxPL8JDB+V66gt409k0HwAz5dvKzVDIfPTGAoXt3rWg5Md6Sy1qZgCUuCw/UxahV 96I2HEuu+dGQtIsHOpQo3Vpd2aFa2RyPfqWQz+dtblajTQlBp07LUyePVMOVFUj9irBY wCCs0toHwA+O1ZVuUGoBsDXvngQgbsNa8gst71O3OpnGyTdKVALEy5+4qxEjVR1Qd5Ur Jb/TY1/8QgyNqVLxeTOocmx5dE7RehKmRmarsY7lmvcrxWnjR83nKW7WlTtCIoii5Je+ HBs8gMS9CbUqnu/2SGB5pReEvytq/YKe7l4nTOtsrobE58uJu0Xq0EZsUoYyv64WPEhi tJSw== ARC-Message-Signature: i=1; a=rsa-sha256; c=relaxed/relaxed; d=google.com; s=arc-20160816; h=list-id:precedence:sender:mime-version:user-agent:message-id :in-reply-to:date:references:subject:cc:to:from; bh=n0qs5mTOmeXPNqoJcWdPWz6WWW7kLsDhI9vXqAR9BsY=; b=kkaEP2BzX/x5rgRAuiz/ya8UbNE3BEfYlbG9qBIGxJ8eX4T/XjJyLGVAIqcDT9jL4V FY5A0JXTa9hac0EQ2fMz+sf9oRQ+HU0oxR8xkWiLU9q0KGAOYSv56TeIrd9wPUyLH1OL 8if6aMohkYFYGBzafRuhzEqshkwPmAzZPZZhEGhNv4ipmK98z11duBlro3l2BoyLMXPw e4o4Vw4qLNE4YSzZ7EfLSB7jGgpLZR9vPBdgaUnviJ+WnHR1eo+jZDQzo8B423V5ORYH 8vBKYCi+Gi+rFuuiX2WdfhNG2fIlsj7hBwm8M9UnevmzYBG3p0sc1a4Qwuo4OOZQHBc8 G2Uw== ARC-Authentication-Results: i=1; mx.google.com; spf=pass (google.com: best guess record for domain of linux-kernel-owner@vger.kernel.org designates 209.132.180.67 as permitted sender) smtp.mailfrom=linux-kernel-owner@vger.kernel.org; dmarc=fail (p=NONE sp=NONE dis=NONE) header.from=intel.com Return-Path: Received: from vger.kernel.org (vger.kernel.org. [209.132.180.67]) by mx.google.com with ESMTP id a23si1458775oie.81.2020.02.28.00.56.12; Fri, 28 Feb 2020 00:56:23 -0800 (PST) Received-SPF: pass (google.com: best guess record for domain of linux-kernel-owner@vger.kernel.org designates 209.132.180.67 as permitted sender) client-ip=209.132.180.67; Authentication-Results: mx.google.com; spf=pass (google.com: best guess record for domain of linux-kernel-owner@vger.kernel.org designates 209.132.180.67 as permitted sender) smtp.mailfrom=linux-kernel-owner@vger.kernel.org; dmarc=fail (p=NONE sp=NONE dis=NONE) header.from=intel.com Received: (majordomo@vger.kernel.org) by vger.kernel.org via listexpand id S1726673AbgB1Izz (ORCPT + 99 others); Fri, 28 Feb 2020 03:55:55 -0500 Received: from mga03.intel.com ([134.134.136.65]:44919 "EHLO mga03.intel.com" rhost-flags-OK-OK-OK-OK) by vger.kernel.org with ESMTP id S1726005AbgB1Izz (ORCPT ); Fri, 28 Feb 2020 03:55:55 -0500 X-Amp-Result: SKIPPED(no attachment in message) X-Amp-File-Uploaded: False Received: from fmsmga002.fm.intel.com ([10.253.24.26]) by orsmga103.jf.intel.com with ESMTP/TLS/DHE-RSA-AES256-GCM-SHA384; 28 Feb 2020 00:55:44 -0800 X-ExtLoop1: 1 X-IronPort-AV: E=Sophos;i="5.70,495,1574150400"; d="scan'208";a="272566543" Received: from yhuang-dev.sh.intel.com (HELO yhuang-dev) ([10.239.159.23]) by fmsmga002.fm.intel.com with ESMTP; 28 Feb 2020 00:55:40 -0800 From: "Huang\, Ying" To: David Hildenbrand Cc: Matthew Wilcox , Andrew Morton , , , Mel Gorman , Vlastimil Babka , Zi Yan , Michal Hocko , Peter Zijlstra , Dave Hansen , Minchan Kim , "Johannes Weiner" , Hugh Dickins , "Alexander Duyck" Subject: Re: [RFC 0/3] mm: Discard lazily freed pages when migrating References: <20200228033819.3857058-1-ying.huang@intel.com> <20200228034248.GE29971@bombadil.infradead.org> <87a7538977.fsf@yhuang-dev.intel.com> Date: Fri, 28 Feb 2020 16:55:40 +0800 In-Reply-To: (David Hildenbrand's message of "Fri, 28 Feb 2020 09:22:56 +0100") Message-ID: <871rqf850z.fsf@yhuang-dev.intel.com> User-Agent: Gnus/5.13 (Gnus v5.13) Emacs/26.1 (gnu/linux) MIME-Version: 1.0 Content-Type: text/plain; charset=ascii Sender: linux-kernel-owner@vger.kernel.org Precedence: bulk List-ID: X-Mailing-List: linux-kernel@vger.kernel.org David Hildenbrand writes: > On 28.02.20 08:25, Huang, Ying wrote: >> Hi, Matthew, >> >> Matthew Wilcox writes: >> >>> On Fri, Feb 28, 2020 at 11:38:16AM +0800, Huang, Ying wrote: >>>> MADV_FREE is a lazy free mechanism in Linux. According to the manpage >>>> of mavise(2), the semantics of MADV_FREE is, >>>> >>>> The application no longer requires the pages in the range specified >>>> by addr and len. The kernel can thus free these pages, but the >>>> freeing could be delayed until memory pressure occurs. ... >>>> >>>> Originally, the pages freed lazily by MADV_FREE will only be freed >>>> really by page reclaiming when there is memory pressure or when >>>> unmapping the address range. In addition to that, there's another >>>> opportunity to free these pages really, when we try to migrate them. >>>> >>>> The main value to do that is to avoid to create the new memory >>>> pressure immediately if possible. Instead, even if the pages are >>>> required again, they will be allocated gradually on demand. That is, >>>> the memory will be allocated lazily when necessary. This follows the >>>> common philosophy in the Linux kernel, allocate resources lazily on >>>> demand. >>> >>> Do you have an example program which does this (and so benefits)? >> >> Sorry, what do you mean exactly for "this" here? Call >> madvise(,,MADV_FREE)? Or migrate pages? >> >>> If so, can you quantify the benefit at all? >> >> The question is what is the right workload? For example, I can build a >> scenario as below to show benefit. > > We usually don't optimize for theoretical issues. Is there a real-life > workload you are trying to optimize this code for? We don't use a specific workload because we thought this is a general optimization. I will explain more later in this email. >> >> - run program A in node 0 with many lazily freed pages >> >> - run program B in node 1, so that the free memory on node 1 is low >> >> - migrate the program A from node 0 to node 1, so that the program B is >> influenced by the memory pressure created by migrating lazily freed >> pages. >> > > E.g., free page reporting in QEMU wants to use MADV_FREE. The guest will > report currently free pages to the hypervisor, which will MADV_FREE the > reported memory. As long as there is no memory pressure, there is no > need to actually free the pages. Once the guest reuses such a page, it > could happen that there is still the old page and pulling in in a fresh > (zeroed) page can be avoided. > > AFAIKs, after your change, we would get more pages discarded from our > guest, resulting in more fresh (zeroed) pages having to be pulled in > when a guest touches a reported free page again. But OTOH, page > migration is speed up (avoiding to migrate these pages). Let's look at this problem in another perspective. To migrate the MADV_FREE pages of the QEMU process from the node A to the node B, we need to free the original pages in the node A, and (maybe) allocate the same number of pages in the node B. So the question becomes - we may need to allocate some pages in the node B - these pages may be accessed by the application or not - we should allocate all these pages in advance or allocate them lazily when they are accessed. We thought the common philosophy in Linux kernel is to allocate lazily. That is, because we will always free the original pages in the node A, the question isn't whether we should free these MADV_FREE pages, but whether we should allocate the same number of pages in the node B before we know whether they are really needed. We thought this is similar as whether we should allocate all physical pages when mmap(). > However, one important question, will you always discard memory when > migrating pages, or only if there is memory pressure on the migration > target? We will always discard memory when migrating pages. Our reasoning is as above. Best Regards, Huang, Ying