Received: by 2002:ac0:a582:0:0:0:0:0 with SMTP id m2-v6csp1055516imm; Wed, 10 Oct 2018 08:28:43 -0700 (PDT) X-Google-Smtp-Source: ACcGV63mHMItB0B+YhY02bJ465i2Re9+LrKbQBNjYsdeGxyI9QQqsEJbxyWLnRl+JV8dAk6mHPnO X-Received: by 2002:a17:902:be18:: with SMTP id r24-v6mr4842913pls.341.1539185323889; Wed, 10 Oct 2018 08:28:43 -0700 (PDT) ARC-Seal: i=1; a=rsa-sha256; t=1539185323; cv=none; d=google.com; s=arc-20160816; b=ssmyFyBjAtZrBs+Z99+i8DXYX+3wBqtExwlZU5u+VxOWW8rQ4f/D1DaUdlJ4UuR3PH JR60i09q/rGQKt/kdnbUty8hm/L7h+MHFpI3jV3x95vZeQaK+EaH20Dol6hFB4J9DnnL m+j1qvf3f+ZDjH47rUAgoKGoPuBj719yr3n5zreNfv2AgsNCg5d62XTpOOr++X69EWj0 lHjpU6P8rXD9zTQbVebTWVA/isFPCWHva/moz66exaPfREJ6qfJBFZBg9N9DkEEMEg0m 5AxUaoLMsURQPBkNg8w/HPxvyh3nx2Q6MKDnloEbsKCrM2QNoFngQOxz6u6QdIvt0o3/ 1bvA== ARC-Message-Signature: i=1; a=rsa-sha256; c=relaxed/relaxed; d=google.com; s=arc-20160816; h=list-id:precedence:sender:content-transfer-encoding :content-language:in-reply-to:mime-version:user-agent:date :message-id:from:references:to:subject; bh=NP9AH6jrvhhDcUQdGQjesgFQxxzb3jqydyo7xC6kNBc=; b=H3nuExuq2wJB3c+K2TUgqrY3TJYc7xK27hO9T4+vIkuJU9kFvedc39JBV116+VsqPV +QscM1c9Tn1X3+QULVaZLPt2CjtCWCFL6CEKXOEAPCDfysA9De+hi9E1UI/OuarUQbIR Tc5kXryeqI8Af+YtEj6WSGJklW50NvGpGsnURQECsnUzlJY4doZ9MtjYYU5sG/j++oNV TUBQk3XqA8mb02k2xzoxIgyjx1+MDorpArK0BPDZwuNG+EJR/qiyyzMX48/okk7Y5O+d jTCP4XTCs2GK1YpTUpoRH2d8coE1efCquPelSPv+F6rMavmH7aYxa7H4w4VDd/ToDo8x I1Jg== ARC-Authentication-Results: i=1; mx.google.com; spf=pass (google.com: best guess record for domain of linux-kernel-owner@vger.kernel.org designates 209.132.180.67 as permitted sender) smtp.mailfrom=linux-kernel-owner@vger.kernel.org; dmarc=fail (p=NONE sp=NONE dis=NONE) header.from=intel.com Return-Path: Received: from vger.kernel.org (vger.kernel.org. [209.132.180.67]) by mx.google.com with ESMTP id l70-v6si24145619pga.498.2018.10.10.08.28.27; Wed, 10 Oct 2018 08:28:43 -0700 (PDT) Received-SPF: pass (google.com: best guess record for domain of linux-kernel-owner@vger.kernel.org designates 209.132.180.67 as permitted sender) client-ip=209.132.180.67; Authentication-Results: mx.google.com; spf=pass (google.com: best guess record for domain of linux-kernel-owner@vger.kernel.org designates 209.132.180.67 as permitted sender) smtp.mailfrom=linux-kernel-owner@vger.kernel.org; dmarc=fail (p=NONE sp=NONE dis=NONE) header.from=intel.com Received: (majordomo@vger.kernel.org) by vger.kernel.org via listexpand id S1726717AbeJJWup (ORCPT + 99 others); Wed, 10 Oct 2018 18:50:45 -0400 Received: from mga01.intel.com ([192.55.52.88]:14158 "EHLO mga01.intel.com" rhost-flags-OK-OK-OK-OK) by vger.kernel.org with ESMTP id S1726666AbeJJWup (ORCPT ); Wed, 10 Oct 2018 18:50:45 -0400 X-Amp-Result: SKIPPED(no attachment in message) X-Amp-File-Uploaded: False Received: from orsmga001.jf.intel.com ([10.7.209.18]) by fmsmga101.fm.intel.com with ESMTP/TLS/DHE-RSA-AES256-GCM-SHA384; 10 Oct 2018 08:28:04 -0700 X-ExtLoop1: 1 X-IronPort-AV: E=Sophos;i="5.54,364,1534834800"; d="scan'208";a="98198130" Received: from ahduyck-mobl.amr.corp.intel.com (HELO [10.7.198.156]) ([10.7.198.156]) by orsmga001.jf.intel.com with ESMTP; 10 Oct 2018 08:27:59 -0700 Subject: Re: [PATCH v5 4/4] mm: Defer ZONE_DEVICE page initialization to the point where we init pgmap To: Dan Williams , Pasha Tatashin , Michal Hocko , linux-nvdimm , Dave Hansen , Linux Kernel Mailing List , Linux MM , =?UTF-8?B?SsOpcsO0bWUgR2xpc3Nl?= , rppt@linux.vnet.ibm.com, Andrew Morton , Ingo Molnar , "Kirill A. Shutemov" References: <20180925200551.3576.18755.stgit@localhost.localdomain> <20180925202053.3576.66039.stgit@localhost.localdomain> <20181009170051.GA40606@tiger-server> <25092df0-b7b4-d456-8409-9c004cb6e422@linux.intel.com> <20181010125211.GA45572@tiger-server> From: Alexander Duyck Message-ID: Date: Wed, 10 Oct 2018 08:27:58 -0700 User-Agent: Mozilla/5.0 (Windows NT 10.0; WOW64; rv:60.0) Gecko/20100101 Thunderbird/60.2.1 MIME-Version: 1.0 In-Reply-To: <20181010125211.GA45572@tiger-server> Content-Type: text/plain; charset=utf-8; format=flowed Content-Language: en-US Content-Transfer-Encoding: 7bit Sender: linux-kernel-owner@vger.kernel.org Precedence: bulk List-ID: X-Mailing-List: linux-kernel@vger.kernel.org On 10/10/2018 5:52 AM, Yi Zhang wrote: > On 2018-10-09 at 14:19:32 -0700, Dan Williams wrote: >> On Tue, Oct 9, 2018 at 1:34 PM Alexander Duyck >> wrote: >>> >>> On 10/9/2018 11:04 AM, Dan Williams wrote: >>>> On Tue, Oct 9, 2018 at 3:21 AM Yi Zhang wrote: >> [..] >>>> That comment is incorrect, device-pages are never onlined. So I think >>>> we can just skip that call to __SetPageReserved() unless the memory >>>> range is MEMORY_DEVICE_{PRIVATE,PUBLIC}. >>>> >>> >>> When pages are "onlined" via __free_pages_boot_core they clear the >>> reserved bit, that is the reason for the comment. The reserved bit is >>> meant to indicate that the page cannot be swapped out or moved based on >>> the description of the bit. >> >> ...but ZONE_DEVICE pages are never onlined so I would expect >> memmap_init_zone_device() to know that detail. >> >>> I would think with that being the case we still probably need the call >>> to __SetPageReserved to set the bit with the expectation that it will >>> not be cleared for device-pages since the pages are not onlined. >>> Removing the call to __SetPageReserved would probably introduce a number >>> of regressions as there are multiple spots that use the reserved bit to >>> determine if a page can be swapped out to disk, mapped as system memory, >>> or migrated. > > Another things, it seems page_init/set_reserved already been done in the > move_pfn_range_to_zone > |-->memmap_init_zone > |-->for_each_page_in_pfn > |-->__init_single_page > |-->SetPageReserved > > Why we haven't remove these redundant initial in memmap_init_zone? > > Correct me if I missed something. In this case it isn't redundant as only the vmmemmap pages are initialized in memmap_init_zone now. So all of the pages that are going to be used as device pages are not initialized until the call to memmap_init_zone_device. What I did is split the initialization of the pages into two parts in order to allow us to initialize the pages outside of the hotplug lock. >> >> Right, this is what Yi is working on... the PageReserved flag is >> problematic for KVM. Auditing those locations it seems as long as we >> teach hibernation to avoid ZONE_DEVICE ranges we can safely not set >> the reserved flag for DAX pages. What I'm trying to avoid is a local >> KVM hack to check for DAX pages when the Reserved flag is not >> otherwise needed. > Thanks Dan. Provide the patch link. > > https://lore.kernel.org/lkml/cover.1536342881.git.yi.z.zhang@linux.intel.com So it looks like your current logic is just working around the bit then since it just allows for reserved DAX pages.