Received-SPF: pass (google.com: domain of linux-kernel-owner@vger.kernel.org designates 2620:137:e000::3:6 as permitted sender) client-ip=2620:137:e000::3:6;
Date:   Tue, 28 Nov 2023 08:54:42 +0100
From:   Sumanth Korikkar <sumanthk@linux.ibm.com>
To:     David Hildenbrand <david@redhat.com>
Cc:     linux-mm <linux-mm@kvack.org>,
        Andrew Morton <akpm@linux-foundation.org>,
        Oscar Salvador <osalvador@suse.de>,
        Michal Hocko <mhocko@suse.com>,
        "Aneesh Kumar K.V" <aneesh.kumar@linux.ibm.com>,
        Anshuman Khandual <anshuman.khandual@arm.com>,
        Gerald Schaefer <gerald.schaefer@linux.ibm.com>,
        Alexander Gordeev <agordeev@linux.ibm.com>,
        Heiko Carstens <hca@linux.ibm.com>,
        Vasily Gorbik <gor@linux.ibm.com>,
        linux-s390 <linux-s390@vger.kernel.org>,
        LKML <linux-kernel@vger.kernel.org>
Subject: Re: [PATCH v3 1/5] mm/memory_hotplug: introduce
 MEM_PREPARE_ONLINE/MEM_FINISH_OFFLINE notifiers
Message-ID: <ZWWcwvcohncIH/RI@li-2b55cdcc-350b-11b2-a85c-a78bff51fc11.ibm.com>
References: <20231127082023.2079810-1-sumanthk@linux.ibm.com>
 <20231127082023.2079810-2-sumanthk@linux.ibm.com>
 <c7512dd7-5009-4230-a29e-ea13c2e3be3e@redhat.com>
MIME-Version: 1.0
Content-Type: text/plain; charset=us-ascii
Content-Disposition: inline
In-Reply-To: <c7512dd7-5009-4230-a29e-ea13c2e3be3e@redhat.com>
Precedence: bulk

On Mon, Nov 27, 2023 at 04:16:18PM +0100, David Hildenbrand wrote:
> On 27.11.23 09:20, Sumanth Korikkar wrote:
> > Introduce  MEM_PREPARE_ONLINE/MEM_FINISH_OFFLINE memory notifiers to
> > prepare the transition of memory to and from a physically accessible
> > state. This enhancement is crucial for implementing the "memmap on
> > memory" feature for s390 in a subsequent patch.
> > 
> > Platforms such as x86 can support physical memory hotplug via ACPI. When
> > there is physical memory hotplug, ACPI event leads to the memory
> > addition with the following callchain:
> > acpi_memory_device_add()
> >    -> acpi_memory_enable_device()
> >       -> __add_memory()
> > 
> > After this, the hotplugged memory is physically accessible, and altmap
> > support prepared, before the "memmap on memory" initialization in
> > memory_block_online() is called.
> > 
> > On s390, memory hotplug works in a different way. The available hotplug
> > memory has to be defined upfront in the hypervisor, but it is made
> > physically accessible only when the user sets it online via sysfs,
> > currently in the MEM_GOING_ONLINE notifier. This is too late and "memmap
> > on memory" initialization is performed before calling MEM_GOING_ONLINE
> > notifier.
> > 
> > During the memory hotplug addition phase, altmap support is prepared and
> > during the memory onlining phase s390 requires memory to be physically
> > accessible and then subsequently initiate the "memmap on memory"
> > initialization process.
> > 
> > The memory provider will handle new MEM_PREPARE_ONLINE /
> > MEM_FINISH_OFFLINE notifications and make the memory accessible.
> > 
> > The mhp_flag MHP_OFFLINE_INACCESSIBLE is introduced and is relevant when
> > used along with MHP_MEMMAP_ON_MEMORY, because the altmap cannot be
> > written (e.g., poisoned) when adding memory -- before it is set online.
> > This allows for adding memory with an altmap that is not currently made
> > available by a hypervisor. When onlining that memory, the hypervisor can
> > be instructed to make that memory accessible via the new notifiers and
> > the onlining phase will not require any memory allocations, which is
> > helpful in low-memory situations.
> > 
> > All architectures ignore unknown memory notifiers.  Therefore, the
> > introduction of these new notifiers does not result in any functional
> > modifications across architectures.
> > 
> > Suggested-by: Gerald Schaefer <gerald.schaefer@linux.ibm.com>
> > Suggested-by: David Hildenbrand <david@redhat.com>
> > Signed-off-by: Sumanth Korikkar <sumanthk@linux.ibm.com>
> > ---
...
> >   int mhp_init_memmap_on_memory(unsigned long pfn, unsigned long nr_pages,
> > -			      struct zone *zone)
> > +			      struct zone *zone, bool mhp_off_inaccessible)
> >   {
> >   	unsigned long end_pfn = pfn + nr_pages;
> >   	int ret, i;
> > @@ -1092,7 +1109,14 @@ int mhp_init_memmap_on_memory(unsigned long pfn, unsigned long nr_pages,
> >   	ret = kasan_add_zero_shadow(__va(PFN_PHYS(pfn)), PFN_PHYS(nr_pages));
> >   	if (ret)
> >   		return ret;
> > -
> > +	/*
> > +	 * Memory block is accessible at this stage and hence poison the struct
> > +	 * pages now.  If the memory block is accessible during memory hotplug
> > +	 * addition phase, then page poisining is already performed in
> > +	 * sparse_add_section().
> > +	 */
> > +	if (mhp_off_inaccessible)
> > +		page_init_poison_with_resched(pfn, nr_pages);
> 
> Can you elaborate why a simple page_init_poison() as for
> sparse_add_section() is insufficient?
>
Looks like cond_resched() is not needed. page_init_poison() could be
performed similar to when adding new memory in sparse_add_section().
IIUC, As memory is onlined in memory block granuality, this
cond_resched() wouldnt be needed then.

Thanks