Received-SPF: pass (google.com: domain of linux-kernel-owner@vger.kernel.org designates 23.128.96.18 as permitted sender) client-ip=23.128.96.18;
Date:   Fri, 26 Jun 2020 14:47:06 +0200
From:   Michal Hocko <mhocko@kernel.org>
To:     Daniel Jordan <daniel.m.jordan@oracle.com>
Cc:     linux-mm@kvack.org, linux-kernel@vger.kernel.org,
        Andrew Morton <akpm@linux-foundation.org>,
        Andy Lutomirski <luto@kernel.org>,
        Dave Hansen <dave.hansen@linux.intel.com>,
        David Hildenbrand <david@redhat.com>,
        Pavel Tatashin <pasha.tatashin@soleen.com>,
        Peter Zijlstra <peterz@infradead.org>,
        Steven Sistare <steven.sistare@oracle.com>
Subject: Re: [PATCH v2] x86/mm: use max memory block size on bare metal
Message-ID: <20200626124706.GZ1320@dhcp22.suse.cz>
References: <20200609225451.3542648-1-daniel.m.jordan@oracle.com>
 <20200619120704.GD12177@dhcp22.suse.cz>
 <20200622191739.4lekqrmjnzv2vwl2@ca-dmjordan1.us.oracle.com>
MIME-Version: 1.0
Content-Type: text/plain; charset=us-ascii
Content-Disposition: inline
In-Reply-To: <20200622191739.4lekqrmjnzv2vwl2@ca-dmjordan1.us.oracle.com>
Sender: linux-kernel-owner@vger.kernel.org
Precedence: bulk

On Mon 22-06-20 15:17:39, Daniel Jordan wrote:
> Hello Michal,
> 
> (I've been away and may be slow to respond for a little while)
> 
> On Fri, Jun 19, 2020 at 02:07:04PM +0200, Michal Hocko wrote:
> > On Tue 09-06-20 18:54:51, Daniel Jordan wrote:
> > [...]
> > > @@ -1390,6 +1391,15 @@ static unsigned long probe_memory_block_size(void)
> > >  		goto done;
> > >  	}
> > >  
> > > +	/*
> > > +	 * Use max block size to minimize overhead on bare metal, where
> > > +	 * alignment for memory hotplug isn't a concern.
> > 
> > This really begs a clarification why this is not a concern. Bare metal
> > can see physical memory hotadd as well. I just suspect that you do not
> > consider that to be very common so it is not a big deal?
> 
> It's not only uncommon, it's also that boot_mem_end on bare metal may not align
> with any available memory block size.  For instance, this server's boot_mem_end
> is only 4M aligned and FWIW my desktop's is 2M aligned.  As far as I can tell,
> the logic that picks the size wasn't intended for bare metal.

> 
> > And I would
> > tend to agree but still we are just going to wait until first user
> > stumbles over this.
> 
> This isn't something new with this patch, 2G has been the default on big
> machines for years.  This is addressing an unintended side effect of
> 078eb6aa50dc50, which was for qemu, by restoring the original behavior on bare
> metal to avoid oodles of sysfs files.

I am not really sure the qemu was a target. I suspect it was just easier
to test in qemu.

[...]

> > I believe that we should think about a future interface rather than
> > trying to ducktape the blocksize anytime it causes problems. I would be
> > even tempted to simply add a kernel command line option 
> > memory_hotplug=disable,legacy,new_shiny
> > 
> > for disable it would simply drop all the sysfs crud and speed up boot
> > for most users who simply do not care about memory hotplug. new_shiny
> > would ideally provide an interface that would either export logically
> > hotplugable memory ranges (e.g. DIMMs) or a query/action interface which
> > accepts physical ranges as input. Having gazillions of sysfs files is
> > simply unsustainable.
> 
> So in this idea, presumably the default would start off being legacy and then
> later be changed to new_shiny?

Well it really depends. Going with disable as a default would suit most
users much better because the vast majority simply doesn't use the
functionality. On the other hand real users would regress unless they
enable the option. Which is definitely not nice. Another and much less
intrusive change would be creating sysfs interface on-demand. So until
somebody actually tries to use the interface it won't exist. I haven't
tried to explore how complex that would be. I am not really familiar
with sysfs to be honest. But fundamentally nothing should prevent such a
solution.

Another option would be to create sysfs interface only if there is a
hotplugable memory reported by the platform. But I am not sure we have a
proper interface for that for all arches.
-- 
Michal Hocko
SUSE Labs