MIME-Version: 1.0
References: <20230215145425.420125-1-usama.arif@bytedance.com> <62ee53770b4010f065346b7f2a1200013836be97.camel@infradead.org>
In-Reply-To: <62ee53770b4010f065346b7f2a1200013836be97.camel@infradead.org>
From:   Brian Gerst <brgerst@gmail.com>
Date:   Wed, 22 Feb 2023 07:08:04 -0500
Message-ID: <CAMzpN2gEeNY87+ywgizusPVG2fLiRx3v__3Bmqjo_eJLv+5k7Q@mail.gmail.com>
Subject: Re: [PATCH v9 0/8] Parallel CPU bringup for x86_64
To:     David Woodhouse <dwmw2@infradead.org>
Cc:     Usama Arif <usama.arif@bytedance.com>, tglx@linutronix.de,
        kim.phillips@amd.com, arjan@linux.intel.com, mingo@redhat.com,
        bp@alien8.de, dave.hansen@linux.intel.com, hpa@zytor.com,
        x86@kernel.org, pbonzini@redhat.com, paulmck@kernel.org,
        linux-kernel@vger.kernel.org, kvm@vger.kernel.org,
        rcu@vger.kernel.org, mimoja@mimoja.de, hewenliang4@huawei.com,
        thomas.lendacky@amd.com, seanjc@google.com, pmenzel@molgen.mpg.de,
        fam.zheng@bytedance.com, punit.agrawal@bytedance.com,
        simon.evans@bytedance.com, liangma@liangbit.com
Content-Type: text/plain; charset="UTF-8"
Content-Transfer-Encoding: quoted-printable
Precedence: bulk

On Wed, Feb 22, 2023 at 5:44 AM David Woodhouse <dwmw2@infradead.org> wrote=
:
>
> On Wed, 2023-02-15 at 14:54 +0000, Usama Arif wrote:
> > The main change over v8 is dropping the patch to avoid repeated saves o=
f MTRR
> > at boot time. It didn't make a difference to smpboot time and is indepe=
ndent
> > of parallel CPU bringup, so if needed can be explored in a separate pat=
chset.
> >
> > The patches have also been rebased to v6.2-rc8 and retested and the
> > improvement in boot time is the same as v8.
>
> Thanks for picking this up, Usama.
>
> So the next thing that might be worth looking at is allowing the APs
> all to be running their hotplug thread simultaneously, bringing
> themselves from CPUHP_BRINGUP_CPU to CPUHP_AP_ONLINE. This series eats
> the initial INIT/SIPI/SIPI latency, but if there's any significant time
> in the AP hotplug thread, that could be worth parallelising.
>
> There may be further wins in the INIT/SIPI/SIPI too. Currently we
> process each CPU at a time, sending INIT, SIPI, waiting 10=C2=B5s and
> sending another SIPI.
>
> What if we sent the first INIT+SIPI to all CPUs, then did another pass
> sending another SIPI only to those which hadn't already started running
> and set their bit in cpu_initialized_mask ?
>
> Might not be worth it, and there's an added complexity that they all
> have to wait for each other (on the real mode trampoline lock) before
> they can take their turn and get as far as setting their bit in
> cpu_initialized_mask. So we'd probably end up sending the second SIPI
> to most of them *anyway*.

Speaking of next steps, I have a followup patchset ready to go that
removes the global variables initial_stack, initial_gs, and
early_gdt_descr.  Should I send that now or wait until this patchset
lands in -tip?

--
Brian Gerst