Received-SPF: pass (google.com: domain of linux-kernel-owner@vger.kernel.org designates 23.128.96.38 as permitted sender) client-ip=23.128.96.38;
MIME-Version: 1.0
References: <20230915184904.1976183-1-evan@rivosinc.com>
In-Reply-To: <20230915184904.1976183-1-evan@rivosinc.com>
From:   Evan Green <evan@rivosinc.com>
Date:   Thu, 2 Nov 2023 15:41:58 -0700
Message-ID: <CALs-Hsu_1x2FpjWXbmioSi6x30NXdNgrQ=t=EBknB0O06DA12A@mail.gmail.com>
Subject: Re: [PATCH] RISC-V: Probe misaligned access speed in parallel
To:     Palmer Dabbelt <palmer@rivosinc.com>
Cc:     David Laight <David.Laight@aculab.com>,
        Jisheng Zhang <jszhang@kernel.org>,
        Albert Ou <aou@eecs.berkeley.edu>,
        Andrew Jones <ajones@ventanamicro.com>,
        Anup Patel <apatel@ventanamicro.com>,
        Conor Dooley <conor.dooley@microchip.com>,
        Greentime Hu <greentime.hu@sifive.com>,
        Ley Foon Tan <leyfoon.tan@starfivetech.com>,
        Marc Zyngier <maz@kernel.org>,
        Palmer Dabbelt <palmer@dabbelt.com>,
        Paul Walmsley <paul.walmsley@sifive.com>,
        Sunil V L <sunilvl@ventanamicro.com>,
        linux-kernel@vger.kernel.org, linux-riscv@lists.infradead.org
Content-Type: text/plain; charset="UTF-8"
Content-Transfer-Encoding: quoted-printable
Precedence: bulk

On Fri, Sep 15, 2023 at 11:49=E2=80=AFAM Evan Green <evan@rivosinc.com> wro=
te:
>
> Probing for misaligned access speed takes about 0.06 seconds. On a
> system with 64 cores, doing this in smp_callin() means it's done
> serially, extending boot time by 3.8 seconds. That's a lot of boot time.
>
> Instead of measuring each CPU serially, let's do the measurements on
> all CPUs in parallel. If we disable preemption on all CPUs, the
> jiffies stop ticking, so we can do this in stages of 1) everybody
> except core 0, then 2) core 0.
>
> The measurement call in smp_callin() stays around, but is now
> conditionalized to only run if a new CPU shows up after the round of
> in-parallel measurements has run. The goal is to have the measurement
> call not run during boot or suspend/resume, but only on a hotplug
> addition.
>
> Signed-off-by: Evan Green <evan@rivosinc.com>

Shoot, I saw the other thread [1] where it seems like my use of
alloc_pages() in this context is improper? I had thought I was
alright, as Documentation/core-api/memory-allocation.rst says:

 > If the allocation is performed from an atomic context, e.g interrupt
 > handler, use ``GFP_NOWAIT``.

Any tips for reproducing that splat? I have CONFIG_DEBUG_ATOMIC_SLEEP
on (it's in the defconfig), and lockdep, and I'm on Conor's
linux-6.6.y-rt, but so far I'm not seeing it.

-Evan

[1] https://lore.kernel.org/linux-riscv/ZUPWc7sY47l34lV+@xhacker/T/#t