MIME-Version: 1.0
References: <IA1PR11MB617175CA398D425B7489CE4889AF9@IA1PR11MB6171.namprd11.prod.outlook.com>
 <764CA486-6FB2-4667-B8CB-56E3AC31FD58@joelfernandes.org> <IA1PR11MB617118DAE8D72E2419558D6689AC9@IA1PR11MB6171.namprd11.prod.outlook.com>
In-Reply-To: <IA1PR11MB617118DAE8D72E2419558D6689AC9@IA1PR11MB6171.namprd11.prod.outlook.com>
From:   Joel Fernandes <joel@joelfernandes.org>
Date:   Tue, 28 Feb 2023 09:27:01 -0500
Message-ID: <CAEXW_YQwu4ccqWLTfFSwcJQOH6fbhe4SiHHa4XPUCvefiBZi9g@mail.gmail.com>
Subject: Re: [PATCH RFC v2] rcu: Add a minimum time for marking boot as completed
To:     "Zhuo, Qiuxu" <qiuxu.zhuo@intel.com>
Cc:     "linux-kernel@vger.kernel.org" <linux-kernel@vger.kernel.org>,
        Frederic Weisbecker <frederic@kernel.org>,
        Lai Jiangshan <jiangshanlai@gmail.com>,
        "linux-doc@vger.kernel.org" <linux-doc@vger.kernel.org>,
        "Paul E. McKenney" <paulmck@kernel.org>,
        "rcu@vger.kernel.org" <rcu@vger.kernel.org>
Content-Type: text/plain; charset="UTF-8"
Content-Transfer-Encoding: quoted-printable
Precedence: bulk

On Tue, Feb 28, 2023 at 1:40=E2=80=AFAM Zhuo, Qiuxu <qiuxu.zhuo@intel.com> =
wrote:
>
[...]
> > > If so, we might make rcupdate.boot_end_delay =3D 0 as the default val=
ue
> > > (NOT the default 20s) for CONFIG_PREEMPT_RT=3Dy kernels?
> >
> > Could you measure how much time your RT system takes to boot before the
> > application runs?
>
> I don't have a real-time OS environment to measure the OS boot time.
> I tried to measure the OS boot time of my "CentOS Stream 8" w/o and
> w/ Joel=E2=80=99s patch.
>
> My testing showed the positive result that the OS boot time was
> reduced by ~4.6% on my side after applying Joel=E2=80=99s patch.

Wow, this is great! I am guessing you have CONFIG_RCU_LAZY disabled,
when you tested. If so, that is great news that expediting RCU for a
bit longer improves boot time! Please confirm that your config had
LAZY disabled.

> For testing details, please see the below:
>
> 1) Testing environment:
>     OS            : CentOS Stream 8 (non-RT OS)
>     Kernel     : v6.2
>     Machine : Intel Cascade Lake server (2 sockets, each with 44 logical =
threads)
>     Qemu  args  : -cpu host -enable-kvm, -smp 88,threads=3D2,sockets=3D2,=
 =E2=80=A6
>
> 2) My OS boot time definition:
>     The time from the start of the kernel boot to the shell command line
>     prompt is shown from the console. [ Different people may have
>     different OS boot time definitions. ]
>
> 3) My measurement method (very rough method):
>     A timer in the kernel periodically prints the boot time every 100ms.
>     As soon as the shell command line prompt is shown from the console,
>     we record the boot time printed by the timer, then the printed boot
>     time is the OS boot time.

Hmm, Can you not just print the boot time from userspace using
clock_gettime() and CLOCK_BOOTTIME? But yeah either way, good data!

>     The console log (mixed userspace and kernel logs) looked like this:
>
>            [  OK  ] Started Permit User Sessions.
>                         Starting Terminate Plymouth Boot Screen...
>                         Starting Hold until boot process finishes up...
>            [  OK  ] Started Command Scheduler.
>            [    6.824466] input: ImExPS/2 Generic Explorer ...
>            [    6.884685] Boot ms 6863
>                 ...
>            [    7.170920] Spectre V2 : WARNING: Unprivileged eBPF ...
>            [    7.173140] Spectre V2 : WARNING: Unprivileged eBPF ...
>            [    7.196741] Boot ms 7175
>                 ...
>            [    8.236757] Boot ms 8215
>
>            CentOS Stream 8
>            Kernel 6.2.0-rcu+ on an x86_64
>
>            login: [    8.340751] Boot ms 8319
>            [    8.444756] Boot ms 8423
>                 ...
>
>      Then the log "login: [    8.340751] Boot ms 8319" roughly showed the=
 OS boot time was ~8.3s.
>
> 4) Measured OS boot time (in seconds)
>    a) Measured 10 times w/o Joel's patch:
>         8.7s, 8.4s, 8.6s, 8.2s, 9.0s, 8.7s, 8.8s, 9.3s, 8.8s, 8.3s
>         The average OS boot time was: ~8.7s
>
>    b) Measure 10 times w/ Joel's patch:
>         8.5s, 8.2s, 7.6s, 8.2s, 8.7s, 8.2s, 7.8s, 8.2s, 9.3s, 8.4s
>         The average OS boot time was: ~8.3s.
>
> The OS boot time was reduced by : 8.7 =E2=80=93 8.3 =3D 0.4 second
> The reduction percentage was       : 0.4/8.7 * 100% =3D 4.6%
>
> If the testing above makes sense to you, please feel free to
> add
>
>       Tested-by: Qiuxu Zhuo <qiuxu.zhuo@intel.com>

Yes, it makes sense. I will add these. Thanks!

 - Joel

>
> Thanks!
> -Qiuxu
>
> > I can change it to default 0 essentially NOOPing it, but I would rather=
 have a
> > saner default (10 seconds even), than having someone forget to tune thi=
s for
> > their system.
> >
> > Thanks,
> >
> >  - Joel
>