Received-SPF: pass (google.com: domain of linux-kernel-owner@vger.kernel.org designates 23.128.96.18 as permitted sender) client-ip=23.128.96.18;
To:     Thomas Gleixner <tglx@linutronix.de>,
        Ingo Molnar <mingo@redhat.com>, Borislav Petkov <bp@alien8.de>
Cc:     x86@kernel.org, LKML <linux-kernel@vger.kernel.org>,
        Tom Lendacky <thomas.lendacky@amd.com>
From:   Paul Menzel <pmenzel@molgen.mpg.de>
Subject: `calibrate_APIC_clock()` takes 100 ms on AMD systems
Message-ID: <d6d50a65-59bc-aaa0-3f15-01d71496ef89@molgen.mpg.de>
Date:   Fri, 24 Apr 2020 16:59:21 +0200
User-Agent: Mozilla/5.0 (X11; Linux x86_64; rv:68.0) Gecko/20100101
 Thunderbird/68.7.0
MIME-Version: 1.0
Content-Type: text/plain; charset=utf-8; format=flowed
Content-Language: en-US
Content-Transfer-Encoding: 7bit
Sender: linux-kernel-owner@vger.kernel.org
Precedence: bulk

Dear Linux folks,


Trying to decrease the boot time, I noticed on AMD systems 
`calibrate_APIC_clock()` takes around 100 ms.

     [    0.004764] Freeing SMP alternatives memory: 32K
     [    0.107676] smpboot: CPU0: AMD Ryzen 3 2200G with Radeon Vega 
Graphics (family: 0x17, model: 0x11, stepping: 0x0)

On a different system with `apic=verbose`:

     [    0.209161] Freeing SMP alternatives memory: 28K
     [    0.212830] Using local APIC timer interrupts.
            calibrating APIC timer ...
     [    0.320928] ... lapic delta = 625044
     [    0.320928] ... PM-Timer delta = 357959
     [    0.320928] ... PM-Timer result ok
     [    0.320929] ..... delta 625044
     [    0.320930] ..... mult: 26844619
     [    0.320930] ..... calibration result: 400028
     [    0.320931] ..... CPU clock speed is 4400.1228 MHz.
     [    0.320931] ..... host bus clock speed is 100.0028 MHz.
     [    0.320936] smpboot: CPU0: AMD A6-6400K APU with Radeon(tm) HD 
Graphics (family: 0x15, model: 0x13, stepping: 0x1)

On Intel systems TSC deadline is used, and the function returns right away.

         if (boot_cpu_has(X86_FEATURE_TSC_DEADLINE_TIMER))
                 return 0;

     [    0.078373] Freeing SMP alternatives memory: 32K
     [    0.080373] TSC deadline timer enabled
     [    0.080375] smpboot: CPU0: Intel(R) Core(TM) i7-5600U CPU @ 
2.60GHz (family: 0x6, model: 0x3d, stepping: 0x4)

100 ms is almost 10 % percent of the total one second the Linux kernel 
takes here on the AMD systems, so I wonder if there are ways to reduce that.

     #define LAPIC_CAL_LOOPS         (HZ/10)

is 100 on my system, so the while loop below runs 100 times.

         while (lapic_cal_loops <= LAPIC_CAL_LOOPS) {
                 /* Wait for a tick to elapse */
                 while (1) {
                         if (tsc_khz) {
                                 u64 tsc_now = rdtsc();
                                 if ((tsc_now - tsc_start) >= tsc_perj) {
                                         tsc_start += tsc_perj;
                                         break;
                                 }
                         } else {
                                 unsigned long jif_now = READ_ONCE(jiffies);

                                 if (time_after(jif_now, jif_start)) {
                                         jif_start = jif_now;
                                         break;
                                 }
                         }
                         cpu_relax();
                 }

                 /* Invoke the calibration routine */
                 local_irq_disable();
                 lapic_cal_handler(NULL);
                 local_irq_enable();
         }

Can you give more information, why HZ/10 is chosen? Is there a way to 
break out earlier?

Do AMD devices feature an alternatives like for Intel? What other 
possibilities are there? (Please excuse my ignorance.) For example, 
taking the calibration values from the firmware, or could they be passed 
on the command line, if the user knows, the system has not changed?


Kind regards,

Paul