Return-Path: Received: (majordomo@vger.kernel.org) by vger.kernel.org via listexpand id S1752820AbcLIUuu (ORCPT ); Fri, 9 Dec 2016 15:50:50 -0500 Received: from vie01a-dmta-ch02-2.mx.upcmail.net ([84.116.36.95]:33596 "EHLO vie01a-dmta-ch02-2.mx.upcmail.net" rhost-flags-OK-OK-OK-OK) by vger.kernel.org with ESMTP id S1752200AbcLIUus (ORCPT ); Fri, 9 Dec 2016 15:50:48 -0500 X-SourceIP: 77.56.147.151 X-Authenticated-Sender: rscheidegger_lists@hispeed.ch Subject: Re: [PATCH] x86/tsc: RFC: re-synchronize TSCs to boot cpu TSC To: Thomas Gleixner References: <1481179042-968-1-git-send-email-sroland@vmware.com> <6194ea73-99dd-ceca-6f8d-e945ca717c75@hispeed.ch> <38f3ca77-ce32-d180-106a-e7242c635394@hispeed.ch> Cc: Ingo Molnar , "H. Peter Anvin" , x86@kernel.org, linux-kernel@vger.kernel.org From: Roland Scheidegger Message-ID: <50876e71-ade8-3c3b-db8b-8f5ae136157e@hispeed.ch> Date: Fri, 9 Dec 2016 21:50:43 +0100 User-Agent: Mozilla/5.0 (X11; Linux x86_64; rv:45.0) Gecko/20100101 Thunderbird/45.5.1 MIME-Version: 1.0 In-Reply-To: Content-Type: text/plain; charset=windows-1252 Content-Transfer-Encoding: 7bit Sender: linux-kernel-owner@vger.kernel.org List-ID: X-Mailing-List: linux-kernel@vger.kernel.org Content-Length: 22032 Lines: 516 Am 09.12.2016 um 18:33 schrieb Thomas Gleixner: > On Fri, 9 Dec 2016, Roland Scheidegger wrote: >> Am 09.12.2016 um 10:59 schrieb Thomas Gleixner: >>> On Fri, 9 Dec 2016, Roland Scheidegger wrote: >>>> >>>> I saw some system lockups though: >>>> When doing a cold boot, this kernel never managed to boot up. The last >>>> message seen is: >>>> x86: Booting SMP configuration: >>>> .... node #0, CPUs: #1 >>> >>> Weird. That really would be interesting to figure out what goes wrong >>> there. What bothers me is that we don't see something like this: >>> >>>> [ 0.172334] TSC ADJUST differs: Reference CPU0: -577421768610 CPU1: >>>> -577423766270 >>> >>> Can you please apply the debug patch below and provide the output ? >> Ok, this is the output (minus some typos maybe...): >> >> x86: Booting SMP configuration: >> .... node #0, CPUs: #1 >> TSC ADJUST: CPU1: -2806491604 >> TSC source sync 0 -> 1 runs 3 >> TSC ADJUST differs: Reference CPU0: -2805503200 CPU1: -2806491604 >> TSC ADJUST synchronize: Reference CPU0: -2805503200 CPU1: -2806491604 >> TSC target sync skip >> TSC source sync skipped >> >> And that's it. > > That's a cold boot (power on), right? > > That looks like I expected. As we know that the CPUs are in the same > package simply resynchronizing the TSC ADJUST MSR is enough. And the sync > test is skipped as we have resynced TSC ADJUST already to be the same > value. > > What's unexpected is that the thing stops working :( > > I'm pretty sure the boot cpu hangs in that endless loop waiting for CPU1 to > set the online bit, but for whatever reason this does not happen. > > Can you add the patch below to gather more information? There is a hunk in > there with an '#if 0' which sets the TSC ADJUST to 0 on boot, which you can > turn on as second step. Ok, here's the results: ... TSC ADJUST synchronize: Reference CPU0: -2820267100 CPU1: -2822498296 TSC target sync skipped smpboot: Vector locked smpboot: Vector setup done smpboot: Clock setup TSC source sync skipped smpboot: Target CPU is online With the #if 0 block activated, it boots up fine, the output was: [ 1.038892] x86: Booting SMP configuration: [ 1.038930] .... node #0, CPUs: #1 [ 0.171851] TSC ADJUST: CPU1: -2830353064 218577682002 [ 1.117495] TSC source sync 0 -> 1 runs 3 [ 0.171852] TSC ADJUST differs: Reference CPU0: -2828600940 CPU1: -2830353064 [ 0.171853] TSC ADJUST synchronize: Reference CPU0: 0 CPU1: -2830353064 [ 1.117497] TSC target sync skip [ 1.117497] smpboot: Vector locked [ 1.117506] smpboot: Vector setup done [ 1.117508] smpboot: Clock setup [ 1.117510] smpboot: Enter idle startup [ 1.117860] TSC source sync skipped [ 1.117896] smpboot: Target CPU is online [ 1.117990] #2 [ 0.252336] TSC ADJUST: CPU2: -2828948550 218818578394 [ 1.197511] TSC source sync 0 -> 2 runs 3 [ 0.252337] TSC ADJUST differs: Reference CPU0: -2828600940 CPU2: -2828948550 [ 0.252338] TSC ADJUST synchronize: Reference CPU0: 0 CPU2: -2828948550 [ 1.197513] TSC target sync skip [ 1.197513] smpboot: Vector locked [ 1.197523] smpboot: Vector setup done [ 1.197524] smpboot: Clock setup [ 1.197527] smpboot: Enter idle startup [ 1.197866] TSC source sync skipped [ 1.197902] smpboot: Target CPU is online [ 1.197997] #3 [ 0.332199] TSC ADJUST: CPU3: -2829409422 219057612986 [ 1.277528] TSC source sync 0 -> 3 runs 3 [ 0.332201] TSC ADJUST differs: Reference CPU0: -2828600940 CPU3: -2829409422 [ 0.332202] TSC ADJUST synchronize: Reference CPU0: 0 CPU3: -2829409422 [ 1.277530] TSC target sync skip [ 1.277530] smpboot: Vector locked [ 1.277540] smpboot: Vector setup done [ 1.277542] smpboot: Clock setup [ 1.277544] smpboot: Enter idle startup [ 1.277884] TSC source sync skipped [ 1.277920] smpboot: Target CPU is online [ 1.278016] #4 [ 0.411955] TSC ADJUST: CPU4: -2830194520 219296322250 [ 1.357546] TSC source sync 0 -> 4 runs 3 [ 0.411956] TSC ADJUST differs: Reference CPU0: -2828600940 CPU4: -2830194520 [ 0.411957] TSC ADJUST synchronize: Reference CPU0: 0 CPU4: -2830194520 [ 1.357548] TSC target sync skip [ 1.357548] smpboot: Vector locked [ 1.357558] smpboot: Vector setup done [ 1.357560] smpboot: Clock setup [ 1.357562] smpboot: Enter idle startup [ 1.357901] TSC source sync skipped [ 1.357937] smpboot: Target CPU is online [ 1.358030] #5 [ 0.491957] TSC ADJUST: CPU5: -2830235658 219535775620 [ 1.437562] TSC source sync 0 -> 5 runs 3 [ 0.491959] TSC ADJUST differs: Reference CPU0: -2828600940 CPU5: -2830235658 [ 0.491960] TSC ADJUST synchronize: Reference CPU0: 0 CPU5: -2830235658 [ 1.437564] TSC target sync skip [ 1.437565] smpboot: Vector locked [ 1.437574] smpboot: Vector setup done [ 1.437575] smpboot: Clock setup [ 1.437578] smpboot: Enter idle startup [ 1.437917] TSC source sync skipped [ 1.437953] smpboot: Target CPU is online [ 1.438047] #6 [ 0.571943] TSC ADJUST: CPU6: -2830332712 219775177676 [ 1.517581] TSC source sync 0 -> 6 runs 3 [ 0.571945] TSC ADJUST differs: Reference CPU0: -2828600940 CPU6: -2830332712 [ 0.571946] TSC ADJUST synchronize: Reference CPU0: 0 CPU6: -2830332712 [ 1.517583] TSC target sync skip [ 1.517583] smpboot: Vector locked [ 1.517592] smpboot: Vector setup done [ 1.517594] smpboot: Clock setup [ 1.517597] smpboot: Enter idle startup [ 1.517935] TSC source sync skipped [ 1.517971] smpboot: Target CPU is online [ 1.518066] #7 [ 0.652035] TSC ADJUST: CPU7: -2830107566 220014895498 [ 1.597597] TSC source sync 0 -> 7 runs 3 [ 0.652037] TSC ADJUST differs: Reference CPU0: -2828600940 CPU7: -2830107566 [ 0.652037] TSC ADJUST synchronize: Reference CPU0: 0 CPU7: -2830107566 [ 1.597599] TSC target sync skip [ 1.597600] smpboot: Vector locked [ 1.597609] smpboot: Vector setup done [ 1.597610] smpboot: Clock setup [ 1.597613] smpboot: Enter idle startup [ 1.597952] TSC source sync skipped [ 1.597988] smpboot: Target CPU is online [ 1.598083] #8 [ 0.732530] TSC ADJUST: CPU8: -2828717392 220255817924 [ 1.677628] TSC source sync 0 -> 8 runs 3 [ 0.732532] TSC ADJUST differs: Reference CPU0: -2828600940 CPU8: -2828717392 [ 0.732532] TSC ADJUST synchronize: Reference CPU0: 0 CPU8: -2828717392 [ 1.677630] TSC target sync skip [ 1.677630] smpboot: Vector locked [ 1.677643] smpboot: Vector setup done [ 1.677644] smpboot: Clock setup [ 1.677647] smpboot: Enter idle startup [ 1.677984] TSC source sync skipped [ 1.678020] smpboot: Target CPU is online [ 1.678123] #9 [ 0.812034] TSC ADJUST: CPU9: -2830240300 220493781182 [ 1.757641] TSC source sync 0 -> 9 runs 3 [ 0.812036] TSC ADJUST differs: Reference CPU0: -2828600940 CPU9: -2830240300 [ 0.812036] TSC ADJUST synchronize: Reference CPU0: 0 CPU9: -2830240300 [ 1.757642] TSC target sync skip [ 1.757643] smpboot: Vector locked [ 1.757652] smpboot: Vector setup done [ 1.757654] smpboot: Clock setup [ 1.757656] smpboot: Enter idle startup [ 1.757996] TSC source sync skipped [ 1.758032] smpboot: Target CPU is online [ 1.758130] #10 [ 0.892439] TSC ADJUST: CPU10: -2829079162 220734436186 [ 1.837658] TSC source sync 0 -> 10 runs 3 [ 0.892441] TSC ADJUST differs: Reference CPU0: -2828600940 CPU10: -2829079162 [ 0.892441] TSC ADJUST synchronize: Reference CPU0: 0 CPU10: -2829079162 [ 1.837659] TSC target sync skip [ 1.837660] smpboot: Vector locked [ 1.837669] smpboot: Vector setup done [ 1.837671] smpboot: Clock setup [ 1.837673] smpboot: Enter idle startup [ 1.838023] TSC source sync skipped [ 1.838059] smpboot: Target CPU is online [ 1.838155] #11 [ 0.972104] TSC ADJUST: CPU11: -2830133522 220972875474 [ 1.917674] TSC source sync 0 -> 11 runs 3 [ 0.972105] TSC ADJUST differs: Reference CPU0: -2828600940 CPU11: -2830133522 [ 0.972106] TSC ADJUST synchronize: Reference CPU0: 0 CPU11: -2830133522 [ 1.917676] TSC target sync skip [ 1.917677] smpboot: Vector locked [ 1.917685] smpboot: Vector setup done [ 1.917686] smpboot: Clock setup [ 1.917689] smpboot: Enter idle startup [ 1.918039] TSC source sync skipped [ 1.918075] smpboot: Target CPU is online [ 1.918171] #12 [ 1.052297] TSC ADJUST: CPU12: -2829606124 221212897186 [ 1.997691] TSC source sync 0 -> 12 runs 3 [ 1.052298] TSC ADJUST differs: Reference CPU0: -2828600940 CPU12: -2829606124 [ 1.052299] TSC ADJUST synchronize: Reference CPU0: 0 CPU12: -2829606124 [ 1.997693] TSC target sync skip [ 1.997693] smpboot: Vector locked [ 1.997702] smpboot: Vector setup done [ 1.997704] smpboot: Clock setup [ 1.997706] smpboot: Enter idle startup [ 1.998057] TSC source sync skipped [ 1.998093] smpboot: Target CPU is online [ 1.998189] #13 [ 1.132267] TSC ADJUST: CPU13: -2829742434 221452252884 [ 2.077708] TSC source sync 0 -> 13 runs 3 [ 1.132269] TSC ADJUST differs: Reference CPU0: -2828600940 CPU13: -2829742434 [ 1.132270] TSC ADJUST synchronize: Reference CPU0: 0 CPU13: -2829742434 [ 2.077709] TSC target sync skip [ 2.077710] smpboot: Vector locked [ 2.077719] smpboot: Vector setup done [ 2.077720] smpboot: Clock setup [ 2.077722] smpboot: Enter idle startup [ 2.078073] TSC source sync skipped [ 2.078109] smpboot: Target CPU is online [ 2.078203] #14 [ 1.212227] TSC ADJUST: CPU14: -2829914872 221691574212 [ 2.157724] TSC source sync 0 -> 14 runs 3 [ 1.212228] TSC ADJUST differs: Reference CPU0: -2828600940 CPU14: -2829914872 [ 1.212229] TSC ADJUST synchronize: Reference CPU0: 0 CPU14: -2829914872 [ 2.157726] TSC target sync skip [ 2.157726] smpboot: Vector locked [ 2.157735] smpboot: Vector setup done [ 2.157737] smpboot: Clock setup [ 2.157740] smpboot: Enter idle startup [ 2.159900] TSC source sync skipped [ 2.159936] smpboot: Target CPU is online [ 2.160032] #15 [ 1.292113] TSC ADJUST: CPU15: -2830310478 221930677086 [ 2.237743] TSC source sync 0 -> 15 runs 3 [ 1.292115] TSC ADJUST differs: Reference CPU0: -2828600940 CPU15: -2830310478 [ 1.292115] TSC ADJUST synchronize: Reference CPU0: 0 CPU15: -2830310478 [ 2.237745] TSC target sync skip [ 2.237745] smpboot: Vector locked [ 2.237755] smpboot: Vector setup done [ 2.237756] smpboot: Clock setup [ 2.237759] smpboot: Enter idle startup [ 2.238112] TSC source sync skipped [ 2.238148] smpboot: Target CPU is online [ 2.238186] x86: Booted up 1 node, 16 CPUs [ 2.238224] smpboot: Total of 16 processors activated (95793.84 BogoMIPS) (And fwiw with my quick hack the lockups disappear to when I change that back to blast a zero into TSC_ADJ for all cpus.) The system also came back up fine from suspend with this (well - still minus graphics...), however disabled tsc clocksource: [ 579.931739] Enabling non-boot CPUs ... [ 579.943106] x86: Booting SMP configuration: [ 579.943107] smpboot: Booting Node 0 Processor 1 APIC 0x2 [ 579.943189] TSC ADJUST: CPU1: -1504429974 21601834126 [ 579.943837] TSC source sync 0 -> 1 runs 3 [ 579.943191] TSC ADJUST differs: Reference CPU0: -2828600940 CPU1: -1504429974 [ 579.943191] TSC ADJUST synchronize: Reference CPU0: 0 CPU1: -1504429974 [ 580.445833] TSC target sync skip [ 580.445834] smpboot: Vector locked [ 579.943839] TSC source sync skipped [ 580.445845] smpboot: Vector setup done [ 579.943850] smpboot: Target CPU is online [ 580.445847] smpboot: Clock setup [ 580.445852] smpboot: Enter idle startup [ 580.445914] cache: parent cpu1 should not be sleeping [ 579.944093] CPU1 is up [ 580.458983] clocksource: timekeeping watchdog on CPU1: Marking clocksource 'tsc' as unstable because the skew is too large: [ 580.458985] clocksource: 'hpet' wd_now: 587c1 wd_last: 437c7 mask: ffffffff [ 580.458986] clocksource: 'tsc' cs_now: 563963cd8 cs_last: 508f5a02a mask: ffffffffffffffff [ 579.959249] smpboot: Booting Node 0 Processor 2 APIC 0x4 [ 579.959454] TSC ADJUST: CPU2: -1504299836 21650515856 [ 579.960058] TSC source sync 0 -> 2 runs 3 [ 579.959456] TSC ADJUST differs: Reference CPU0: -2828600940 CPU2: -1504299836 [ 579.959456] TSC ADJUST synchronize: Reference CPU0: 0 CPU2: -1504299836 [ 580.462055] TSC target sync skip [ 579.960060] TSC source sync skipped [ 580.462056] smpboot: Vector locked [ 580.462067] smpboot: Vector setup done [ 579.960072] smpboot: Target CPU is online [ 580.462068] smpboot: Clock setup [ 580.462071] smpboot: Enter idle startup [ 580.462136] cache: parent cpu2 should not be sleeping [ 579.960327] CPU2 is up [ 580.503230] smpboot: Booting Node 0 Processor 3 APIC 0x6 [ 580.503782] TSC ADJUST: CPU3: -1504357188 23279711288 [ 580.504405] TSC source sync 0 -> 3 runs 3 [ 580.503784] TSC ADJUST differs: Reference CPU0: -2828600940 CPU3: -1504357188 [ 580.503784] TSC ADJUST synchronize: Reference CPU0: 0 CPU3: -1504357188 [ 581.006402] TSC target sync skip [ 581.006403] smpboot: Vector locked [ 580.504408] TSC source sync skipped [ 581.006416] smpboot: Vector setup done [ 580.504421] smpboot: Target CPU is online [ 581.006418] smpboot: Clock setup [ 581.006424] smpboot: Enter idle startup [ 581.006506] cache: parent cpu3 should not be sleeping [ 580.504741] CPU3 is up [ 581.006760] [Firmware Bug]: TSC ADJUST differs: CPU0 0 --> -1502494750. Restoring [ 581.019271] smpboot: Booting Node 0 Processor 4 APIC 0x8 [ 580.517802] TSC ADJUST: CPU4: -1504419274 23321675136 [ 581.020442] TSC source sync 0 -> 4 runs 3 [ 580.517804] TSC ADJUST differs: Reference CPU0: -2828600940 CPU4: -1504419274 [ 580.517805] TSC ADJUST synchronize: Reference CPU0: 0 CPU4: -1504419274 [ 581.020444] TSC target sync skip [ 581.020444] TSC source sync skipped [ 581.020445] smpboot: Vector locked [ 581.020459] smpboot: Vector setup done [ 581.020459] smpboot: Target CPU is online [ 581.020461] smpboot: Clock setup [ 581.020465] smpboot: Enter idle startup [ 581.020548] cache: parent cpu4 should not be sleeping [ 581.020785] CPU4 is up [ 581.035383] smpboot: Booting Node 0 Processor 5 APIC 0xa [ 580.534095] TSC ADJUST: CPU5: -1503892568 23370438110 [ 581.036558] TSC source sync 0 -> 5 runs 3 [ 580.534096] TSC ADJUST differs: Reference CPU0: -2828600940 CPU5: -1503892568 [ 580.534097] TSC ADJUST synchronize: Reference CPU0: 0 CPU5: -1503892568 [ 581.036560] TSC target sync skip [ 581.036560] TSC source sync skipped [ 581.036561] smpboot: Vector locked [ 581.036574] smpboot: Vector setup done [ 581.036575] smpboot: Target CPU is online [ 581.036576] smpboot: Clock setup [ 581.036580] smpboot: Enter idle startup [ 581.036664] cache: parent cpu5 should not be sleeping [ 581.036913] CPU5 is up [ 581.051460] smpboot: Booting Node 0 Processor 6 APIC 0xc [ 580.550097] TSC ADJUST: CPU6: -1504107826 23418332356 [ 581.052632] TSC source sync 0 -> 6 runs 3 [ 580.550098] TSC ADJUST differs: Reference CPU0: -2828600940 CPU6: -1504107826 [ 580.550099] TSC ADJUST synchronize: Reference CPU0: 0 CPU6: -1504107826 [ 581.052634] TSC target sync skip [ 581.052634] smpboot: Vector locked [ 581.052634] TSC source sync skipped [ 581.052647] smpboot: Vector setup done [ 581.052648] smpboot: Target CPU is online [ 581.052650] smpboot: Clock setup [ 581.052654] smpboot: Enter idle startup [ 581.052738] cache: parent cpu6 should not be sleeping [ 581.053002] CPU6 is up [ 581.071544] smpboot: Booting Node 0 Processor 7 APIC 0xe [ 580.570040] TSC ADJUST: CPU7: -1504535440 23478024002 [ 581.072718] TSC source sync 1 -> 7 runs 3 [ 580.570041] TSC ADJUST differs: Reference CPU0: -2828600940 CPU7: -1504535440 [ 580.570042] TSC ADJUST synchronize: Reference CPU0: 0 CPU7: -1504535440 [ 581.072720] TSC target sync skip [ 581.072720] smpboot: Vector locked [ 581.072720] TSC source sync skipped [ 581.072733] smpboot: Vector setup done [ 581.072734] smpboot: Target CPU is online [ 581.072736] smpboot: Clock setup [ 581.072740] smpboot: Enter idle startup [ 581.072822] cache: parent cpu7 should not be sleeping [ 581.073092] CPU7 is up [ 581.095281] smpboot: Booting Node 0 Processor 8 APIC 0x1 [ 580.594433] TSC ADJUST: CPU8: -1502571034 23551035204 [ 581.096455] TSC source sync 1 -> 8 runs 3 [ 580.594435] TSC ADJUST differs: Reference CPU0: -2828600940 CPU8: -1502571034 [ 580.594436] TSC ADJUST synchronize: Reference CPU0: 0 CPU8: -1502571034 [ 581.096457] TSC target sync skip [ 581.096457] smpboot: Vector locked [ 581.096458] TSC source sync skipped [ 581.096470] smpboot: Vector setup done [ 581.096470] smpboot: Target CPU is online [ 581.096472] smpboot: Clock setup [ 581.096476] smpboot: Enter idle startup [ 581.096593] cache: parent cpu8 should not be sleeping [ 581.096881] CPU8 is up [ 581.111327] smpboot: Booting Node 0 Processor 9 APIC 0x3 [ 580.609809] TSC ADJUST: CPU9: -1504556458 23597051168 [ 581.112494] TSC source sync 1 -> 9 runs 3 [ 580.609810] TSC ADJUST differs: Reference CPU0: -2828600940 CPU9: -1504556458 [ 580.609811] TSC ADJUST synchronize: Reference CPU0: 0 CPU9: -1504556458 [ 581.112496] TSC target sync skip [ 581.112496] TSC source sync skipped [ 581.112496] smpboot: Vector locked [ 581.112515] smpboot: Vector setup done [ 581.112515] smpboot: Target CPU is online [ 581.112518] smpboot: Clock setup [ 581.112522] smpboot: Enter idle startup [ 581.112635] cache: parent cpu9 should not be sleeping [ 581.112919] CPU9 is up [ 581.127367] smpboot: Booting Node 0 Processor 10 APIC 0x5 [ 580.626298] TSC ADJUST: CPU10: -1503281384 23646406682 [ 581.128558] TSC source sync 1 -> 10 runs 3 [ 580.626300] TSC ADJUST differs: Reference CPU0: -2828600940 CPU10: -1503281384 [ 580.626301] TSC ADJUST synchronize: Reference CPU0: 0 CPU10: -1503281384 [ 581.128559] TSC target sync skip [ 581.128560] TSC source sync skipped [ 581.128560] smpboot: Vector locked [ 581.128573] smpboot: Vector setup done [ 581.128573] smpboot: Target CPU is online [ 581.128575] smpboot: Clock setup [ 581.128579] smpboot: Enter idle startup [ 581.128689] cache: parent cpu10 should not be sleeping [ 581.128997] CPU10 is up [ 581.143754] smpboot: Booting Node 0 Processor 11 APIC 0x7 [ 580.642580] TSC ADJUST: CPU11: -1503566952 23695138352 [ 581.144935] TSC source sync 1 -> 11 runs 3 [ 580.642582] TSC ADJUST differs: Reference CPU0: -2828600940 CPU11: -1503566952 [ 580.642583] TSC ADJUST synchronize: Reference CPU0: 0 CPU11: -1503566952 [ 581.144937] TSC target sync skip [ 581.144937] smpboot: Vector locked [ 581.144938] TSC source sync skipped [ 581.144950] smpboot: Vector setup done [ 581.144950] smpboot: Target CPU is online [ 581.144952] smpboot: Clock setup [ 581.144956] smpboot: Enter idle startup [ 581.145069] cache: parent cpu11 should not be sleeping [ 581.145493] CPU11 is up [ 581.163769] smpboot: Booting Node 0 Processor 12 APIC 0x9 [ 580.662529] TSC ADJUST: CPU12: -1503752538 23754846788 [ 581.164945] TSC source sync 1 -> 12 runs 3 [ 580.662531] TSC ADJUST differs: Reference CPU0: -2828600940 CPU12: -1503752538 [ 580.662531] TSC ADJUST synchronize: Reference CPU0: 0 CPU12: -1503752538 [ 581.164947] TSC target sync skip [ 581.164948] TSC source sync skipped [ 581.164949] smpboot: Vector locked [ 581.164961] smpboot: Vector setup done [ 581.164962] smpboot: Target CPU is online [ 581.164963] smpboot: Clock setup [ 581.164967] smpboot: Enter idle startup [ 581.165084] cache: parent cpu12 should not be sleeping [ 581.165564] CPU12 is up [ 581.191866] smpboot: Booting Node 0 Processor 13 APIC 0xb [ 580.690388] TSC ADJUST: CPU13: -1504470824 23838230778 [ 581.193045] TSC source sync 1 -> 13 runs 3 [ 580.690390] TSC ADJUST differs: Reference CPU0: -2828600940 CPU13: -1504470824 [ 580.690391] TSC ADJUST synchronize: Reference CPU0: 0 CPU13: -1504470824 [ 581.193047] TSC target sync skip [ 581.193047] smpboot: Vector locked [ 581.193048] TSC source sync skipped [ 581.193060] smpboot: Vector setup done [ 581.193061] smpboot: Target CPU is online [ 581.193063] smpboot: Clock setup [ 581.193066] smpboot: Enter idle startup [ 581.193184] cache: parent cpu13 should not be sleeping [ 581.193637] CPU13 is up [ 581.215919] smpboot: Booting Node 0 Processor 14 APIC 0xd [ 580.714423] TSC ADJUST: CPU14: -1504515582 23910167978 [ 581.217095] TSC source sync 1 -> 14 runs 3 [ 580.714425] TSC ADJUST differs: Reference CPU0: -2828600940 CPU14: -1504515582 [ 580.714426] TSC ADJUST synchronize: Reference CPU0: 0 CPU14: -1504515582 [ 581.217097] TSC target sync skip [ 581.217097] smpboot: Vector locked [ 581.217098] TSC source sync skipped [ 581.217110] smpboot: Vector setup done [ 581.217111] smpboot: Target CPU is online [ 581.217113] smpboot: Clock setup [ 581.217116] smpboot: Enter idle startup [ 581.217234] cache: parent cpu14 should not be sleeping [ 581.217706] CPU14 is up [ 581.251975] smpboot: Booting Node 0 Processor 15 APIC 0xf [ 580.750545] TSC ADJUST: CPU15: -1504326376 24018283718 [ 581.253154] TSC source sync 1 -> 15 runs 3 [ 580.750547] TSC ADJUST differs: Reference CPU0: -2828600940 CPU15: -1504326376 [ 580.750548] TSC ADJUST synchronize: Reference CPU0: 0 CPU15: -1504326376 [ 581.253156] TSC target sync skip [ 581.253156] TSC source sync skipped [ 581.253157] smpboot: Vector locked [ 581.253169] smpboot: Vector setup done [ 581.253169] smpboot: Target CPU is online [ 581.253171] smpboot: Clock setup [ 581.253175] smpboot: Enter idle startup [ 581.253294] cache: parent cpu15 should not be sleeping [ 581.253774] CPU15 is up [ 581.308954] ACPI: Waking up from system sleep state S3 [ 581.308984] clocksource: Switched to clocksource hpet > > This BIOS seems to be rather strange ... Don't tell me... No idea what it's doing, but I think it's safe to say whatever it's trying to do, it's doing it wrong... Roland