Return-Path: Received: (majordomo@vger.kernel.org) by vger.kernel.org via listexpand id S1751622AbbHRUSh (ORCPT ); Tue, 18 Aug 2015 16:18:37 -0400 Received: from www.linutronix.de ([62.245.132.108]:58117 "EHLO Galois.linutronix.de" rhost-flags-OK-OK-OK-OK) by vger.kernel.org with ESMTP id S1750871AbbHRUSg (ORCPT ); Tue, 18 Aug 2015 16:18:36 -0400 Date: Tue, 18 Aug 2015 22:18:09 +0200 (CEST) From: Thomas Gleixner To: John Stultz cc: lkml , Shaohua Li , Prarit Bhargava , Richard Cochran , Daniel Lezcano , Ingo Molnar , Clark Williams , Steven Rostedt Subject: Re: [PATCH 8/9] clocksource: Improve unstable clocksource detection In-Reply-To: Message-ID: References: <1439844063-7957-1-git-send-email-john.stultz@linaro.org> <1439844063-7957-9-git-send-email-john.stultz@linaro.org> User-Agent: Alpine 2.11 (DEB 23 2013-08-11) MIME-Version: 1.0 Content-Type: TEXT/PLAIN; charset=US-ASCII X-Linutronix-Spam-Score: -1.0 X-Linutronix-Spam-Level: - X-Linutronix-Spam-Status: No , -1.0 points, 5.0 required, ALL_TRUSTED=-1,SHORTCIRCUIT=-0.0001,URIBL_BLOCKED=0.001 Sender: linux-kernel-owner@vger.kernel.org List-ID: X-Mailing-List: linux-kernel@vger.kernel.org Content-Length: 2904 Lines: 62 On Tue, 18 Aug 2015, John Stultz wrote: > On Tue, Aug 18, 2015 at 12:28 PM, Thomas Gleixner wrote: > > On Tue, 18 Aug 2015, John Stultz wrote: > >> On Tue, Aug 18, 2015 at 1:38 AM, Thomas Gleixner wrote: > >> > On Mon, 17 Aug 2015, John Stultz wrote: > >> >> On Mon, Aug 17, 2015 at 3:04 PM, Thomas Gleixner wrote: > >> >> > On Mon, 17 Aug 2015, John Stultz wrote: > >> >> > > >> >> >> From: Shaohua Li > >> >> >> > >> >> >> >From time to time we saw TSC is marked as unstable in our systems, while > >> >> > > >> >> > Stray '>' > >> >> > > >> >> >> the CPUs declare to have stable TSC. Looking at the clocksource unstable > >> >> >> detection, there are two problems: > >> >> >> - watchdog clock source wrap. HPET is the most common watchdog clock > >> >> >> source. It's 32-bit and runs in 14.3Mhz. That means the hpet counter > >> >> >> can wrap in about 5 minutes. > >> >> >> - threshold isn't scaled against interval. The threshold is 0.0625s in > >> >> >> 0.5s interval. What if the actual interval is bigger than 0.5s? > >> >> >> > >> >> >> The watchdog runs in a timer bh, so hard/soft irq can defer its running. > >> >> >> Heavy network stack softirq can hog a cpu. IPMI driver can disable > >> >> >> interrupt for a very long time. > >> >> > > >> >> > And they hold off the timer softirq for more than a second? Don't you > >> >> > think that's the problem which needs to be fixed? > >> >> > >> >> Though this is an issue I've experienced (and tried unsuccessfully to > >> >> fix in a more complicated way) with the RT kernel, where high priority > >> >> tasks blocked the watchdog long enough that we'd disqualify the TSC. > >> > > >> > Did it disqualify the watchdog due to HPET wraparounds (5 minutes) or > >> > due to the fixed threshold being applied? > >> > >> This was years ago, but in my experience, the watchdog false positives > >> were due to HPET wraparounds. > > > > Blocking stuff for 5 minutes is insane .... > > Yea. It was usually due to -RT stress testing, which keept the > machines busy for quite awhile. But again, if you have machines being > maxed out with networking load, etc, even for long amounts of time, we > still want to avoid false positives. Because after the watchdog The networking softirq does not hog the other softirqs. It has a limit on processing loops and then goes back to let the other softirqs be handled. So no, I doubt that heavy networking can cause this. If it does then we have some other way more serious problems. I can see the issue with RT stress testing, but not with networking in mainline. Thanks, tglx -- To unsubscribe from this list: send the line "unsubscribe linux-kernel" in the body of a message to majordomo@vger.kernel.org More majordomo info at http://vger.kernel.org/majordomo-info.html Please read the FAQ at http://www.tux.org/lkml/