Received: by 2002:a25:8b12:0:0:0:0:0 with SMTP id i18csp113313ybl; Mon, 12 Aug 2019 12:43:25 -0700 (PDT) X-Google-Smtp-Source: APXvYqyzk1zjAO5mbLgIetijb83d7l0RZceR+uR1Y2dD7jQZMQPoHMu/NcgSSDxlMbUPXun5M9uw X-Received: by 2002:a17:90a:3646:: with SMTP id s64mr869861pjb.44.1565639005406; Mon, 12 Aug 2019 12:43:25 -0700 (PDT) ARC-Seal: i=1; a=rsa-sha256; t=1565639005; cv=none; d=google.com; s=arc-20160816; b=VM/+LKqoxYltqVZ37WnCMNuQMHQrNymXOBhFk+chtqHHC3ORLIVOLFMwgKxu/sfQvV RraXbJtoIWaJbqvOA2T/axxKqvvjSG82wgO87Fe0DSdKaBEvtGlGWsFifUslVVQXDzGa 6BgxGXLgvJJjgXS3ijVa0tYa8IA8ERgwQ5JrdIJwGgMq3eVRH3Lv9x9MxCRsw0Vqhlww hFijaOQblYl8lJUzErbmiV94s92qfIZseickoczfYBfVOn1dkgAcTlA6lCqTkUC8flKL RNsd5MFAlUSmpC15g6jedd9Pj5kNAbC7bw2ONalSVv3Wb0IZKGdUXgcgzbHObcD1y8KT YY3A== ARC-Message-Signature: i=1; a=rsa-sha256; c=relaxed/relaxed; d=google.com; s=arc-20160816; h=list-id:precedence:sender:cc:to:subject:message-id:date:from :in-reply-to:references:mime-version:dkim-signature; bh=i9ag70mY/H+XzREddXT8m/GXsHt6XXH2r3/lGBLxgjQ=; b=BTadd7KiO9O3saEfnct8CZvC0YKu5YGJumL3Q+g5k6XAYDcBp4AUOsUyvA8GnSajrk EP3w8F/I3qkW/I6dYcjUFEH4aeTLpwPB3WRq9YQrFjUUIOm/aOwdnttGe1onu/VFhowd Y08c5CdTPCc6t0loNbYUHOxkDheBOf5j/2WEHzZNSTiUQWAfVKEZRHVbrRGDbOby+xBv jZbXyBlVAAdyGwqv0V+mjfjgAysZwpFwu9Fwm2wimGf35m31YP3mbwxFO1tZDNScourk Ufo2seN5yJDqALQYO9DR8Uog0ho0B/byKYdX0Kv0FqLLwXZtwYhPxu9CHq4HWPKqzgyO 8eGw== ARC-Authentication-Results: i=1; mx.google.com; dkim=pass header.i=@gmail.com header.s=20161025 header.b=ZRo1bNo+; spf=pass (google.com: best guess record for domain of linux-kernel-owner@vger.kernel.org designates 209.132.180.67 as permitted sender) smtp.mailfrom=linux-kernel-owner@vger.kernel.org; dmarc=pass (p=NONE sp=QUARANTINE dis=NONE) header.from=gmail.com Return-Path: Received: from vger.kernel.org (vger.kernel.org. [209.132.180.67]) by mx.google.com with ESMTP id r190si62286895pfr.102.2019.08.12.12.43.03; Mon, 12 Aug 2019 12:43:25 -0700 (PDT) Received-SPF: pass (google.com: best guess record for domain of linux-kernel-owner@vger.kernel.org designates 209.132.180.67 as permitted sender) client-ip=209.132.180.67; Authentication-Results: mx.google.com; dkim=pass header.i=@gmail.com header.s=20161025 header.b=ZRo1bNo+; spf=pass (google.com: best guess record for domain of linux-kernel-owner@vger.kernel.org designates 209.132.180.67 as permitted sender) smtp.mailfrom=linux-kernel-owner@vger.kernel.org; dmarc=pass (p=NONE sp=QUARANTINE dis=NONE) header.from=gmail.com Received: (majordomo@vger.kernel.org) by vger.kernel.org via listexpand id S1727060AbfHLTmP (ORCPT + 99 others); Mon, 12 Aug 2019 15:42:15 -0400 Received: from mail-lj1-f194.google.com ([209.85.208.194]:41803 "EHLO mail-lj1-f194.google.com" rhost-flags-OK-OK-OK-OK) by vger.kernel.org with ESMTP id S1726200AbfHLTmP (ORCPT ); Mon, 12 Aug 2019 15:42:15 -0400 Received: by mail-lj1-f194.google.com with SMTP id d24so99365612ljg.8 for ; Mon, 12 Aug 2019 12:42:14 -0700 (PDT) DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=gmail.com; s=20161025; h=mime-version:references:in-reply-to:from:date:message-id:subject:to :cc; bh=i9ag70mY/H+XzREddXT8m/GXsHt6XXH2r3/lGBLxgjQ=; b=ZRo1bNo+Aw7DiqwxJFn3dOhsQ/fbVTXH8r5yWiltl7XdssZ/k7B0Yz7F5FHLa/WdkS MeT+k+iNxhKOWiIKH9o5/D2tDHeeWPmSNDPkw5skE0iSvjSCpsUqe9K1out3ZZied9kc gNrTfUjruX87SkWqrLtPjiNb/QoE16r3csI4KMZERkYHTzqUJ7RydCIZBq09Qd2WRwqb gTagedY8bdHU7LMOADp8JxIY+PGeA1Qu6Pyh0hjfZcmbgeOzgu5bqQL2ddXmPBQYhXV0 KuXQQmq/eYfGsLYe0vO7pvr0hYBW5f9fQI0FE/TzXU5vJmNZpIEeypVaGuZQOiY54giO 7Xrg== X-Google-DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=1e100.net; s=20161025; h=x-gm-message-state:mime-version:references:in-reply-to:from:date :message-id:subject:to:cc; bh=i9ag70mY/H+XzREddXT8m/GXsHt6XXH2r3/lGBLxgjQ=; b=dZcImyTHkcz5hzt+HMQAHBPkHt8d28CCQiTiqi08g5euuakSAxZvvZ1uj6VCEFLmys dhhahoVxUeftJd+L0MBxwDfMEtMtYG/Q0s/Krmc4LRCZnInoTcq9h8VwdEUTPzxgXZ1x 7E7DStWTOV6fuN/9RtIqCvzPUUcjsu1AagTZ9x+Hcu2QIAraFEYrX9C8ZwUWr9Y6DaVB 3IJjKYpnlvjp8xfq9bLlDCYs2wWOVu/a5rApG01TBJtEHHgkDT40yzigfy0xv160mJe1 vcDyIct4Ex6fqFepRQdHRuupdQuJc42UB3VnlcAWVmTnAKXaNkzll44tws0nBehpINkV npKA== X-Gm-Message-State: APjAAAUQDFuq9EIEhF2M17j72lhLbc/yzfNNGgEIHA2rHSOvCwMij0JF sGBWNeAI8u3Jl2AuNoQrXO4GTModEVkSZAmiKME= X-Received: by 2002:a2e:8059:: with SMTP id p25mr7751366ljg.120.1565638933507; Mon, 12 Aug 2019 12:42:13 -0700 (PDT) MIME-Version: 1.0 References: <20180223121456.GZ25201@hirez.programming.kicks-ass.net> <20180226203937.GA21543@tassilo.jf.intel.com> In-Reply-To: From: Josh Hunt Date: Mon, 12 Aug 2019 12:42:02 -0700 Message-ID: Subject: Re: Long standing kernel warning: perfevents: irq loop stuck! To: Thomas Gleixner Cc: Andi Kleen , Peter Zijlstra , Cong Wang , "Liang, Kan" , jolsa@redhat.com, bigeasy@linutronix.de, "H. Peter Anvin" , Ingo Molnar , x86 , LKML Content-Type: text/plain; charset="UTF-8" Sender: linux-kernel-owner@vger.kernel.org Precedence: bulk List-ID: X-Mailing-List: linux-kernel@vger.kernel.org On Mon, Aug 12, 2019 at 12:34 PM Thomas Gleixner wrote: > > On Mon, 12 Aug 2019, Josh Hunt wrote: > > On Mon, Aug 12, 2019 at 10:55 AM Thomas Gleixner wrote: > > > > > > On Mon, 12 Aug 2019, Josh Hunt wrote: > > > > Was there any progress made on debugging this issue? We are still > > > > seeing it on 4.19.44: > > > > > > I haven't seen anyone looking at this. > > > > > > Can you please try the patch Ingo posted: > > > > > > https://lore.kernel.org/lkml/20150501070226.GB18957@gmail.com/ > > > > > > and if it fixes the issue decrease the value from 128 to the point where it > > > comes back, i.e. 128 -> 64 -> 32 ... > > > > > > Thanks, > > > > > > tglx > > > > I just checked the machines where this problem occurs and they're both > > Nehalem boxes. I think Ingo's patch would only help Haswell machines. > > Please let me know if I misread the patch or if what I'm seeing is a > > different issue than the one Cong originally reported. > > Find the NHM hack below. > > Thanks, > > tglx > > 8<---------------- > > diff --git a/arch/x86/events/intel/core.c b/arch/x86/events/intel/core.c > index 648260b5f367..93c1a4f0e73e 100644 > --- a/arch/x86/events/intel/core.c > +++ b/arch/x86/events/intel/core.c > @@ -3572,6 +3572,11 @@ static u64 bdw_limit_period(struct perf_event *event, u64 left) > return left; > } > > +static u64 nhm_limit_period(struct perf_event *event, u64 left) > +{ > + return max(left, 128ULL); > +} > + > PMU_FORMAT_ATTR(event, "config:0-7" ); > PMU_FORMAT_ATTR(umask, "config:8-15" ); > PMU_FORMAT_ATTR(edge, "config:18" ); > @@ -4606,6 +4611,7 @@ __init int intel_pmu_init(void) > x86_pmu.pebs_constraints = intel_nehalem_pebs_event_constraints; > x86_pmu.enable_all = intel_pmu_nhm_enable_all; > x86_pmu.extra_regs = intel_nehalem_extra_regs; > + x86_pmu.limit_period = nhm_limit_period; > > mem_attr = nhm_mem_events_attrs; > Thanks Thomas. Will try this and let you know. -- Josh