Received: by 2002:ac0:bc90:0:0:0:0:0 with SMTP id a16csp5613361img; Wed, 27 Mar 2019 11:43:23 -0700 (PDT) X-Google-Smtp-Source: APXvYqy1CE/5bsNZ0CfLb/NAkTJ3DybnZiK9GH8GRZaZmHIUv5vQOPuBufDYBeICn/3suPkydKhH X-Received: by 2002:a17:902:380c:: with SMTP id l12mr38747670plc.238.1553712202923; Wed, 27 Mar 2019 11:43:22 -0700 (PDT) ARC-Seal: i=1; a=rsa-sha256; t=1553712202; cv=none; d=google.com; s=arc-20160816; b=l79J5RMIriMzz5hqT5IAyBHyj3YC6PgWjDmCelg5syRbb12UgfzWBqEEaAhO0l2oe3 thy95gyEGkY1fNW9E1no3eIvBg6vKkCxl7B+eV+7QKHZPPPy3UFA/nJHaV4cAn5XVQm5 ekMKxRsQ96fnfoPZP7dPVzvvyJ9UWk2VbY2NM5XoAFDvCJwyUYujVQx6P9h0R9ePqQPm kjF8R/JgIIgvyRTu1WRgRlVoQ/wUmnFWRFjO3oM2kTbGW1lvB4Bv8gP8DINBSigAofcE nEQm0jwKi9/8pwi2RpiLuv6OW+CopEyIFgFQYnpRMafI/AEzCmhhbPZOVv7PrDZ5mI0q 5jew== ARC-Message-Signature: i=1; a=rsa-sha256; c=relaxed/relaxed; d=google.com; s=arc-20160816; h=list-id:precedence:sender:content-transfer-encoding:mime-version :references:in-reply-to:message-id:date:subject:cc:to:from :dkim-signature; bh=zr0B7FshyLZ14KX2wms2A5ozxegvuz1TjcXwdCcT2tA=; b=mkXzHrHKnqU81aDjCQYoZenHK4UG/AMsHbnc59mEmRhATOTdsrQoLz4twJ+mFIemnl qcpaxtq4CliBzNzV9MLoRsxtXyA5x9UsPRMEnW53bATiviCRId9BuJ2398fhLrpDZBJn MMqai4Vy4Yt2xnvNw4L9OPPNiowXfbV+QJisOnh4G5lraV1Vhg8vxgN3BsimvKwS6E2G KUEoiDeOKuFrR5n4cgeawBYKFM5aqzJPM+AHZo5nAFdFpkDsUiL3OrYyRZgSPo7QAMn4 O1kPy0KZ9q17VB9JhRv17e+7B58blAwNxy6JFI2kTJP9Mb4iDeAwu7hCw/A3Fhcn/m2W ZsLw== ARC-Authentication-Results: i=1; mx.google.com; dkim=pass header.i=@kernel.org header.s=default header.b=QtxcAEnf; spf=pass (google.com: best guess record for domain of linux-kernel-owner@vger.kernel.org designates 209.132.180.67 as permitted sender) smtp.mailfrom=linux-kernel-owner@vger.kernel.org; dmarc=pass (p=NONE sp=NONE dis=NONE) header.from=kernel.org Return-Path: Received: from vger.kernel.org (vger.kernel.org. [209.132.180.67]) by mx.google.com with ESMTP id x6si2593074pga.310.2019.03.27.11.43.07; Wed, 27 Mar 2019 11:43:22 -0700 (PDT) Received-SPF: pass (google.com: best guess record for domain of linux-kernel-owner@vger.kernel.org designates 209.132.180.67 as permitted sender) client-ip=209.132.180.67; Authentication-Results: mx.google.com; dkim=pass header.i=@kernel.org header.s=default header.b=QtxcAEnf; spf=pass (google.com: best guess record for domain of linux-kernel-owner@vger.kernel.org designates 209.132.180.67 as permitted sender) smtp.mailfrom=linux-kernel-owner@vger.kernel.org; dmarc=pass (p=NONE sp=NONE dis=NONE) header.from=kernel.org Received: (majordomo@vger.kernel.org) by vger.kernel.org via listexpand id S2391776AbfC0Sll (ORCPT + 99 others); Wed, 27 Mar 2019 14:41:41 -0400 Received: from mail.kernel.org ([198.145.29.99]:40426 "EHLO mail.kernel.org" rhost-flags-OK-OK-OK-OK) by vger.kernel.org with ESMTP id S2403847AbfC0SWV (ORCPT ); Wed, 27 Mar 2019 14:22:21 -0400 Received: from sasha-vm.mshome.net (c-73-47-72-35.hsd1.nh.comcast.net [73.47.72.35]) (using TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 (128/128 bits)) (No client certificate requested) by mail.kernel.org (Postfix) with ESMTPSA id 07C1E2063F; Wed, 27 Mar 2019 18:22:18 +0000 (UTC) DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/simple; d=kernel.org; s=default; t=1553710940; bh=OCVPpK2ztKpcXNE3xhRbxdrR3/nCBbHVBN7mXaAJFhs=; h=From:To:Cc:Subject:Date:In-Reply-To:References:From; b=QtxcAEnf0C2Q+k0TGRONu4GJcpEC3xM5JkMQGy6mFxNN7iX/Sf6djgyDUh9x7hRlV KKdWXTHSMPA7puH9tT7K72XsdGstJFCzet0rIC9nTF5AkQp+DO/aM9wxhJr0KOkBwR SAD/yBpAwu5/l2UiyUe4Dn0PonFp7/Q9WYGj02G0= From: Sasha Levin To: linux-kernel@vger.kernel.org, stable@vger.kernel.org Cc: Thomas Gleixner , Matthew Wilcox , Andrew Morton , Alexey Dobriyan , Kees Cook , linux-fsdevel@vger.kernel.org, Davidlohr Bueso , Miklos Szeredi , Daniel Colascione , Dave Chinner , Randy Dunlap , Sasha Levin Subject: [PATCH AUTOSEL 4.9 55/87] genirq: Avoid summation loops for /proc/stat Date: Wed, 27 Mar 2019 14:20:08 -0400 Message-Id: <20190327182040.17444-55-sashal@kernel.org> X-Mailer: git-send-email 2.19.1 In-Reply-To: <20190327182040.17444-1-sashal@kernel.org> References: <20190327182040.17444-1-sashal@kernel.org> MIME-Version: 1.0 X-Patchwork-Hint: Ignore Content-Transfer-Encoding: 8bit Sender: linux-kernel-owner@vger.kernel.org Precedence: bulk List-ID: X-Mailing-List: linux-kernel@vger.kernel.org From: Thomas Gleixner [ Upstream commit 1136b0728969901a091f0471968b2b76ed14d9ad ] Waiman reported that on large systems with a large amount of interrupts the readout of /proc/stat takes a long time to sum up the interrupt statistics. In principle this is not a problem. but for unknown reasons some enterprise quality software reads /proc/stat with a high frequency. The reason for this is that interrupt statistics are accounted per cpu. So the /proc/stat logic has to sum up the interrupt stats for each interrupt. This can be largely avoided for interrupts which are not marked as 'PER_CPU' interrupts by simply adding a per interrupt summation counter which is incremented along with the per interrupt per cpu counter. The PER_CPU interrupts need to avoid that and use only per cpu accounting because they share the interrupt number and the interrupt descriptor and concurrent updates would conflict or require unwanted synchronization. Reported-by: Waiman Long Signed-off-by: Thomas Gleixner Reviewed-by: Waiman Long Reviewed-by: Marc Zyngier Reviewed-by: Davidlohr Bueso Cc: Matthew Wilcox Cc: Andrew Morton Cc: Alexey Dobriyan Cc: Kees Cook Cc: linux-fsdevel@vger.kernel.org Cc: Davidlohr Bueso Cc: Miklos Szeredi Cc: Daniel Colascione Cc: Dave Chinner Cc: Randy Dunlap Link: https://lkml.kernel.org/r/20190208135020.925487496@linutronix.de 8<------------- v2: Undo the unintentional layout change of struct irq_desc. include/linux/irqdesc.h | 1 + kernel/irq/chip.c | 12 ++++++++++-- kernel/irq/internals.h | 8 +++++++- kernel/irq/irqdesc.c | 7 ++++++- 4 files changed, 24 insertions(+), 4 deletions(-) Signed-off-by: Sasha Levin --- include/linux/irqdesc.h | 1 + kernel/irq/chip.c | 12 ++++++++++-- kernel/irq/internals.h | 8 +++++++- kernel/irq/irqdesc.c | 7 ++++++- 4 files changed, 24 insertions(+), 4 deletions(-) diff --git a/include/linux/irqdesc.h b/include/linux/irqdesc.h index c9be57931b58..bb5547a83daf 100644 --- a/include/linux/irqdesc.h +++ b/include/linux/irqdesc.h @@ -61,6 +61,7 @@ struct irq_desc { unsigned int core_internal_state__do_not_mess_with_it; unsigned int depth; /* nested irq disables */ unsigned int wake_depth; /* nested wake enables */ + unsigned int tot_count; unsigned int irq_count; /* For detecting broken IRQs */ unsigned long last_unhandled; /* Aging timer for unhandled count */ unsigned int irqs_unhandled; diff --git a/kernel/irq/chip.c b/kernel/irq/chip.c index f30110e1b8c9..9e745cc0726d 100644 --- a/kernel/irq/chip.c +++ b/kernel/irq/chip.c @@ -729,7 +729,11 @@ void handle_percpu_irq(struct irq_desc *desc) { struct irq_chip *chip = irq_desc_get_chip(desc); - kstat_incr_irqs_this_cpu(desc); + /* + * PER CPU interrupts are not serialized. Do not touch + * desc->tot_count. + */ + __kstat_incr_irqs_this_cpu(desc); if (chip->irq_ack) chip->irq_ack(&desc->irq_data); @@ -758,7 +762,11 @@ void handle_percpu_devid_irq(struct irq_desc *desc) unsigned int irq = irq_desc_get_irq(desc); irqreturn_t res; - kstat_incr_irqs_this_cpu(desc); + /* + * PER CPU interrupts are not serialized. Do not touch + * desc->tot_count. + */ + __kstat_incr_irqs_this_cpu(desc); if (chip->irq_ack) chip->irq_ack(&desc->irq_data); diff --git a/kernel/irq/internals.h b/kernel/irq/internals.h index bc226e783bd2..22e3f29a30d8 100644 --- a/kernel/irq/internals.h +++ b/kernel/irq/internals.h @@ -199,12 +199,18 @@ static inline bool irqd_has_set(struct irq_data *d, unsigned int mask) #undef __irqd_to_state -static inline void kstat_incr_irqs_this_cpu(struct irq_desc *desc) +static inline void __kstat_incr_irqs_this_cpu(struct irq_desc *desc) { __this_cpu_inc(*desc->kstat_irqs); __this_cpu_inc(kstat.irqs_sum); } +static inline void kstat_incr_irqs_this_cpu(struct irq_desc *desc) +{ + __kstat_incr_irqs_this_cpu(desc); + desc->tot_count++; +} + static inline int irq_desc_get_node(struct irq_desc *desc) { return irq_common_data_get_node(&desc->irq_common_data); diff --git a/kernel/irq/irqdesc.c b/kernel/irq/irqdesc.c index 77977f55dff7..5e0ea17d01a6 100644 --- a/kernel/irq/irqdesc.c +++ b/kernel/irq/irqdesc.c @@ -109,6 +109,7 @@ static void desc_set_defaults(unsigned int irq, struct irq_desc *desc, int node, desc->depth = 1; desc->irq_count = 0; desc->irqs_unhandled = 0; + desc->tot_count = 0; desc->name = NULL; desc->owner = owner; for_each_possible_cpu(cpu) @@ -880,11 +881,15 @@ unsigned int kstat_irqs_cpu(unsigned int irq, int cpu) unsigned int kstat_irqs(unsigned int irq) { struct irq_desc *desc = irq_to_desc(irq); - int cpu; unsigned int sum = 0; + int cpu; if (!desc || !desc->kstat_irqs) return 0; + if (!irq_settings_is_per_cpu_devid(desc) && + !irq_settings_is_per_cpu(desc)) + return desc->tot_count; + for_each_possible_cpu(cpu) sum += *per_cpu_ptr(desc->kstat_irqs, cpu); return sum; -- 2.19.1