Received: by 2002:ac0:a582:0:0:0:0:0 with SMTP id m2-v6csp1512293imm; Wed, 10 Oct 2018 16:16:20 -0700 (PDT) X-Google-Smtp-Source: ACcGV62uWw9Lq6PRER7ySRQHX3y/fy5MxPdpThj9gqWhDEIGHZu0oDkMXdV5I8zeZj4f+8qXb26U X-Received: by 2002:a17:902:4503:: with SMTP id m3-v6mr33896027pld.168.1539213380344; Wed, 10 Oct 2018 16:16:20 -0700 (PDT) ARC-Seal: i=1; a=rsa-sha256; t=1539213380; cv=none; d=google.com; s=arc-20160816; b=NuWUl4ap/2hDU9QRHhx5A7mG8dECWlMncjkplmOPenaRWb0EG2iYJ9bOzdzEzMtUsc P+iSRIuwLUN0fha/AUVra+p/VxSckaQLxQ/0rAAL8FRRlc9i3UHSSMe7rFbs7gjr4zOI 3+GSC24b7h43vutgt9WCDFWNESZ8nct2P8mHG3O2Rsc6kWy0QikyHNvudKJRglqsGSUe EIVGfX5ycRJ3Y3drQukgZTFBsgepD6MYUl+uJXTSln4N1/bFi7dLjXZ1j5Gi2wLdALM9 5zrtG6jpehDrFcnCfgw6BQFkLcFjsJ8DieUu+CnMNbt7Z0lh3b56icVrpzLyxCsikcaA fF8A== ARC-Message-Signature: i=1; a=rsa-sha256; c=relaxed/relaxed; d=google.com; s=arc-20160816; h=list-id:precedence:sender:references:in-reply-to:message-id:date :subject:cc:to:from:dkim-signature; bh=It9R06wyX29ctPPXezNc2oZCdHnndfrL98C2F6pAWF8=; b=xOoZCXOD7V7NMCjv8n0tb+6njWPABD2i8oCiER7XpqdU+S7X6Ac05cwRL+ZvIjQmwh m1cdpcW66xc7qAhLchceArA+cHeWIUc/tsFZuJaiSF6z4Q9Dml9iRtcXFalg8YdfMCVR ePUGJ7bl2KaD0x5VfHD3U3FNyPTPiQMW+SkKKaduihY7n9xg34OduAGvE5HQDm4V0fk0 2G/wPWSe1JwLioHiuW/m1DRFiuQPJXorL4s4ZISSfCtyGyTvwT/Hs1Z+zawwDPjLSdsf DUH6NRB683m4ez0Jkb/ywkOP+V3ml4mYyVt1EDgQeuBKyC3F2r/y/QerEttsPMIkA00r w9Fw== ARC-Authentication-Results: i=1; mx.google.com; dkim=pass header.i=@kernel.org header.s=default header.b=aWh5I5RJ; spf=pass (google.com: best guess record for domain of linux-kernel-owner@vger.kernel.org designates 209.132.180.67 as permitted sender) smtp.mailfrom=linux-kernel-owner@vger.kernel.org; dmarc=pass (p=NONE sp=NONE dis=NONE) header.from=kernel.org Return-Path: Received: from vger.kernel.org (vger.kernel.org. [209.132.180.67]) by mx.google.com with ESMTP id r198-v6si26516678pgr.456.2018.10.10.16.16.06; Wed, 10 Oct 2018 16:16:20 -0700 (PDT) Received-SPF: pass (google.com: best guess record for domain of linux-kernel-owner@vger.kernel.org designates 209.132.180.67 as permitted sender) client-ip=209.132.180.67; Authentication-Results: mx.google.com; dkim=pass header.i=@kernel.org header.s=default header.b=aWh5I5RJ; spf=pass (google.com: best guess record for domain of linux-kernel-owner@vger.kernel.org designates 209.132.180.67 as permitted sender) smtp.mailfrom=linux-kernel-owner@vger.kernel.org; dmarc=pass (p=NONE sp=NONE dis=NONE) header.from=kernel.org Received: (majordomo@vger.kernel.org) by vger.kernel.org via listexpand id S1727955AbeJKGiY (ORCPT + 99 others); Thu, 11 Oct 2018 02:38:24 -0400 Received: from mail.kernel.org ([198.145.29.99]:36748 "EHLO mail.kernel.org" rhost-flags-OK-OK-OK-OK) by vger.kernel.org with ESMTP id S1726607AbeJKGiX (ORCPT ); Thu, 11 Oct 2018 02:38:23 -0400 Received: from lerouge.suse.de (LFbn-NCY-1-241-207.w83-194.abo.wanadoo.fr [83.194.85.207]) (using TLSv1.2 with cipher ECDHE-RSA-AES128-SHA256 (128/128 bits)) (No client certificate requested) by mail.kernel.org (Postfix) with ESMTPSA id E81B12075C; Wed, 10 Oct 2018 23:13:58 +0000 (UTC) DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/simple; d=kernel.org; s=default; t=1539213241; bh=I2AafpeYaUXl4kN5eeTd4hfwxbUvuIlODQlvFz//8yk=; h=From:To:Cc:Subject:Date:In-Reply-To:References:From; b=aWh5I5RJeK4SC49vQfWcW5uAvzRdeyxCKFviG4Zo2sJSYZnfMSb1AjOHEikByMhYO MsWEHxA4L853V7BWsptxEkfhsodo/iO64eZEHiM5B1bD1UlIGaovK5cJAQEinYbOmV lB3qRaPyfakGdVkt71rXMCrfF05v7RAD5CBIAALs= From: Frederic Weisbecker To: LKML Cc: Frederic Weisbecker , Sebastian Andrzej Siewior , Peter Zijlstra , "David S . Miller" , Linus Torvalds , Thomas Gleixner , Frederic Weisbecker , "Paul E . McKenney" , Ingo Molnar , Mauro Carvalho Chehab Subject: [RFC PATCH 29/30] softirq: Make softirq processing softinterruptible Date: Thu, 11 Oct 2018 01:12:16 +0200 Message-Id: <1539213137-13953-30-git-send-email-frederic@kernel.org> X-Mailer: git-send-email 2.7.4 In-Reply-To: <1539213137-13953-1-git-send-email-frederic@kernel.org> References: <1539213137-13953-1-git-send-email-frederic@kernel.org> Sender: linux-kernel-owner@vger.kernel.org Precedence: bulk List-ID: X-Mailing-List: linux-kernel@vger.kernel.org From: Frederic Weisbecker Make do_softirq() re-entrant and allow a vector, being either processed or disabled, to be interrupted by another vector. This way a vector won't be able to monopolize the CPU for a long while at the expense of the others that may rely on some predictable latency, especially on softirq disabled sections that used to disable all vectors. Signed-off-by: Frederic Weisbecker Cc: Ingo Molnar Cc: Sebastian Andrzej Siewior Cc: Thomas Gleixner Cc: Peter Zijlstra Cc: Linus Torvalds Cc: David S. Miller Cc: Mauro Carvalho Chehab Cc: Paul E. McKenney --- kernel/softirq.c | 16 ++++++++-------- 1 file changed, 8 insertions(+), 8 deletions(-) diff --git a/kernel/softirq.c b/kernel/softirq.c index 457bf60..f4cb1ea 100644 --- a/kernel/softirq.c +++ b/kernel/softirq.c @@ -195,7 +195,7 @@ void __local_bh_enable_ip(unsigned long ip, unsigned int cnt, unsigned int bh) if (cnt) preempt_count_sub(cnt - 1); - if (unlikely(!in_interrupt() && local_softirq_pending())) { + if (unlikely(!in_irq() && (local_softirq_pending() & local_softirq_enabled()))) { /* * Run softirq if any pending. And do it in its own stack * as we may be calling this deep in a task call stack already. @@ -387,7 +387,7 @@ asmlinkage __visible void __softirq_entry __do_softirq(void) lockdep_softirq_end(in_hardirq); account_irq_exit_time(current); local_bh_exit(); - WARN_ON_ONCE(in_interrupt()); + WARN_ON_ONCE(in_irq()); current_restore_flags(old_flags, PF_MEMALLOC); } @@ -396,12 +396,12 @@ asmlinkage __visible void do_softirq(void) __u32 pending; unsigned long flags; - if (in_interrupt()) + if (in_irq()) return; local_irq_save(flags); - pending = local_softirq_pending(); + pending = local_softirq_pending() & local_softirq_enabled(); if (pending && !ksoftirqd_running(pending)) do_softirq_own_stack(); @@ -432,7 +432,7 @@ void irq_enter(void) static inline void invoke_softirq(void) { - if (ksoftirqd_running(local_softirq_pending())) + if (ksoftirqd_running(local_softirq_pending() & local_softirq_enabled())) return; if (!force_irqthreads) { @@ -481,7 +481,7 @@ void irq_exit(void) #endif account_irq_exit_time(current); preempt_count_sub(HARDIRQ_OFFSET); - if (!in_interrupt() && local_softirq_pending()) + if (!in_irq() && (local_softirq_pending() & local_softirq_enabled())) invoke_softirq(); tick_irq_exit(); @@ -712,13 +712,13 @@ void __init softirq_init(void) static int ksoftirqd_should_run(unsigned int cpu) { - return local_softirq_pending(); + return local_softirq_pending() & local_softirq_enabled(); } static void run_ksoftirqd(unsigned int cpu) { local_irq_disable(); - if (local_softirq_pending()) { + if (local_softirq_pending() & local_softirq_enabled()) { /* * We can safely run softirq on inline stack, as we are not deep * in the task stack here. -- 2.7.4