Received: by 2002:ac0:a582:0:0:0:0:0 with SMTP id m2-v6csp1511588imm; Wed, 10 Oct 2018 16:15:27 -0700 (PDT) X-Google-Smtp-Source: ACcGV610FZ5GSicOuxQx4G7eX+YAbyrfeogGi+YTeoNYD6ylUU/tEX3fd79LM1kwpChawTsz3gpz X-Received: by 2002:a63:1148:: with SMTP id 8-v6mr31945860pgr.172.1539213327874; Wed, 10 Oct 2018 16:15:27 -0700 (PDT) ARC-Seal: i=1; a=rsa-sha256; t=1539213327; cv=none; d=google.com; s=arc-20160816; b=OqXPH5RASXcJKYQBpXUfY8Qb+nI39nR1AMs3LtJTkzSuefRrhbIAriMtOUjVBfaJ2E Vgfx8G4q++LUzaTqbr6lY4vP1yc/HVVSPPD3/eoOnTzeaCGd8g2brSjt0kuDC07JkkxZ LtIDZAEVmgV/MK0A/Dx3Kw1drQbK+fKPiYhaPTH/PC/e3soMgZOgrY45yd0UdTXr9bxY 7HL2n0ujx8l/mcsf/7J+rFmBAh/cbssFFCnloIn65pdducxc5YKZakrZB67yOwLWkdX6 78ucRYwY7okQ8BEOT1JqLYiyGRgOH0omXQhUPwPDgiMwkGAFTGRC26HTXd/OAuTdYzKB 160g== ARC-Message-Signature: i=1; a=rsa-sha256; c=relaxed/relaxed; d=google.com; s=arc-20160816; h=list-id:precedence:sender:references:in-reply-to:message-id:date :subject:cc:to:from:dkim-signature; bh=ESwp6yB4D63F0qwgnvwh16rNIltM+zYx+tVT+6lKuN8=; b=jIplfxtMklTxlPs+bg8YnNK0dG3HlJ6Z7P4iFAvP+nYbqCukms4N1rLwz4KpTEoyGN AZxTrFwhnLtmz6wMqJTD4y3zmkXOjA5WknoyRoNmNXH5p1pLLvPWywkl3TdqatQR0OR7 L3eiklzIfGPbiWKTuROkWmtfadtIf5xchTDyXEA6oo5ueIdYv80sapY5kff07jSGmHRl Wu4WiTzMTBHGTU7m8k00pq4WXHRdAvw8EcznVA33riP2qaA4B/LyQVWW2Uu+r+ql1B8l s3wfuQ1FxkLe5q5MMfyeVy0eCux68+Mo9Utq8lsFW9ikz23mD1V7np4v3DxYG/vPB70V DGoQ== ARC-Authentication-Results: i=1; mx.google.com; dkim=pass header.i=@kernel.org header.s=default header.b=IOAVb9W5; spf=pass (google.com: best guess record for domain of linux-kernel-owner@vger.kernel.org designates 209.132.180.67 as permitted sender) smtp.mailfrom=linux-kernel-owner@vger.kernel.org; dmarc=pass (p=NONE sp=NONE dis=NONE) header.from=kernel.org Return-Path: Received: from vger.kernel.org (vger.kernel.org. [209.132.180.67]) by mx.google.com with ESMTP id c6-v6si24934083plr.91.2018.10.10.16.15.13; Wed, 10 Oct 2018 16:15:27 -0700 (PDT) Received-SPF: pass (google.com: best guess record for domain of linux-kernel-owner@vger.kernel.org designates 209.132.180.67 as permitted sender) client-ip=209.132.180.67; Authentication-Results: mx.google.com; dkim=pass header.i=@kernel.org header.s=default header.b=IOAVb9W5; spf=pass (google.com: best guess record for domain of linux-kernel-owner@vger.kernel.org designates 209.132.180.67 as permitted sender) smtp.mailfrom=linux-kernel-owner@vger.kernel.org; dmarc=pass (p=NONE sp=NONE dis=NONE) header.from=kernel.org Received: (majordomo@vger.kernel.org) by vger.kernel.org via listexpand id S1727819AbeJKGiO (ORCPT + 99 others); Thu, 11 Oct 2018 02:38:14 -0400 Received: from mail.kernel.org ([198.145.29.99]:36478 "EHLO mail.kernel.org" rhost-flags-OK-OK-OK-OK) by vger.kernel.org with ESMTP id S1727691AbeJKGiO (ORCPT ); Thu, 11 Oct 2018 02:38:14 -0400 Received: from lerouge.suse.de (LFbn-NCY-1-241-207.w83-194.abo.wanadoo.fr [83.194.85.207]) (using TLSv1.2 with cipher ECDHE-RSA-AES128-SHA256 (128/128 bits)) (No client certificate requested) by mail.kernel.org (Postfix) with ESMTPSA id 0B9ED2150C; Wed, 10 Oct 2018 23:13:49 +0000 (UTC) DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/simple; d=kernel.org; s=default; t=1539213232; bh=mREl0F64l1DcezjL7CWP/q16Hm49wIRPeAkvn0EubGc=; h=From:To:Cc:Subject:Date:In-Reply-To:References:From; b=IOAVb9W51zmMM6HgXwkgY5xO3a2TIJMhL8D19ruWleRJ8jd4U7apn92Tldmc1LWnS U4Tq8aVj75vk+6PIBAT8lB+NoFnCNe28e5tyKi3H3sZKIQ4fpDcCK7xSCRSAA4qymH 6Dw1DgEoM/zCTEfuV8vlrjdtkN1fFOWB10fjHHn8= From: Frederic Weisbecker To: LKML Cc: Frederic Weisbecker , Sebastian Andrzej Siewior , Peter Zijlstra , "David S . Miller" , Linus Torvalds , Thomas Gleixner , "Paul E . McKenney" , Ingo Molnar , Frederic Weisbecker , Mauro Carvalho Chehab Subject: [RFC PATCH 26/30] softirq: Increment the softirq offset on top of enabled bits Date: Thu, 11 Oct 2018 01:12:13 +0200 Message-Id: <1539213137-13953-27-git-send-email-frederic@kernel.org> X-Mailer: git-send-email 2.7.4 In-Reply-To: <1539213137-13953-1-git-send-email-frederic@kernel.org> References: <1539213137-13953-1-git-send-email-frederic@kernel.org> Sender: linux-kernel-owner@vger.kernel.org Precedence: bulk List-ID: X-Mailing-List: linux-kernel@vger.kernel.org Now we can rely on the vector enabled bits to know if some vector is disabled. Hence we can also now drive the softirq offset on top of it. As a result, the softirq offset don't need to nest anymore as the vector enable mask does it on the stack on its behalf: // Start with local_bh_disabled() == SOFTIRQ_ALL_MASK ... bh = local_bh_disable(BIT(NET_RX_SOFTIRQ)) { bh = local_bh_disabled(); local_bh_disabled() &= ~BIT(NET_RX_SOFTIRQ); // First vector disabled, inc preempt count preempt_count += SOFTIRQ_DISABLE_OFFSET; return bh; } .... bh2 = local_bh_disable(BIT(BLOCK_SOFTIRQ)) { bh2 = local_bh_disabled(); local_bh_disabled() &= ~BIT(NET_RX_SOFTIRQ); // No need to inc preempt count return bh2; } ... local_bh_enable(bh2) { local_bh_disabled() = bh2; // No need to dec preempt count } ... local_bh_enable(bh1) { local_bh_disabled() = bh; preempt_count -= SOFTIRQ_DISABLE_OFFSET; } Signed-off-by: Frederic Weisbecker Cc: Ingo Molnar Cc: Sebastian Andrzej Siewior Cc: Thomas Gleixner Cc: Peter Zijlstra Cc: Linus Torvalds Cc: David S. Miller Cc: Mauro Carvalho Chehab Cc: Paul E. McKenney --- kernel/softirq.c | 25 ++++++++++++++++--------- 1 file changed, 16 insertions(+), 9 deletions(-) diff --git a/kernel/softirq.c b/kernel/softirq.c index e2435b0..84da16c 100644 --- a/kernel/softirq.c +++ b/kernel/softirq.c @@ -117,6 +117,10 @@ unsigned int __local_bh_disable_ip(unsigned long ip, unsigned int cnt, raw_local_irq_save(flags); + enabled = local_softirq_enabled(); + if (enabled != SOFTIRQ_ALL_MASK) + cnt &= ~SOFTIRQ_MASK; + /* * The preempt tracer hooks into preempt_count_add and will break * lockdep because it calls back into lockdep after SOFTIRQ_OFFSET @@ -131,7 +135,6 @@ unsigned int __local_bh_disable_ip(unsigned long ip, unsigned int cnt, if (softirq_count() == (cnt & SOFTIRQ_MASK)) trace_softirqs_off(ip); - enabled = local_softirq_enabled(); softirq_enabled_nand(mask); raw_local_irq_restore(flags); @@ -157,6 +160,9 @@ void local_bh_enable_no_softirq(unsigned int bh) softirq_enabled_set(bh); + if (bh != SOFTIRQ_ALL_MASK) + return; + if (preempt_count() == SOFTIRQ_DISABLE_OFFSET) trace_preempt_on(CALLER_ADDR0, get_lock_parent_ip()); @@ -175,18 +181,18 @@ void __local_bh_enable_ip(unsigned long ip, unsigned int cnt, unsigned int bh) local_irq_disable(); #endif softirq_enabled_set(bh); - - /* - * Are softirqs going to be turned on now: - */ - if (softirq_count() == SOFTIRQ_DISABLE_OFFSET) + if (bh != SOFTIRQ_ALL_MASK) { + cnt &= ~SOFTIRQ_MASK; + } else if (!(softirq_count() & SOFTIRQ_OFFSET)) { + /* Are softirqs going to be turned on now: */ trace_softirqs_on(ip); + } /* * Keep preemption disabled until we are done with * softirq processing: */ - preempt_count_sub(cnt - 1); - + if (cnt) + preempt_count_sub(cnt - 1); if (unlikely(!in_interrupt() && local_softirq_pending())) { /* @@ -196,7 +202,8 @@ void __local_bh_enable_ip(unsigned long ip, unsigned int cnt, unsigned int bh) do_softirq(); } - preempt_count_dec(); + if (cnt) + preempt_count_dec(); #ifdef CONFIG_TRACE_IRQFLAGS local_irq_enable(); #endif -- 2.7.4