Received: by 2002:ad5:474a:0:0:0:0:0 with SMTP id i10csp1583730imu; Wed, 12 Dec 2018 00:11:48 -0800 (PST) X-Google-Smtp-Source: AFSGD/X6bpcDVx3KWLeTEyW1fayJnv0Da7FZ7qDFhCmxqPzSMvpQztdj7ID0+0RP3Y1J3MtI/D7+ X-Received: by 2002:a62:ce0e:: with SMTP id y14mr20165266pfg.100.1544602308426; Wed, 12 Dec 2018 00:11:48 -0800 (PST) ARC-Seal: i=1; a=rsa-sha256; t=1544602308; cv=none; d=google.com; s=arc-20160816; b=lyx9+50NW59fD/n2/OvK6s158VvXI8PoqlOAC9xXooAXygS8LOFALn8QAIFMEXDhgZ GMh8Mcy1uUULNX3lsP4yitLbWe5dRNmEHzD3FjLCJH/rfaqBj0Dkmqf3ZKE9RBa6RrYp 5DIec9GtRbG0Rw38H0O6Lrcn2CEOKTkZcWjLgMnpl/LQnqBXnr4n9/zWmRrjZNnYQhcQ 3H715ag2x3R7t/CYTGCWQ84e5HBLOMZWdOhGQ/xtQN6YKu8LQuBjAPc33iQCU8AjFsnC IfMsMMs9MIKAF/suwiUbiBY2Nq5jme17mG/ApAkVOiMj/UZ02exliQ/viRMIczFUTl93 yo6w== ARC-Message-Signature: i=1; a=rsa-sha256; c=relaxed/relaxed; d=google.com; s=arc-20160816; h=list-id:precedence:sender:user-agent:in-reply-to :content-disposition:mime-version:references:message-id:subject:cc :to:from:date:dkim-signature; bh=CJwlyndDJBGW56rrYVUjYBY47O8iVFOkSIn5bU8BPRA=; b=SEj1F6i2X/7qyaVc1R6cF1nyGWIS5YjAx9J2GILcG6L7PEvU0ToZkHIHItfUqbwpMR UxQuXTmDOqas2EuSzIHBQVjo6S97pQdE94R/2ZdiLt49AnMb074krx/wOi1k/tK/vNdk tXsNqtM5ijNADwku3dDy75NjA6mL5DQjvKOPGxhm3/ou59o+grpjqddYvLDxoCzTCxZl qNVR3qG0s64lfUfpb6TTWN/bhSmovsn0VOmO12zEhQ1oCLYa4F7eJOki8Bc64XjeiSaw SARkBGPlRW5jjpwP4RBFQxxIygDQQcu5Bepg0o0wGDGrs3a+OU32TuRR/mZM761wojoZ Ev+g== ARC-Authentication-Results: i=1; mx.google.com; dkim=pass header.i=@gmail.com header.s=20161025 header.b=PCI6tprS; spf=pass (google.com: best guess record for domain of linux-kernel-owner@vger.kernel.org designates 209.132.180.67 as permitted sender) smtp.mailfrom=linux-kernel-owner@vger.kernel.org; dmarc=pass (p=NONE sp=QUARANTINE dis=NONE) header.from=gmail.com Return-Path: Received: from vger.kernel.org (vger.kernel.org. [209.132.180.67]) by mx.google.com with ESMTP id b36si15147557pla.354.2018.12.12.00.11.32; Wed, 12 Dec 2018 00:11:48 -0800 (PST) Received-SPF: pass (google.com: best guess record for domain of linux-kernel-owner@vger.kernel.org designates 209.132.180.67 as permitted sender) client-ip=209.132.180.67; Authentication-Results: mx.google.com; dkim=pass header.i=@gmail.com header.s=20161025 header.b=PCI6tprS; spf=pass (google.com: best guess record for domain of linux-kernel-owner@vger.kernel.org designates 209.132.180.67 as permitted sender) smtp.mailfrom=linux-kernel-owner@vger.kernel.org; dmarc=pass (p=NONE sp=QUARANTINE dis=NONE) header.from=gmail.com Received: (majordomo@vger.kernel.org) by vger.kernel.org via listexpand id S1726635AbeLLIKk (ORCPT + 99 others); Wed, 12 Dec 2018 03:10:40 -0500 Received: from mail-pl1-f194.google.com ([209.85.214.194]:39996 "EHLO mail-pl1-f194.google.com" rhost-flags-OK-OK-OK-OK) by vger.kernel.org with ESMTP id S1726007AbeLLIKk (ORCPT ); Wed, 12 Dec 2018 03:10:40 -0500 Received: by mail-pl1-f194.google.com with SMTP id u18so8238975plq.7; Wed, 12 Dec 2018 00:10:39 -0800 (PST) DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=gmail.com; s=20161025; h=date:from:to:cc:subject:message-id:references:mime-version :content-disposition:in-reply-to:user-agent; bh=CJwlyndDJBGW56rrYVUjYBY47O8iVFOkSIn5bU8BPRA=; b=PCI6tprSi4UDEP0LPvawg1WwzJ17ehiV8nqvKt6Aee5vNTQqKYCeI0BIRbG7fXVk7o Z2MrjtjqRiSKTRPk9jBASKCiGqRqdEtsEvR7gj1sIj8pn5PIr+LZDcPLDq7MA30aFs2F hKhehJ6EAqSU0gLTRkd6M1zk2B4WMRfWMLlmulQjhSkbLtNxEIzngcDmhF6wyVcQ7Er9 u/g0xQ0NW8gIvfr/vWS2I9Z3HEluKegC9KlXSUml+z9RwDXFfCcP1LLtR6Yh6+zqnVvm Ma/TCnFQ4zORoi806ymz2a6l8+kPmr/I5CrlP9basTUinxQ6oPGN1MLHyvPnMOuuTfOV IuvA== X-Google-DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=1e100.net; s=20161025; h=x-gm-message-state:date:from:to:cc:subject:message-id:references :mime-version:content-disposition:in-reply-to:user-agent; bh=CJwlyndDJBGW56rrYVUjYBY47O8iVFOkSIn5bU8BPRA=; b=XaGNETGW/LYCUQZVcBEYBm5hN1bIBljflRTHV+xrkW5hjy1sNW4FoHmo77asa0IzPY SvsUvwb7SFnhRyCjfph2MNv2j+Wg7BTeXNKr4Fh7sNfOOAFf+c/sN0BvwXX+NwGyEezl KEubdVJot/DZ3uopE9AzCXJChmQtD1Hg0qSvfI5przO6Thztf7zZcP7QUCS8MCgN3rSs c672bRCwDpT1/D4VEL5R24mgFTGPKNC5Jokc0Q/4YPjQtk2+ZfqR7ArL/Ck/lHkr4Qdw uVHGIsYM79xk2DKWAxYPk6+filLdpsTLEoTve0G8ZkL1MG0Sc2L0xHLd0XXENneKx8+l OtyQ== X-Gm-Message-State: AA+aEWb15f1lKBP4zhkfpUPKFdc/cwHbsQMcVBQ+Iegn5dYTNqKWS6u1 nVsTS0wimUmKScr7aewIQW0= X-Received: by 2002:a17:902:298a:: with SMTP id h10mr19210726plb.312.1544602239310; Wed, 12 Dec 2018 00:10:39 -0800 (PST) Received: from localhost ([211.246.69.141]) by smtp.gmail.com with ESMTPSA id d18sm25578911pfj.47.2018.12.12.00.10.37 (version=TLS1_2 cipher=ECDHE-RSA-CHACHA20-POLY1305 bits=256/256); Wed, 12 Dec 2018 00:10:37 -0800 (PST) Date: Wed, 12 Dec 2018 17:10:34 +0900 From: Sergey Senozhatsky To: Sasha Levin Cc: Sergey Senozhatsky , Daniel Wang , Petr Mladek , Steven Rostedt , stable@vger.kernel.org, Alexander.Levin@microsoft.com, Andrew Morton , byungchul.park@lge.com, dave.hansen@intel.com, hannes@cmpxchg.org, jack@suse.cz, linux-kernel@vger.kernel.org, linux-mm@kvack.org, Mathieu Desnoyers , Mel Gorman , mhocko@kernel.org, pavel@ucw.cz, penguin-kernel@i-love.sakura.ne.jp, Peter Zijlstra , tj@kernel.org, Linus Torvalds , vbabka@suse.cz, Cong Wang , Peter Feiner Subject: Re: 4.14 backport request for dbdda842fe96f: "printk: Add console owner and waiter logic to load balance console writes" Message-ID: <20181212081034.GA32687@jagdpanzerIV> References: <20181004085515.GC12879@jagdpanzerIV> <20181022100952.GA1147@jagdpanzerIV> <20181109064740.GE599@jagdpanzerIV> <20181212052126.GF431@jagdpanzerIV> <20181212062841.GI431@jagdpanzerIV> <20181212064841.GB2746@sasha-vm> MIME-Version: 1.0 Content-Type: text/plain; charset=us-ascii Content-Disposition: inline In-Reply-To: <20181212064841.GB2746@sasha-vm> User-Agent: Mutt/1.11.1 (2018-12-01) Sender: linux-kernel-owner@vger.kernel.org Precedence: bulk List-ID: X-Mailing-List: linux-kernel@vger.kernel.org On (12/12/18 01:48), Sasha Levin wrote: > > > > I guess we still don't have a really clear understanding of what exactly > > > is going in your system > > > > > > I would also like to get to the bottom of it. Unfortunately I haven't > > > got the expertise in this area nor the time to do it yet. Hence the > > > intent to take a step back and backport Steven's patch to fix the > > > issue that has resurfaced in our production recently. > > > > No problem. > > I just meant that -stable people can be a bit "unconvinced". > > The -stable people tried adding this patch back in April, but ended up > getting complaints up the wazoo (https://lkml.org/lkml/2018/4/9/154) > about how this is not -stable material. OK, really didn't know that! I wasn't Cc-ed on that AUTOSEL email, and I wasn't Cc-ed on this whole discussion and found it purely accidentally while browsing linux-mm list. I understand what Petr meant by his email. Not arguing; below are just my 5 cents. > So yes, testing/acks welcome :) OK. The way I see it (and I can be utterly wrong here): The patch set in question, most likely and probably (*and those are theories*), makes panic() deadlock less likely because panic_cpu waits for console_sem owner to release uart_port/console_owner locks before panic_cpu pr_emerg("Kernel panic - not syncing"), dump_stack()-s and brings other CPUs down via stop IPI or NMI. So a precondition is panic CPU != uart_port->lock owner CPU If the panic happens on the same CPU which holds the uart_port spin_lock, then the deadlock is still there, just like before; we have another patch which attempts to fix this (it makes console drivers re-entrant from panic()). So if you are willing to backport this set to -stable, then I wouldn't mind, probably would be more correct if we don't advertise this as a "panic() deadlock fix" tho; we know that deadlock is still possible. And there will be another -stable backport request in a week or so. In the meantime, I can add my Acked-by to this backport if it helps. /* Assuming that my theories explain what's happening with Daniel's systems. */ -ss