Received: by 2002:a6b:500f:0:0:0:0:0 with SMTP id e15csp1496868iob; Thu, 5 May 2022 02:42:13 -0700 (PDT) X-Google-Smtp-Source: ABdhPJyVOFmJPdFXxZvDhnAS5thIaEdP1oKzl/6O+vE4GGUzcTh7xLd+IGutF/BJOeZJdlxMlOmm X-Received: by 2002:a17:907:2cc4:b0:6ef:8108:ad11 with SMTP id hg4-20020a1709072cc400b006ef8108ad11mr24800245ejc.20.1651743732910; Thu, 05 May 2022 02:42:12 -0700 (PDT) ARC-Seal: i=1; a=rsa-sha256; t=1651743732; cv=none; d=google.com; s=arc-20160816; b=0Cnw2yhzZoilKmSsINnjVPjvd2yHlA8ORaU66IUIz1hxRdfkUUtYvVd5mvJm/md/yF 7ilerNWdMImQjWql0OEvOIoUAJ46hvtmxh4Sk3aNCwk5+OfSfXU37KNHQLdY8HvRb88J wezaVDTCPGoReFxC30ZvCnxNX+gyoK6V4x4DrBNUSywn0Jn2d9yTWUwEkFcbS/Usc8Q5 K+D+zw52bjr1F0LPoXd18hDL/eNix6rzkWq0SA3F3Xpek8W7lfMWQ2GUpwVMbq+uVhVO hQQBLTa3qOzhLsYoI+uVJ4QkCZwRTAkcoPWN3h8emgle3c1FwXqSCI8j2FsipjXD2GzX Ak5Q== ARC-Message-Signature: i=1; a=rsa-sha256; c=relaxed/relaxed; d=google.com; s=arc-20160816; h=list-id:precedence:mime-version:message-id:date:references :in-reply-to:subject:cc:to:dkim-signature:dkim-signature:from; bh=bIg1HscWN1tvVWMSh8fcrYwHSnmqdBrRg0rNnb5YAr4=; b=oVlY5tMRewz+Q9ZFV3K7W2a7eh6aqZW1/gDI8qO0gJP/g+DRNvs2YAiK8SJ+BkxPLL K1pk5FSaMd8VtjRvn9Q73RBTBW1szlp2tCknCvkKiqG0dF0j8GsotCVhUHynj0qgzp7M OlQRByT6fmwC+z2yL+x7K/TAQGMDTv/OmENHGr07s+2+iwXGmb3a/qzqMSlDMCRs5L54 hZT2//eiv056FqRc8frxYqaiWdZp78ZHNVV9qvCMOVfFdFVQ7dGXcJxYIHRmPogChK6W ud1JBRqDRtCIoYwkPnUOAOk1oJXAskp0hciY57nOWnF/tUICWkNsY4695TOYOP1oxyDM 7R6A== ARC-Authentication-Results: i=1; mx.google.com; dkim=pass header.i=@linutronix.de header.s=2020 header.b=B6aIpuDJ; dkim=neutral (no key) header.i=@linutronix.de header.b=m98oYRuo; spf=pass (google.com: domain of linux-kernel-owner@vger.kernel.org designates 2620:137:e000::1:20 as permitted sender) smtp.mailfrom=linux-kernel-owner@vger.kernel.org; dmarc=pass (p=NONE sp=QUARANTINE dis=NONE) header.from=linutronix.de Return-Path: Received: from out1.vger.email (out1.vger.email. [2620:137:e000::1:20]) by mx.google.com with ESMTP id qc17-20020a170906d8b100b006eff1696032si1644508ejb.14.2022.05.05.02.41.49; Thu, 05 May 2022 02:42:12 -0700 (PDT) Received-SPF: pass (google.com: domain of linux-kernel-owner@vger.kernel.org designates 2620:137:e000::1:20 as permitted sender) client-ip=2620:137:e000::1:20; Authentication-Results: mx.google.com; dkim=pass header.i=@linutronix.de header.s=2020 header.b=B6aIpuDJ; dkim=neutral (no key) header.i=@linutronix.de header.b=m98oYRuo; spf=pass (google.com: domain of linux-kernel-owner@vger.kernel.org designates 2620:137:e000::1:20 as permitted sender) smtp.mailfrom=linux-kernel-owner@vger.kernel.org; dmarc=pass (p=NONE sp=QUARANTINE dis=NONE) header.from=linutronix.de Received: (majordomo@vger.kernel.org) by vger.kernel.org via listexpand id S1359121AbiEDWqv (ORCPT + 99 others); Wed, 4 May 2022 18:46:51 -0400 Received: from lindbergh.monkeyblade.net ([23.128.96.19]:60264 "EHLO lindbergh.monkeyblade.net" rhost-flags-OK-OK-OK-OK) by vger.kernel.org with ESMTP id S1379088AbiEDWqn (ORCPT ); Wed, 4 May 2022 18:46:43 -0400 Received: from galois.linutronix.de (Galois.linutronix.de [IPv6:2a0a:51c0:0:12e:550::1]) by lindbergh.monkeyblade.net (Postfix) with ESMTPS id 4176A53726 for ; Wed, 4 May 2022 15:42:32 -0700 (PDT) From: John Ogness DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=linutronix.de; s=2020; t=1651704149; h=from:from:reply-to:subject:subject:date:date:message-id:message-id: to:to:cc:cc:mime-version:mime-version:content-type:content-type: in-reply-to:in-reply-to:references:references; bh=bIg1HscWN1tvVWMSh8fcrYwHSnmqdBrRg0rNnb5YAr4=; b=B6aIpuDJHUxIAuuxMW19vF2YskBMeJJY1RaYy1313Uyk1P/Z3GVoSvY0AbQFxrYcX1l0Az 2/+LxzzMrc1k1ZWkIqwQ8W3k4ne/1w5GJuxZvSBg2nx92EhrCkYuJ2wCRUN7EiSMXMzYDd l1zVlr9+kWhZ1O7tHnfvpUy6/izefbTfP7q2LCkJgKbaY4KB4Mig6HzKfJE6CZIwC6h8bu vBvgOPtqmZ+JYem07KSOyh4vmjnHfcNrTUIhwocHTXrKf9CbgBxcjyD2Ul+RpraPyBVlBj 55XhJKVhvDEXyxGfxU+jB6XLkv8B4/eKL61MRBrXnIptEUAYhLuU+qKkxUksGg== DKIM-Signature: v=1; a=ed25519-sha256; c=relaxed/relaxed; d=linutronix.de; s=2020e; t=1651704149; h=from:from:reply-to:subject:subject:date:date:message-id:message-id: to:to:cc:cc:mime-version:mime-version:content-type:content-type: in-reply-to:in-reply-to:references:references; bh=bIg1HscWN1tvVWMSh8fcrYwHSnmqdBrRg0rNnb5YAr4=; b=m98oYRuoP7rXD4qoKzjZfEYooVQPaQUBsEyWbXMrczs81p1HFw9GRfR/HRpjmTjK8iPQCm sjWxzDfchSYoCEBQ== To: Marek Szyprowski , Petr Mladek Cc: Sergey Senozhatsky , Steven Rostedt , Thomas Gleixner , linux-kernel@vger.kernel.org, Greg Kroah-Hartman , linux-amlogic@lists.infradead.org Subject: Re: [PATCH printk v5 1/1] printk: extend console_lock for per-console locking In-Reply-To: <87pmktm2a9.fsf@jogness.linutronix.de> References: <2a82eae7-a256-f70c-fd82-4e510750906e@samsung.com> <87fslyv6y3.fsf@jogness.linutronix.de> <51dfc4a0-f6cf-092f-109f-a04eeb240655@samsung.com> <87k0b6blz2.fsf@jogness.linutronix.de> <32bba8f8-dec7-78aa-f2e5-f62928412eda@samsung.com> <45849b63-d7a8-5cc3-26ad-256a28d09991@samsung.com> <87pmktm2a9.fsf@jogness.linutronix.de> Date: Thu, 05 May 2022 00:48:28 +0206 Message-ID: <87a6bwapij.fsf@jogness.linutronix.de> MIME-Version: 1.0 Content-Type: text/plain X-Spam-Status: No, score=-3.9 required=5.0 tests=BAYES_00,DKIM_SIGNED, DKIM_VALID,DKIM_VALID_AU,DKIM_VALID_EF,INVALID_DATE_TZ_ABSURD, RCVD_IN_DNSWL_MED,SPF_HELO_NONE,SPF_PASS,T_SCC_BODY_TEXT_LINE autolearn=ham autolearn_force=no version=3.4.6 X-Spam-Checker-Version: SpamAssassin 3.4.6 (2021-04-09) on lindbergh.monkeyblade.net Precedence: bulk List-ID: X-Mailing-List: linux-kernel@vger.kernel.org On 2022-05-04, John Ogness wrote: > I can reproduce the apparent stack corruption with qemu: > > [ 5.545268] task:pr/ttyAMA0 state:S stack: 0 pid: 26 ppid: 2 flags:0x00000008 > [ 5.545520] Call trace: > [ 5.545620] __switch_to+0x104/0x160 > [ 5.545796] __schedule+0x2f4/0x9f0 > [ 5.546122] schedule+0x54/0xd0 > [ 5.546206] 0x0 I believe I am chasing a ghost. I can rather easily reproduce these strange call traces, but if another sysrq-t is sent afterwards, the call trace is OK. Also, I added trace_dump_stack() into the printk-kthread main loop to dump the stack on every iteration. There I never see any corruption, even though the timestamps are near the sysrq-t dump showing corruption. Moving trace_dump_stack() into amba-pl011:pl011_console_write() also showed no stack corruption at very near times when sysrq-t did. And it should be noted that the console-hanging issues reported in this thread _cannot_ be reproduced with qemu. So I will stop focussing on this "corrupt stack" thing and instead investigate what the meson driver is doing that causes it to get stuck. Since interrupts do not even fire, I'm guessing that the RX interrupts are not being re-enabled (AML_UART_RX_INT_EN) for some code path. This bit is only explicitly set once, in meson_uart_startup(). Whenever the bit is cleared, later the previous value is restored. This is assumed to mean the interrupt gets re-enabled. But if there is some code path where multiple CPUs can modify the register, then the interrupt could end up permanently disabled. I will go through and check if all access to AML_UART_CONTROL is protected by port->lock. John