Received: by 2002:a05:6a10:5bc5:0:0:0:0 with SMTP id os5csp6049923pxb; Mon, 8 Nov 2021 02:16:15 -0800 (PST) X-Google-Smtp-Source: ABdhPJxhtTSqVV0urutzJMgH0ePZPYDY2agT53ATouCQvxVFIe4LWfnU1ZrdvssgdjqcALoT9Nv8 X-Received: by 2002:a02:c761:: with SMTP id k1mr23488972jao.74.1636366575545; Mon, 08 Nov 2021 02:16:15 -0800 (PST) ARC-Seal: i=1; a=rsa-sha256; t=1636366575; cv=none; d=google.com; s=arc-20160816; b=mW5aieOcWrZ4+7wFhF1XNuNTZ3cMumXhcTg4aCQmXuOVqzYU8aKJKSFvGbQE0lD1An D9+lSz0vKV23o0AVo+jELg1cMYJZyzb1XO39BmtL/7+a4IQNkWPa2Gc0DmYM4NKdDMKh bZ/u89QNJ/gQJMvAWYb3gVVw0piCo7IR4EHVE1KuVQG+3NF6evCoDpquAhPOceF4m3zX tB3cq8uxkOxB9K8NAiE64MDnrsQJZkbUFt8cPNmt/VaAW2xfhXpNBUOITo0AjOMlJMin 35NXQIuQr+vLopnJGmaFoH4xEUyyeqG4DpFPM4i5JxUeDsMG94p38fCAwEm24aoGjFh4 pwoQ== ARC-Message-Signature: i=1; a=rsa-sha256; c=relaxed/relaxed; d=google.com; s=arc-20160816; h=list-id:precedence:in-reply-to:content-disposition:mime-version :references:message-id:subject:cc:to:from:date:dkim-signature; bh=U4mTTJb6i/blYyrscgM74QGvIFAfksSxGPBnkYKnOms=; b=EvcLJUc8M9RlIdRN8ZyTxTLd3wX67NZQ1HiLA5waYMvN2K/oH8aTMTeU05hk+PceGe gP/+TMZqhRRLs6MSXh1KkCBQbQuuScnu8QrGhDlt/gjd8IUVhRhVDwBUh1vhuu0MT5Zj 2jw0Chk7CQd3C2yODve4c7NxSBEVXNRwvs3f9e3XiVe0pVjbdpQ4f4tgDpIuxbrUsoyD YLvN9Tp72KT32IiJFXYEBQJFctR1eLLT4FmHq2FGRSNtkTtYklu4XyJ+DpqEVVmD3bDW Sq5CUcgO2yZwyKs9i4zI++9GgApLONAJhGBNFZ2udLsJi41zF8hZv8j/OalJENsgFgrd 2Y4w== ARC-Authentication-Results: i=1; mx.google.com; dkim=pass header.i=@infradead.org header.s=desiato.20200630 header.b=mRBCZfR7; spf=pass (google.com: domain of linux-kernel-owner@vger.kernel.org designates 23.128.96.18 as permitted sender) smtp.mailfrom=linux-kernel-owner@vger.kernel.org Return-Path: Received: from vger.kernel.org (vger.kernel.org. [23.128.96.18]) by mx.google.com with ESMTP id k12si32982767jav.102.2021.11.08.02.16.02; Mon, 08 Nov 2021 02:16:15 -0800 (PST) Received-SPF: pass (google.com: domain of linux-kernel-owner@vger.kernel.org designates 23.128.96.18 as permitted sender) client-ip=23.128.96.18; Authentication-Results: mx.google.com; dkim=pass header.i=@infradead.org header.s=desiato.20200630 header.b=mRBCZfR7; spf=pass (google.com: domain of linux-kernel-owner@vger.kernel.org designates 23.128.96.18 as permitted sender) smtp.mailfrom=linux-kernel-owner@vger.kernel.org Received: (majordomo@vger.kernel.org) by vger.kernel.org via listexpand id S237801AbhKHIOx (ORCPT + 99 others); Mon, 8 Nov 2021 03:14:53 -0500 Received: from lindbergh.monkeyblade.net ([23.128.96.19]:39430 "EHLO lindbergh.monkeyblade.net" rhost-flags-OK-OK-OK-OK) by vger.kernel.org with ESMTP id S235100AbhKHIOw (ORCPT ); Mon, 8 Nov 2021 03:14:52 -0500 Received: from desiato.infradead.org (desiato.infradead.org [IPv6:2001:8b0:10b:1:d65d:64ff:fe57:4e05]) by lindbergh.monkeyblade.net (Postfix) with ESMTPS id 9498DC061570 for ; Mon, 8 Nov 2021 00:12:08 -0800 (PST) DKIM-Signature: v=1; a=rsa-sha256; q=dns/txt; c=relaxed/relaxed; d=infradead.org; s=desiato.20200630; h=In-Reply-To:Content-Type:MIME-Version: References:Message-ID:Subject:Cc:To:From:Date:Sender:Reply-To: Content-Transfer-Encoding:Content-ID:Content-Description; bh=U4mTTJb6i/blYyrscgM74QGvIFAfksSxGPBnkYKnOms=; b=mRBCZfR7D6wQOnUZkcNoH+YN5j xHsk3ps0zpw8hF3V+QF3ZJj/N6h57I3Jv3SG8C6o23AieQyQBB5xmqmhFzSsDwD7Isk8dKNco60XX wMNTMK/UJ4ObNI8AxEi4nZWlWKWteXWH6JfZ2LL+PSdmlG1yONF7olBAjbd75ML6O3m/HXXwclo9/ lhdYDgA33fI3NVCf9w+K9VsbK1+foQI8DYoZ4ImyleUp2w2e8dgNWWFy9NfutjcN4CPfNCQg/Tqgx M8rlFVaY9UCVKc/Lwxh7zNLI3XfzrdZ3H3DtykZaRrwVF4n1v2XdvpH1FkrBpm8JPq/SOOgLc8RyS X8QLB7QA==; Received: from j217100.upc-j.chello.nl ([24.132.217.100] helo=noisy.programming.kicks-ass.net) by desiato.infradead.org with esmtpsa (Exim 4.94.2 #2 (Red Hat Linux)) id 1mjzl4-00Eqot-Ep; Mon, 08 Nov 2021 08:12:02 +0000 Received: from hirez.programming.kicks-ass.net (hirez.programming.kicks-ass.net [192.168.1.225]) (using TLSv1.3 with cipher TLS_AES_256_GCM_SHA384 (256/256 bits) key-exchange X25519 server-signature RSA-PSS (2048 bits) server-digest SHA256) (Client did not present a certificate) by noisy.programming.kicks-ass.net (Postfix) with ESMTPS id 2CD183000DD; Mon, 8 Nov 2021 09:12:01 +0100 (CET) Received: by hirez.programming.kicks-ass.net (Postfix, from userid 1000) id 0773F2CD0FE87; Mon, 8 Nov 2021 09:12:01 +0100 (CET) Date: Mon, 8 Nov 2021 09:12:00 +0100 From: Peter Zijlstra To: "Colin King (gmail)" Cc: Thomas Gleixner , "linux-kernel@vger.kernel.org" Subject: Re: DEAD callback error for CPU, WARNING: CPU: 3 PID: 1134 at kernel/cpu.c:1163 _cpu_down+0x20a/0x3a0 Message-ID: References: MIME-Version: 1.0 Content-Type: text/plain; charset=us-ascii Content-Disposition: inline In-Reply-To: Precedence: bulk List-ID: X-Mailing-List: linux-kernel@vger.kernel.org On Sun, Nov 07, 2021 at 11:59:17AM +0000, Colin King (gmail) wrote: > On a SMP system in a VM, a deadlock callback error can be reproduced with > 5.15, tested from head at commit d4439a1189f93d0ac1eaf0197db8e6b3e197d5c7 > > Didn't see this issue on 5.13 > > How to reproduce: > > git clone https://github.com/ColinIanKing/stress-ng > cd stress-ng > make -j $(nproc) > sudo ./stress-ng --cpu-online 0 -t 15 --pathological > > Tested on a 8 thread virtual machine, 4MB of memory. > > [ 2239.378724] smpboot: CPU 6 is now offline > [ 2239.379443] smpboot: Booting Node 0 Processor 6 APIC 0x6 > [ 2239.380169] kvm-clock: cpu 6, msr 79201181, secondary cpu clock > [ 2239.401652] ------------[ cut here ]------------ > [ 2239.401658] DEAD callback error for CPU6 > [ 2239.401721] WARNING: CPU: 3 PID: 1134 at kernel/cpu.c:1163 > _cpu_down+0x20a/0x3a0 Can you do the same with whatever magic is required to get the pr_debug() output from cpuhp_down_callbacks() included?