Received: by 2002:a05:6a10:1287:0:0:0:0 with SMTP id d7csp5118621pxv; Wed, 28 Jul 2021 03:40:21 -0700 (PDT) X-Google-Smtp-Source: ABdhPJzj4pXy/w55TJDJXjvnA6/KncsbzZEvqgPMIQFXxhFsm1FMvWb8541JCwRq0T+Nn32siFf6 X-Received: by 2002:a17:906:cd1a:: with SMTP id oz26mr7773394ejb.101.1627468821729; Wed, 28 Jul 2021 03:40:21 -0700 (PDT) ARC-Seal: i=1; a=rsa-sha256; t=1627468821; cv=none; d=google.com; s=arc-20160816; b=bKnA8x0rZW6GzBKbiTF2IeMN7czOO57pzvX8D9NaSV2bA/VEx6dv6/MOa3YV05iAgq 0mVqtz2MOnFKDeeCn0uqizV2nlSQZsBxq5rKIVjmDkbps8XzyROpY8ejoDebkIu1vhAM TmPvgBUF8rAg/NGs1XxH1sVkBQizVGJea0lt/77OLmjK64QpPLXivpuxQyg9YtXRws51 pOfnj6fd3utSCeTw4cpiXTLOsY0qNf/VyrKO3k4mlCOSMk4pjGPIPyEVayqR8Exw2rrk +7QsYoEumlKEeGcJLMue5ZizcAIcLP89SUkte0U9weXYT3MWUALN1aRzAkKcHmlY6o4E LT2g== ARC-Message-Signature: i=1; a=rsa-sha256; c=relaxed/relaxed; d=google.com; s=arc-20160816; h=list-id:precedence:mime-version:message-id:date:references :in-reply-to:in-reply-to:subject:cc:to:from:dkim-signature; bh=GLHGz1FXxIhQ3XhDOzPPegQx13bF4kzSMFW01F22ZNM=; b=LCwoZ4Qe8X4KH6CE4PwXZ4cIuGWtdHwCxNvRvLZ+imnMXDQ2yiK2wwBOQbiXoefbDC XN8Oyc30Exku2vjeJhtWV/MxGcWY6raaL7kKpg33O7xtF1HaS76xQ5RO+NSufV6mkvu4 JbpPTy1J77pmP3ygtbjKJC/iDpQRpUr9RIYj/e3RBWz1phG9jIhv/fJH7eaIzbnnodNK yf8yp8AsG3yEHM2lEY+fU1ad5sKOXop7mWBW7DF3ltb8KuiShR0Dva9LlXuhau6zybxN aYft+zzWvCULs1sLJFt1VQdkdCBehGYX6GjJ+NR65MonP5sm537vgsdW7l2HJrrv6TSz 0sNg== ARC-Authentication-Results: i=1; mx.google.com; dkim=fail header.i=@ibm.com header.s=pp1 header.b=G30xu61j; spf=pass (google.com: domain of linux-kernel-owner@vger.kernel.org designates 23.128.96.18 as permitted sender) smtp.mailfrom=linux-kernel-owner@vger.kernel.org; dmarc=fail (p=NONE sp=NONE dis=NONE) header.from=ibm.com Return-Path: Received: from vger.kernel.org (vger.kernel.org. [23.128.96.18]) by mx.google.com with ESMTP id d5si6093307ejd.487.2021.07.28.03.39.58; Wed, 28 Jul 2021 03:40:21 -0700 (PDT) Received-SPF: pass (google.com: domain of linux-kernel-owner@vger.kernel.org designates 23.128.96.18 as permitted sender) client-ip=23.128.96.18; Authentication-Results: mx.google.com; dkim=fail header.i=@ibm.com header.s=pp1 header.b=G30xu61j; spf=pass (google.com: domain of linux-kernel-owner@vger.kernel.org designates 23.128.96.18 as permitted sender) smtp.mailfrom=linux-kernel-owner@vger.kernel.org; dmarc=fail (p=NONE sp=NONE dis=NONE) header.from=ibm.com Received: (majordomo@vger.kernel.org) by vger.kernel.org via listexpand id S235538AbhG1Kie (ORCPT + 99 others); Wed, 28 Jul 2021 06:38:34 -0400 Received: from mx0a-001b2d01.pphosted.com ([148.163.156.1]:14032 "EHLO mx0a-001b2d01.pphosted.com" rhost-flags-OK-OK-OK-OK) by vger.kernel.org with ESMTP id S236007AbhG1KiZ (ORCPT ); Wed, 28 Jul 2021 06:38:25 -0400 Received: from pps.filterd (m0098409.ppops.net [127.0.0.1]) by mx0a-001b2d01.pphosted.com (8.16.0.43/8.16.0.43) with SMTP id 16SAXs3A004097; Wed, 28 Jul 2021 06:38:16 -0400 DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=ibm.com; h=from : to : cc : cc : subject : in-reply-to : in-reply-to : references : date : message-id : mime-version : content-type; s=pp1; bh=GLHGz1FXxIhQ3XhDOzPPegQx13bF4kzSMFW01F22ZNM=; b=G30xu61jkJPek22b46lZ6aFxw1xkzwnHlTqFwSpy64bdjwsrPuO6z5ZcppgSP5hXejvZ Qlq0ByH1j7iN1XH6bb+WI0ExMEF+m93Wb3bf3hcUSleYpHOX784ySmiJrGB6A9/aDAoh 3uTxEQTHDdhOL9v56Jq49032Z/ZhfstsEi4gYnx5BwcPsdGuIqkBlWg6A5Vt3oMLV44/ a+CCZswrLFNCbk1LUYBpabpbsnIcX65Vhyu+mkRQdMomM6m7YtALP5XgNpNz99bB2Bjb Cz08jepfkmIPwpooivIPwxAxGjbVhDvjU8Q8W0zqTYNbPd/vOBH73UsPb84TBKHWwz7C aw== Received: from pps.reinject (localhost [127.0.0.1]) by mx0a-001b2d01.pphosted.com with ESMTP id 3a35dcgm7p-1 (version=TLSv1.2 cipher=ECDHE-RSA-AES256-GCM-SHA384 bits=256 verify=NOT); Wed, 28 Jul 2021 06:38:15 -0400 Received: from m0098409.ppops.net (m0098409.ppops.net [127.0.0.1]) by pps.reinject (8.16.0.43/8.16.0.43) with SMTP id 16SAY6cD005414; Wed, 28 Jul 2021 06:38:15 -0400 Received: from ppma06ams.nl.ibm.com (66.31.33a9.ip4.static.sl-reverse.com [169.51.49.102]) by mx0a-001b2d01.pphosted.com with ESMTP id 3a35dcgm5m-1 (version=TLSv1.2 cipher=ECDHE-RSA-AES256-GCM-SHA384 bits=256 verify=NOT); Wed, 28 Jul 2021 06:38:15 -0400 Received: from pps.filterd (ppma06ams.nl.ibm.com [127.0.0.1]) by ppma06ams.nl.ibm.com (8.16.1.2/8.16.1.2) with SMTP id 16SAameH013742; Wed, 28 Jul 2021 10:38:12 GMT Received: from b06cxnps4075.portsmouth.uk.ibm.com (d06relay12.portsmouth.uk.ibm.com [9.149.109.197]) by ppma06ams.nl.ibm.com with ESMTP id 3a235kh01j-1 (version=TLSv1.2 cipher=ECDHE-RSA-AES256-GCM-SHA384 bits=256 verify=NOT); Wed, 28 Jul 2021 10:38:12 +0000 Received: from d06av24.portsmouth.uk.ibm.com (mk.ibm.com [9.149.105.60]) by b06cxnps4075.portsmouth.uk.ibm.com (8.14.9/8.14.9/NCO v10.0) with ESMTP id 16SAc9IN26214682 (version=TLSv1/SSLv3 cipher=DHE-RSA-AES256-GCM-SHA384 bits=256 verify=OK); Wed, 28 Jul 2021 10:38:09 GMT Received: from d06av24.portsmouth.uk.ibm.com (unknown [127.0.0.1]) by IMSVA (Postfix) with ESMTP id 0921C42059; Wed, 28 Jul 2021 10:38:09 +0000 (GMT) Received: from d06av24.portsmouth.uk.ibm.com (unknown [127.0.0.1]) by IMSVA (Postfix) with ESMTP id C6ECB4205E; Wed, 28 Jul 2021 10:38:08 +0000 (GMT) Received: from localhost (unknown [9.171.23.245]) by d06av24.portsmouth.uk.ibm.com (Postfix) with ESMTP; Wed, 28 Jul 2021 10:38:08 +0000 (GMT) From: Alexander Egorenkov To: Luis Chamberlain , Bruno Goncalves Cc: Rasmus Villemoes , akpm@linux-foundation.org, bp@alien8.de, corbet@lwn.net, gregkh@linuxfoundation.org, jeyu@kernel.org, linux-kernel@vger.kernel.org, ndesaulniers@google.com, torvalds@linux-foundation.org, Dave Young Subject: Re: [PATCH v3 1/2] init/initramfs.c: do unpacking asynchronously In-Reply-To: In-Reply-To: References: <87sg04p315.fsf@oc8242746057.ibm.com> Date: Wed, 28 Jul 2021 12:38:08 +0200 Message-ID: <87pmv2g1v3.fsf@oc8242746057.ibm.com> MIME-Version: 1.0 Content-Type: text/plain X-TM-AS-GCONF: 00 X-Proofpoint-ORIG-GUID: okvylCCKtII6JnkUWIkNJJLuwowrJmSr X-Proofpoint-GUID: JG9nErZ17zV5vNk2QRDrKdn4U2vX4oTA X-Proofpoint-Virus-Version: vendor=fsecure engine=2.50.10434:6.0.391,18.0.790 definitions=2021-07-28_07:2021-07-27,2021-07-28 signatures=0 X-Proofpoint-Spam-Details: rule=outbound_notspam policy=outbound score=0 priorityscore=1501 spamscore=0 impostorscore=0 lowpriorityscore=0 bulkscore=0 mlxscore=0 suspectscore=0 phishscore=0 clxscore=1015 mlxlogscore=999 adultscore=0 malwarescore=0 classifier=spam adjust=0 reason=mlx scancount=1 engine=8.12.0-2107140000 definitions=main-2107280058 Precedence: bulk List-ID: X-Mailing-List: linux-kernel@vger.kernel.org Luis Chamberlain writes: > On Tue, Jul 27, 2021 at 09:31:54AM +0200, Bruno Goncalves wrote: >> On Mon, Jul 26, 2021 at 1:46 PM Rasmus Villemoes >> wrote: >> > >> > On 24/07/2021 09.46, Alexander Egorenkov wrote: >> > > Hello, >> > > >> > > since e7cb072eb988 ("init/initramfs.c: do unpacking asynchronously"), we >> > > started seeing the following problem on s390 arch regularly: >> > > >> > > [ 5.039734] wait_for_initramfs() called before rootfs_initcalls > > So some context here, which might help. > > The initramfs_cookie is initialized until a a rootfs_initcall() is > called, in this case populate_rootfs(). The code is small, so might > as well include it: > > static int __init populate_rootfs(void) > { > initramfs_cookie = async_schedule_domain(do_populate_rootfs, NULL, > &initramfs_domain); > if (!initramfs_async) > wait_for_initramfs(); > return 0; > } > rootfs_initcall(populate_rootfs); > > The warning you see comes from a situation where a wait_for_initramfs() > gets called but we haven't yet initialized initramfs_cookie. There are > only a few calls for wait_for_initramfs() in the kernel, and the only > thing I can think of is that somehow s390 may rely on a usermode helper > early on, but not every time. > > What umh calls does s390 issue? > >> Unfortunately, we haven't been able to find the root cause, but since >> June 23rd we haven't hit this panic... >> >> Btw, this panic we were hitting only when testing kernels from "scsi" >> and "block" trees. > > Do you use drdb maybe? > > Luis You are right, it looks like wait_for_initramfs() is being triggered by drbg: [ 3.396467] [<00000001f8d917e4>] call_usermodehelper_exec+0x44/0x1c8 [ 3.396470] [<00000001f8da5524>] __request_module+0x17c/0x3d0 [ 3.396473] [<00000001f93b4808>] crypto_alg_mod_lookup+0x228/0x290 [ 3.396475] [<00000001f93b4aae>] crypto_alloc_tfm_node+0x5e/0x158 [ 3.396477] [<00000001f93bbe14>] crypto_alloc_shash+0x34/0x40 [ 3.396480] [<00000001f93d2ee2>] drbg_init_hash_kernel+0x3a/0x100 [ 3.396482] [<00000001f93d306e>] drbg_alloc_state+0xc6/0x280 [ 3.396484] [<00000001f93d33d8>] drbg_kcapi_seed+0x1b0/0x310 [ 3.396486] [<00000001f93d1884>] crypto_rng_reset+0x5c/0xe0 [ 3.396488] [<00000001f93c1750>] alg_test_drbg+0x300/0x478 [ 3.396490] [<00000001f93c0934>] alg_test+0x304/0x530 [ 3.396493] [<00000001f93bedb8>] cryptomgr_test+0x68/0x70 [ 3.396495] [<00000001f8d9f40c>] kthread+0x14c/0x160 [ 3.396497] [<00000001f8d2af14>] __ret_from_fork+0x3c/0x58 [ 3.396500] [<00000001f984c86a>] ret_from_fork+0xa/0x30 Regards Alex