Received: by 2002:a05:6a10:f347:0:0:0:0 with SMTP id d7csp2465086pxu; Fri, 18 Dec 2020 14:07:27 -0800 (PST) X-Google-Smtp-Source: ABdhPJw9Nk5ev1SJwunV+3z9XQX22jVPZ2Bkq/29ZmC0/v0IZy2/yuqU5WXus8yu8I5hZkAtFOaQ X-Received: by 2002:a50:cd8c:: with SMTP id p12mr6481401edi.380.1608329246891; Fri, 18 Dec 2020 14:07:26 -0800 (PST) ARC-Seal: i=1; a=rsa-sha256; t=1608329246; cv=none; d=google.com; s=arc-20160816; b=mBbzDRHBD/QtzYl4RfL7e2311e566V+EFuiWwLJFtr7GEhf8Z29zoHAqIKQCl2GW5P sQbYMQhnpYRNDUuYmL8hKIx1gX02SPSQJnMEPrk3dbBHgWeMlrHA+oP8aAS/MjQIHZFm xNaqNJSPKf9YY12uT/+h5ruS8kIc2/1ygDu7kBaCOoPbC+YZCTiNd8gO/Djbs/Pn2XrZ jIjJ/J+o6gndc1XV6QCRqflJmDLjheZ7GBHftQzZLEvVJbPxF8FKNIZvZJSdRMtk4TXN VQ/GfgHZbZL+wGNdWUBy9nvehPm9iLU550vB4cxzMHYMFGHNyrYdbpkLpFUqmcWRfqlP tCAg== ARC-Message-Signature: i=1; a=rsa-sha256; c=relaxed/relaxed; d=google.com; s=arc-20160816; h=list-id:precedence:content-transfer-encoding:content-language :in-reply-to:mime-version:user-agent:date:message-id:from:references :cc:to:subject; bh=jYEBiHjjiSoblK+a3RreH1bcGcivL4D0rqeKK629Rkw=; b=hZu7lnI4Y72cV5ipw4SCJ83s+YDOq2PbzyXaNAHCQ+4XQpHDPeorlhbEDLlS2CRX3k SC/jFKwUzardWDDSNcghPpbfE2X/m2YaEk6WbEqNNCyAEchcbipsUI0jGHM4AAgt6I48 ltB8au6Fb20LyiTAGh9vJQxBcMOpAqr/A6wGht+pYdimLWDtt9J0M3dNARe9jN8NI85r OG1f1Ypqg1HbfFVQCFkRbw+b7FKL/wCrfMhFdfLTQehkUoxsTiQXhaylg7/4/zsHQkEg E4L8aZFVERrH4l51Ua8CD5ii+FHmMcC5lxGI9sy57o7qS9RakqEf6D501mT57l5blSHY q9Yw== ARC-Authentication-Results: i=1; mx.google.com; spf=pass (google.com: domain of linux-kernel-owner@vger.kernel.org designates 23.128.96.18 as permitted sender) smtp.mailfrom=linux-kernel-owner@vger.kernel.org; dmarc=fail (p=NONE sp=NONE dis=NONE) header.from=collabora.com Return-Path: Received: from vger.kernel.org (vger.kernel.org. [23.128.96.18]) by mx.google.com with ESMTP id p23si5261504ejf.90.2020.12.18.14.07.04; Fri, 18 Dec 2020 14:07:26 -0800 (PST) Received-SPF: pass (google.com: domain of linux-kernel-owner@vger.kernel.org designates 23.128.96.18 as permitted sender) client-ip=23.128.96.18; Authentication-Results: mx.google.com; spf=pass (google.com: domain of linux-kernel-owner@vger.kernel.org designates 23.128.96.18 as permitted sender) smtp.mailfrom=linux-kernel-owner@vger.kernel.org; dmarc=fail (p=NONE sp=NONE dis=NONE) header.from=collabora.com Received: (majordomo@vger.kernel.org) by vger.kernel.org via listexpand id S1726000AbgLRWAL (ORCPT + 99 others); Fri, 18 Dec 2020 17:00:11 -0500 Received: from bhuna.collabora.co.uk ([46.235.227.227]:42888 "EHLO bhuna.collabora.co.uk" rhost-flags-OK-OK-OK-OK) by vger.kernel.org with ESMTP id S1725925AbgLRWAK (ORCPT ); Fri, 18 Dec 2020 17:00:10 -0500 Received: from [IPv6:2a00:5f00:102:0:944a:5cff:fe54:340e] (unknown [IPv6:2a00:5f00:102:0:944a:5cff:fe54:340e]) (using TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 (128/128 bits)) (No client certificate requested) (Authenticated sender: gtucker) by bhuna.collabora.co.uk (Postfix) with ESMTPSA id E3E4B1F4663F; Fri, 18 Dec 2020 21:59:28 +0000 (GMT) Subject: Re: kernelci/staging-next bisection: sleep.login on rk3288-rock2-square #2286-staging To: Mike Rapoport Cc: Andrea Arcangeli , Andrew Morton , Stephen Rothwell , kernelci-results-staging@groups.io, "kernelci-results@groups.io" , linux-mm@kvack.org, linux-kernel@vger.kernel.org, Mike Rapoport , Baoquan He References: <5fd3e5d9.1c69fb81.f9e69.5028@mx.google.com> <127999c4-7d56-0c36-7f88-8e1a5c934cae@collabora.com> <20201213082314.GA198221@linux.ibm.com> From: Guillaume Tucker Message-ID: <0633d44a-3796-8a1b-e5dc-99fc62aa4dc7@collabora.com> Date: Fri, 18 Dec 2020 21:59:26 +0000 User-Agent: Mozilla/5.0 (X11; Linux x86_64; rv:78.0) Gecko/20100101 Thunderbird/78.4.2 MIME-Version: 1.0 In-Reply-To: <20201213082314.GA198221@linux.ibm.com> Content-Type: text/plain; charset=utf-8 Content-Language: en-US Content-Transfer-Encoding: 7bit Precedence: bulk List-ID: X-Mailing-List: linux-kernel@vger.kernel.org On 13/12/2020 08:23, Mike Rapoport wrote: > Hi Guillaume, > > On Fri, Dec 11, 2020 at 09:53:46PM +0000, Guillaume Tucker wrote: >> Hi Mike, >> >> Please see the bisection report below about a boot failure on >> rk3288 with next-20201210. >> >> Reports aren't automatically sent to the public while we're >> trialing new bisection features on kernelci.org but this one >> looks valid. >> >> There's nothing in the serial console log, probably because it's >> crashing too early during boot. This was confirmed on two rk3288 >> platforms on kernelci.org: rk3288-veyron-jaq and >> rk3288-rock2-square. There's no clear sign about other platforms >> being impacted. >> >> If this looks like something you want to investigate but you >> don't have a platform at hand to reproduce it, please let us know >> if you would like the test to be re-run on kernelci.org with some >> debug config turned on, or if you have a fix to try. > > I'd apprciate if you can build a working kernel with > CONFIG_DEBUG_MEMORY_INIT=y and run it with > > memblock=debug mminit_loglevel=4 > > in the command line. > > If I understand correctly, DEBUG_LL is not an option for these platforms > so if earlyprintk didn't display the log there is not much to do about > it. OK, sorry for the delay. I've built a kernel and booted it as you requested, and also found that the issue was due to this memory area defined in arch/arm/boot/dts/rk3288.dtsi: reserved-memory { #address-cells = <2>; #size-cells = <2>; ranges; /* * The rk3288 cannot use the memory area above 0xfe000000 * for dma operations for some reason. While there is * probably a better solution available somewhere, we * haven't found it yet and while devices with 2GB of ram * are not affected, this issue prevents 4GB from booting. * So to make these devices at least bootable, block * this area for the time being until the real solution * is found. */ dma-unusable@fe000000 { reg = <0x0 0xfe000000 0x0 0x1000000>; }; }; So I've put a hack[1] on top of 950c37691925 to skip adding a node in memblock_enforce_memory_reserved_overlap() if the base address is 0xfe000000, which got the kernel booting. Here's the console log: https://people.collabora.com/~gtucker/tmp/2966825.txt and the full test job details, if this helps: https://lava.collabora.co.uk/scheduler/job/2966825 I haven't really looked much further than that, but I'll be available on Monday to help run other tests if needed. Thanks, Guillaume [1] https://people.collabora.com/~gtucker/tmp/2966825.patch