Received: by 2002:a17:90b:8d0:0:0:0:0 with SMTP id ds16csp5088901pjb; Mon, 27 Jul 2020 12:40:28 -0700 (PDT) X-Google-Smtp-Source: ABdhPJwhbsBDDRuOF6mQlB6VGyt8R5Ak9rQPqh1qoZKvKb2UG/cSro6JlmZAJCRuFInyuRat3rq8 X-Received: by 2002:a05:6402:22d5:: with SMTP id dm21mr22626794edb.26.1595878827960; Mon, 27 Jul 2020 12:40:27 -0700 (PDT) ARC-Seal: i=1; a=rsa-sha256; t=1595878827; cv=none; d=google.com; s=arc-20160816; b=fldrpCJFEk6M4BF2rPZqFyvQPq+1ukDeVzeKGAOCt/M+eAOOjRgsTKf52HFcJt/MCl ZZnFcI1CpRqiOVj4+uhFpiR7+5RtRaH49M1LUC7k9YL/iKpTitG8zG2J71+x+e+e/CJC 96/EWVCiusYo/v24O51UBAGc/v5c7+IwchVnJd7LPtnK1aoCO2BcyyFK8+FtXcKWQt4/ i0gFvhIqRHlTUpaVRiKWb+GwJ8wNSjuozov4yglDslb6rKY9G9ciq+Ou48Nin5AzCg1h LBqgmbVGCA9fa6RwNkKcutGtk9HNbuFSmNR9GhwPuvaXrQzFT3Aq3aLF9hyre5DYIdQx wnnA== ARC-Message-Signature: i=1; a=rsa-sha256; c=relaxed/relaxed; d=google.com; s=arc-20160816; h=list-id:precedence:sender:in-reply-to:content-disposition :mime-version:references:message-id:subject:cc:to:from:date :dkim-signature; bh=/T/e62DHKC//603Xpi1JvAN9yJeAjvgLxSEBrAFT1Sk=; b=kR6IdOAHY1jPg7gUx9xR0ePZF8gYBeD3aJhUo2NJbRbSyI2XcOBmN27EyhnuRumiYC 66GfXUKxwxKWxPFji56A1Go7QXgC7cK+3D/oXvgfi5lrlFU8XgM2xzUAHsYpi59r0a1K 75OTrp9KD62meJMCZzzZRaFALfmN9e9RK9z9mhlxfCXmPvi3X/NFhhrzpPrlGvJboXrS pilUn2kMtpIsnoQjYcOKoZ2ZlFJGBqCVSuf2tZmmPrRLA0lkclEict7UwPGb8tFz1Geq 8rVetLle62zHW594lynXccKAaqo1JwPF2S+7PT+4eggjjKMmfRvj8xTNtlP6ka0OLyKg ZXuQ== ARC-Authentication-Results: i=1; mx.google.com; dkim=pass header.i=@kernel.org header.s=default header.b=2jC35nd4; spf=pass (google.com: domain of linux-kernel-owner@vger.kernel.org designates 23.128.96.18 as permitted sender) smtp.mailfrom=linux-kernel-owner@vger.kernel.org Return-Path: Received: from vger.kernel.org (vger.kernel.org. [23.128.96.18]) by mx.google.com with ESMTP id rl4si3562936ejb.648.2020.07.27.12.40.05; Mon, 27 Jul 2020 12:40:27 -0700 (PDT) Received-SPF: pass (google.com: domain of linux-kernel-owner@vger.kernel.org designates 23.128.96.18 as permitted sender) client-ip=23.128.96.18; Authentication-Results: mx.google.com; dkim=pass header.i=@kernel.org header.s=default header.b=2jC35nd4; spf=pass (google.com: domain of linux-kernel-owner@vger.kernel.org designates 23.128.96.18 as permitted sender) smtp.mailfrom=linux-kernel-owner@vger.kernel.org Received: (majordomo@vger.kernel.org) by vger.kernel.org via listexpand id S1727038AbgG0TfS (ORCPT + 99 others); Mon, 27 Jul 2020 15:35:18 -0400 Received: from mail.kernel.org ([198.145.29.99]:37234 "EHLO mail.kernel.org" rhost-flags-OK-OK-OK-OK) by vger.kernel.org with ESMTP id S1726196AbgG0TfS (ORCPT ); Mon, 27 Jul 2020 15:35:18 -0400 Received: from localhost (83-86-89-107.cable.dynamic.v4.ziggo.nl [83.86.89.107]) (using TLSv1.2 with cipher ECDHE-RSA-AES256-GCM-SHA384 (256/256 bits)) (No client certificate requested) by mail.kernel.org (Postfix) with ESMTPSA id AAFD120738; Mon, 27 Jul 2020 19:35:16 +0000 (UTC) DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/simple; d=kernel.org; s=default; t=1595878517; bh=xsoGM31kW2VrPyMnU5Snl46HwG9kOCh/rN4IW3/JF0E=; h=Date:From:To:Cc:Subject:References:In-Reply-To:From; b=2jC35nd4Q0ZjnCUAnY73gev9iFF2t2wk0UuRnoUMz8dSiFq8MP+4YlRbgmATlLtIG qaae1QhTZ0V2A18mlAkLEJNCLOIS1lpliii17qB8BUfSv4v9/sHPFw4zfNJNB+fbUR igL3KYajQoAZfzyXPbROFj/A2AF2V9hsO87EM/+A= Date: Mon, 27 Jul 2020 21:35:12 +0200 From: Greg KH To: Hugh Dickins Cc: Linus Torvalds , Oleg Nesterov , Michal Hocko , Linux-MM , LKML , Andrew Morton , Tim Chen , Michal Hocko Subject: Re: [RFC PATCH] mm: silence soft lockups from unlock_page Message-ID: <20200727193512.GA236164@kroah.com> References: <20200724152424.GC17209@redhat.com> <20200725101445.GB3870@redhat.com> MIME-Version: 1.0 Content-Type: text/plain; charset=us-ascii Content-Disposition: inline In-Reply-To: Sender: linux-kernel-owner@vger.kernel.org Precedence: bulk List-ID: X-Mailing-List: linux-kernel@vger.kernel.org On Sun, Jul 26, 2020 at 01:30:32PM -0700, Hugh Dickins wrote: > On Sat, 25 Jul 2020, Hugh Dickins wrote: > > On Sat, 25 Jul 2020, Hugh Dickins wrote: > > > On Sat, 25 Jul 2020, Linus Torvalds wrote: > > > > On Sat, Jul 25, 2020 at 3:14 AM Oleg Nesterov wrote: > > > > > > > > > > Heh. I too thought about this. And just in case, your patch looks correct > > > > > to me. But I can't really comment this behavioural change. Perhaps it > > > > > should come in a separate patch? > > > > > > > > We could do that. At the same time, I think both parts change how the > > > > waitqueue works that it might as well just be one "fix page_bit_wait > > > > waitqueue usage". > > > > > > > > But let's wait to see what Hugh's numbers say. > > > > > > Oh no, no no: sorry for getting your hopes up there, I won't come up > > > with any numbers more significant than "0 out of 10" machines crashed. > > > I know it would be *really* useful if I could come up with performance > > > comparisons, or steer someone else to do so: but I'm sorry, cannot. > > > > > > Currently it's actually 1 out of 10 machines crashed, for the same > > > driverland issue seen last time, maybe it's a bad machine; and another > > > 1 out of the 10 machines went AWOL for unknown reasons, but probably > > > something outside the kernel got confused by the stress. No reason > > > to suspect your changes at all (but some unanalyzed "failure"s, of > > > dubious significance, accumulating like last time). > > > > > > I'm optimistic: nothing has happened to warn us off your changes. > > > > Less optimistic now, I'm afraid. > > > > The machine I said had (twice) crashed coincidentally in driverland > > (some USB completion thing): that machine I set running a comparison > > kernel without your changes this morning, while the others still > > running with your changes; and it has now passed the point where it > > twice crashed before (the most troublesome test), without crashing. > > > > Surprising: maybe still just coincidence, but I must look closer at > > the crashes. > > > > The others have now completed, and one other crashed in that > > troublesome test, but sadly without yielding any crash info. > > > > I've just set comparison runs going on them all, to judge whether > > to take the "failure"s seriously; and I'll look more closely at them. > > The comparison runs have not yet completed (except for the one started > early), but they have all got past the most interesting tests, and it's > clear that they do not have the "failure"s seen with your patches. > > >From that I can only conclude that your patches make a difference. > > I've deduced nothing useful from the logs, will have to leave that > to others here with more experience of them. But my assumption now > is that you have successfully removed one bottleneck, so the tests > get somewhat further and now stick in the next bottleneck, whatever > that may be. Which shows up as "failure", where the unlock_page() > wake_up_page_bit() bottleneck had allowed the tests to proceed in > a more serially sedate way. > > The xhci handle_cmd_completion list_del bugs (on an older version > of the driver): weird, nothing to do with page wakeups, I'll just > have to assume that it's some driver bug exposed by the greater > stress allowed down, and let driver people investigate (if it > still manifests) when we take in your improvements. Linus just pointed me at this thread. If you could run: echo -n 'module xhci_hcd =p' > /sys/kernel/debug/dynamic_debug/control and run the same workload to see if anything shows up in the log when xhci crashes, that would be great. Although if you are using an "older version" of the driver, there's not much I can suggest except update to a newer one :) thanks, greg k-h