Received: by 2002:a25:31c3:0:0:0:0:0 with SMTP id x186csp688286ybx; Tue, 5 Nov 2019 04:05:12 -0800 (PST) X-Google-Smtp-Source: APXvYqzk0Lpa7v0kI7voCCLHsFghWpw0J6jYxzMUxbxKaeEOnWuXw+1GHG+0AVDhmuciGExWY6cZ X-Received: by 2002:aa7:c842:: with SMTP id g2mr3642056edt.132.1572955511869; Tue, 05 Nov 2019 04:05:11 -0800 (PST) ARC-Seal: i=1; a=rsa-sha256; t=1572955511; cv=none; d=google.com; s=arc-20160816; b=HwSJdq2OoV9uoVjWxbWL+vAF86adDPXtlIxKiWL3r9YTZRj7ljZO3t4K1E+oNuFY0S xGF/vt9rdiPp6l7OHaXbHfu6gr88qQf4ynlX2HwGOPK75wWHOhlzDQS1fYI3qoQdXT2z MYLPUGYLNYxIqZ1HuN8ZwkhQJ3qFsL0nDWdK/GTGZqXolVpyD1odhCnsAHAS7NLeOlJC hCqH7KtWJ2gs1XD7lPdPRVkUYSdBAQHrZcEgdT43llnlM5rIZahkGw7FdwSv0U9LDZdv tmsE9Pi5Y6HVCHrMMatVwT0m+XCDcSwtA0+WCWf6I9mfQoxzvGFH+d2yZqgdJcoc2bA3 UcBA== ARC-Message-Signature: i=1; a=rsa-sha256; c=relaxed/relaxed; d=google.com; s=arc-20160816; h=list-id:precedence:sender:dkim-signature:content-transfer-encoding :content-language:in-reply-to:mime-version:user-agent:date :message-id:references:to:from:subject; bh=AZfLvetcKty1RUMhPBJVPXae796yrRpZVsA3ANb7qc8=; b=hSr2wdlJzC6AP0OCV7uJnlegnFcKdpMkVhS8UHi7SEhXMpxBuV5tvqMNvzfp0X+5fg MIgk5Zv05ibIE5OP4vwNOYkChuLH3v4tSKezrYkhtF5SEwzPHVtl34l7RM1D13szRdSA 90MZ3VDWF7EwsN5CaISBqH9wqREWQGS9RtYrTyJBQEaTWvTW0RymYLgWmRkTSaLknHQi 5o8xEeQ2IOdI+pCYVsHLY35iGOSwFzAj2nEPZ3ojQgGM1MSqBgE+QO51CAazxMMBvcJ+ fZzka7s15yeNaglY5Dic4NU1JFBWnX9Wvrv633asYx2hAJtjzbgUsGyCEo1fUPlXlFl0 T9iw== ARC-Authentication-Results: i=1; mx.google.com; dkim=pass header.i=@nvidia.com header.s=n1 header.b=XEUxt89i; spf=pass (google.com: best guess record for domain of linux-kernel-owner@vger.kernel.org designates 209.132.180.67 as permitted sender) smtp.mailfrom=linux-kernel-owner@vger.kernel.org; dmarc=pass (p=NONE sp=NONE dis=NONE) header.from=nvidia.com Return-Path: Received: from vger.kernel.org (vger.kernel.org. [209.132.180.67]) by mx.google.com with ESMTP id x51si10492303edd.393.2019.11.05.04.04.43; Tue, 05 Nov 2019 04:05:11 -0800 (PST) Received-SPF: pass (google.com: best guess record for domain of linux-kernel-owner@vger.kernel.org designates 209.132.180.67 as permitted sender) client-ip=209.132.180.67; Authentication-Results: mx.google.com; dkim=pass header.i=@nvidia.com header.s=n1 header.b=XEUxt89i; spf=pass (google.com: best guess record for domain of linux-kernel-owner@vger.kernel.org designates 209.132.180.67 as permitted sender) smtp.mailfrom=linux-kernel-owner@vger.kernel.org; dmarc=pass (p=NONE sp=NONE dis=NONE) header.from=nvidia.com Received: (majordomo@vger.kernel.org) by vger.kernel.org via listexpand id S1730982AbfKEMDy (ORCPT + 99 others); Tue, 5 Nov 2019 07:03:54 -0500 Received: from hqemgate15.nvidia.com ([216.228.121.64]:9260 "EHLO hqemgate15.nvidia.com" rhost-flags-OK-OK-OK-OK) by vger.kernel.org with ESMTP id S1730816AbfKEMDy (ORCPT ); Tue, 5 Nov 2019 07:03:54 -0500 Received: from hqpgpgate101.nvidia.com (Not Verified[216.228.121.13]) by hqemgate15.nvidia.com (using TLS: TLSv1.2, DES-CBC3-SHA) id ; Tue, 05 Nov 2019 04:03:57 -0800 Received: from hqmail.nvidia.com ([172.20.161.6]) by hqpgpgate101.nvidia.com (PGP Universal service); Tue, 05 Nov 2019 04:03:53 -0800 X-PGP-Universal: processed; by hqpgpgate101.nvidia.com on Tue, 05 Nov 2019 04:03:53 -0800 Received: from [10.26.11.93] (10.124.1.5) by HQMAIL107.nvidia.com (172.20.187.13) with Microsoft SMTP Server (TLS) id 15.0.1473.3; Tue, 5 Nov 2019 12:03:51 +0000 Subject: Re: [REGRESSION v5.3] SUNRPC: Replace the queue timer with a delayed work function (7e0a0e38fcfe) From: Jon Hunter To: Trond Myklebust , linux-tegra , Linux Kernel Mailing List References: <271ff39f-1f44-b201-6274-85f1085bfc16@nvidia.com> Message-ID: <329228f8-e194-a021-9226-69a9b6a403ce@nvidia.com> Date: Tue, 5 Nov 2019 12:03:49 +0000 User-Agent: Mozilla/5.0 (X11; Linux x86_64; rv:60.0) Gecko/20100101 Thunderbird/60.9.0 MIME-Version: 1.0 In-Reply-To: <271ff39f-1f44-b201-6274-85f1085bfc16@nvidia.com> X-Originating-IP: [10.124.1.5] X-ClientProxiedBy: HQMAIL101.nvidia.com (172.20.187.10) To HQMAIL107.nvidia.com (172.20.187.13) Content-Type: text/plain; charset="utf-8" Content-Language: en-US Content-Transfer-Encoding: 7bit DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=nvidia.com; s=n1; t=1572955438; bh=AZfLvetcKty1RUMhPBJVPXae796yrRpZVsA3ANb7qc8=; h=X-PGP-Universal:Subject:From:To:References:Message-ID:Date: User-Agent:MIME-Version:In-Reply-To:X-Originating-IP: X-ClientProxiedBy:Content-Type:Content-Language: Content-Transfer-Encoding; b=XEUxt89iA1MfpqxryeY4mRpRa3HkNbFowEirE7A5fqMaiaYH/nhEk6TJwxjaUsZLO z0HHLfyzA9vsznwdyZQTPcAchJ6L3sv2axAvUD3DGV3h5GG+qyumIJM/ReQHQ88LM5 rkLntd2V+Fz99XbD8iNXkdzpQyFYK2ikhgq1Yn9M/6LWZEkkiOZKZM7to/uuTGSJoo 3zrTQS6H+tw5HTxpT18rkMl/f2RgmB4/nEdh78nVwoFKVXBWaFMMcieq8dMCEL0mHq rd4k/KzpCwsXmpvjA1MUsbbh/h11EKDeF5p3BY5sUTeiadahsUzDSMOv+17vOIYycC BAHZwcoY4FaJQ== Sender: linux-kernel-owner@vger.kernel.org Precedence: bulk List-ID: X-Mailing-List: linux-kernel@vger.kernel.org Hi Trond, Any feedback on this? Jon On 25/10/2019 08:28, Jon Hunter wrote: > Hi Trond, > > Similar to the change 431235818bc3 ("SUNRPC: Declare RPC timers as > TIMER_DEFERRABLE") I have been tracking down another suspend/NFS related > issue where again I am seeing random delays exiting suspend. The delays > can be up to a couple minutes in the worst case and this is causing a > suspend test we have to fail. For example, with this change I see ... > > [ 130.599520] PM: suspend entry (deep) > > [ 130.607267] Filesystems sync: 0.000 seconds > > [ 130.615800] Freezing user space processes ... (elapsed 0.001 seconds) done. > > [ 130.628247] OOM killer disabled. > > [ 130.635382] Freezing remaining freezable tasks ... (elapsed 0.001 seconds) done. > > [ 130.648052] printk: Suspending console(s) (use no_console_suspend to debug) > > [ 130.686015] Disabling non-boot CPUs ... > > [ 130.689568] IRQ 17: no longer affine to CPU2 > > [ 130.693435] Entering suspend state LP1 > > [ 130.693489] Enabling non-boot CPUs ... > > [ 130.697108] CPU1 is up > > [ 130.700602] CPU2 is up > > [ 130.704338] CPU3 is up > > [ 130.781259] mmc1: queuing unknown CIS tuple 0x80 (50 bytes) > > [ 130.789742] mmc1: queuing unknown CIS tuple 0x80 (7 bytes) > > [ 130.792793] mmc1: queuing unknown CIS tuple 0x80 (7 bytes) > > [ 130.820913] mmc1: queuing unknown CIS tuple 0x02 (1 bytes) > > [ 131.345569] OOM killer enabled. > > [ 131.352643] Restarting tasks ... done. > > [ 131.365480] PM: suspend exit > > [ 134.524261] asix 1-1:1.0 eth0: link up, 100Mbps, full-duplex, lpa 0xCDE1 > > [ 243.745788] nfs: server 192.168.99.1 not responding, still trying > > [ 243.745811] nfs: server 192.168.99.1 not responding, still trying > > [ 243.767939] nfs: server 192.168.99.1 not responding, still trying > > [ 243.778233] nfs: server 192.168.99.1 OK > > [ 243.787058] nfs: server 192.168.99.1 OK > > [ 243.787542] nfs: server 192.168.99.1 OK > > > Running a git bisect I was able to track it down to the commit referenced > in the $subject. Reverting this on top of the current mainline fixes the > problem and I no longer see these long delays. > > Cheers > Jon > -- nvpublic