Received: by 2002:a25:d7c1:0:0:0:0:0 with SMTP id o184csp4215401ybg; Fri, 25 Oct 2019 15:14:35 -0700 (PDT) X-Google-Smtp-Source: APXvYqz76M9Dtnq8r9NMvvSvx6StLliuCNLKeGTTbg5umamlZdTX8yhO3rh+ydtv/FxRg72jXnL4 X-Received: by 2002:aa7:c24c:: with SMTP id y12mr6470127edo.180.1572041675821; Fri, 25 Oct 2019 15:14:35 -0700 (PDT) ARC-Seal: i=1; a=rsa-sha256; t=1572041675; cv=none; d=google.com; s=arc-20160816; b=Dl5L4TiZJEqvUa8EtfDf/Hr65ol2966Lnqv0/6IMXMiZVPEK6ECwUMN+kPKveF1IA3 nQ6/gw/BemnnqUwzxWmdt0owF4rLqvkOOxIEaS52LWwv46augSdge1pIMM9GLZD8oAxY CS+0a0lyYAIy1+aU1MtLR/cPqAnivVKiKNkYSAm6i74bI8UCE8Jfdn3ruukBsPJlW9Aw Yzow9dyOjvcVMqQGy4p2KvuSZP3syuiroVIchb+nEVkjxvtCIL8cKCgaIbvIEEpnFqT7 jn/XeW37m4IP1r/WslXI1CpKGSesDCJ0s777jUMzdMNugapSdmpBOQiKvkYUp79RXB2J OPDg== ARC-Message-Signature: i=1; a=rsa-sha256; c=relaxed/relaxed; d=google.com; s=arc-20160816; h=list-id:precedence:sender:dkim-signature:content-transfer-encoding :content-language:mime-version:user-agent:date:message-id:subject :from:to; bh=oA1xY6WGO+YbrmP0Bs3pvIsu3O7VlCxPiPqvKf8AA3A=; b=HA7pHTGcZSjAOxxgjcUoi/DcG84S9PCuse3OyeNrKMC6wBLxQM72NLVVbMBXyiR6Pk AXINndTloLTNVgFhEeLqVOAFv590+hoVN02mqPN69qH4BAvYKbG9g4D3EUU+0atyf86R Pt2EBvu5xTSzhB4AM1bxRkCj3RfSJ54AgTCl3LayzyYXDVnzd88E42BuvwtfRBCixeKJ MKi8QiAt0ncFEPQaB4g0ZUHvbFzdxIMXSNmZ4dU2zhfs3aVgAjpkJNn6CgPwlXXwRt+6 XBKAFrHeoUOpSMFgI+eoaPwkBt/UZnNKw4QII5ehwH5frrcEUhz87V4ThC8iIJoKQIyW rDFQ== ARC-Authentication-Results: i=1; mx.google.com; dkim=pass header.i=@nvidia.com header.s=n1 header.b=ehnPVjuY; spf=pass (google.com: best guess record for domain of linux-kernel-owner@vger.kernel.org designates 209.132.180.67 as permitted sender) smtp.mailfrom=linux-kernel-owner@vger.kernel.org; dmarc=pass (p=NONE sp=NONE dis=NONE) header.from=nvidia.com Return-Path: Received: from vger.kernel.org (vger.kernel.org. [209.132.180.67]) by mx.google.com with ESMTP id f8si2494845ede.120.2019.10.25.15.14.07; Fri, 25 Oct 2019 15:14:35 -0700 (PDT) Received-SPF: pass (google.com: best guess record for domain of linux-kernel-owner@vger.kernel.org designates 209.132.180.67 as permitted sender) client-ip=209.132.180.67; Authentication-Results: mx.google.com; dkim=pass header.i=@nvidia.com header.s=n1 header.b=ehnPVjuY; spf=pass (google.com: best guess record for domain of linux-kernel-owner@vger.kernel.org designates 209.132.180.67 as permitted sender) smtp.mailfrom=linux-kernel-owner@vger.kernel.org; dmarc=pass (p=NONE sp=NONE dis=NONE) header.from=nvidia.com Received: (majordomo@vger.kernel.org) by vger.kernel.org via listexpand id S2393085AbfJYH3B (ORCPT + 99 others); Fri, 25 Oct 2019 03:29:01 -0400 Received: from hqemgate15.nvidia.com ([216.228.121.64]:12073 "EHLO hqemgate15.nvidia.com" rhost-flags-OK-OK-OK-OK) by vger.kernel.org with ESMTP id S1725775AbfJYH3B (ORCPT ); Fri, 25 Oct 2019 03:29:01 -0400 Received: from hqpgpgate101.nvidia.com (Not Verified[216.228.121.13]) by hqemgate15.nvidia.com (using TLS: TLSv1.2, DES-CBC3-SHA) id ; Fri, 25 Oct 2019 00:29:09 -0700 Received: from hqmail.nvidia.com ([172.20.161.6]) by hqpgpgate101.nvidia.com (PGP Universal service); Fri, 25 Oct 2019 00:29:00 -0700 X-PGP-Universal: processed; by hqpgpgate101.nvidia.com on Fri, 25 Oct 2019 00:29:00 -0700 Received: from DRHQMAIL107.nvidia.com (10.27.9.16) by HQMAIL111.nvidia.com (172.20.187.18) with Microsoft SMTP Server (TLS) id 15.0.1473.3; Fri, 25 Oct 2019 07:29:00 +0000 Received: from [10.21.133.51] (10.124.1.5) by DRHQMAIL107.nvidia.com (10.27.9.16) with Microsoft SMTP Server (TLS) id 15.0.1473.3; Fri, 25 Oct 2019 07:28:59 +0000 To: Trond Myklebust , linux-tegra , Linux Kernel Mailing List From: Jon Hunter Subject: [REGRESSION v5.3] SUNRPC: Replace the queue timer with a delayed work function (7e0a0e38fcfe) Message-ID: <271ff39f-1f44-b201-6274-85f1085bfc16@nvidia.com> Date: Fri, 25 Oct 2019 08:28:57 +0100 User-Agent: Mozilla/5.0 (X11; Linux x86_64; rv:60.0) Gecko/20100101 Thunderbird/60.9.0 MIME-Version: 1.0 X-Originating-IP: [10.124.1.5] X-ClientProxiedBy: HQMAIL101.nvidia.com (172.20.187.10) To DRHQMAIL107.nvidia.com (10.27.9.16) Content-Type: text/plain; charset="utf-8" Content-Language: en-US Content-Transfer-Encoding: 7bit DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=nvidia.com; s=n1; t=1571988549; bh=oA1xY6WGO+YbrmP0Bs3pvIsu3O7VlCxPiPqvKf8AA3A=; h=X-PGP-Universal:To:From:Subject:Message-ID:Date:User-Agent: MIME-Version:X-Originating-IP:X-ClientProxiedBy:Content-Type: Content-Language:Content-Transfer-Encoding; b=ehnPVjuYCX8OWxsuuALJqjRf9gfryALS5XTI0uWyavaixJ4klOrGajRRy692X4YrO EtOOAzq7fNhanOpw8+0dNLRVeXjQS7QsfYXl9q1x+QDTzrTLkjz75BbaKC54aCDva2 ey04zQ8VtJJrLFLDMGCbo03Vp0xlbTUKCiPSUj8IRY79+rlzq6WGx4JWHPbjBTeiee FTdZZVOSBNvwwb6vvc9QD1bBRsd5yuFl9XQGlOvWSMeCcM7GEMO9ArFvnB5IrV/Seh kligc3+ZGZH5S/6nNTjBJz9IYuKdU+xaU4acOwif2/datDMR81ckNX6XijDNnLO8+q /Cp4srDWGWJYA== Sender: linux-kernel-owner@vger.kernel.org Precedence: bulk List-ID: X-Mailing-List: linux-kernel@vger.kernel.org Hi Trond, Similar to the change 431235818bc3 ("SUNRPC: Declare RPC timers as TIMER_DEFERRABLE") I have been tracking down another suspend/NFS related issue where again I am seeing random delays exiting suspend. The delays can be up to a couple minutes in the worst case and this is causing a suspend test we have to fail. For example, with this change I see ... [ 130.599520] PM: suspend entry (deep) [ 130.607267] Filesystems sync: 0.000 seconds [ 130.615800] Freezing user space processes ... (elapsed 0.001 seconds) done. [ 130.628247] OOM killer disabled. [ 130.635382] Freezing remaining freezable tasks ... (elapsed 0.001 seconds) done. [ 130.648052] printk: Suspending console(s) (use no_console_suspend to debug) [ 130.686015] Disabling non-boot CPUs ... [ 130.689568] IRQ 17: no longer affine to CPU2 [ 130.693435] Entering suspend state LP1 [ 130.693489] Enabling non-boot CPUs ... [ 130.697108] CPU1 is up [ 130.700602] CPU2 is up [ 130.704338] CPU3 is up [ 130.781259] mmc1: queuing unknown CIS tuple 0x80 (50 bytes) [ 130.789742] mmc1: queuing unknown CIS tuple 0x80 (7 bytes) [ 130.792793] mmc1: queuing unknown CIS tuple 0x80 (7 bytes) [ 130.820913] mmc1: queuing unknown CIS tuple 0x02 (1 bytes) [ 131.345569] OOM killer enabled. [ 131.352643] Restarting tasks ... done. [ 131.365480] PM: suspend exit [ 134.524261] asix 1-1:1.0 eth0: link up, 100Mbps, full-duplex, lpa 0xCDE1 [ 243.745788] nfs: server 192.168.99.1 not responding, still trying [ 243.745811] nfs: server 192.168.99.1 not responding, still trying [ 243.767939] nfs: server 192.168.99.1 not responding, still trying [ 243.778233] nfs: server 192.168.99.1 OK [ 243.787058] nfs: server 192.168.99.1 OK [ 243.787542] nfs: server 192.168.99.1 OK Running a git bisect I was able to track it down to the commit referenced in the $subject. Reverting this on top of the current mainline fixes the problem and I no longer see these long delays. Cheers Jon -- nvpublic