Received: by 2002:a05:6a10:1d13:0:0:0:0 with SMTP id pp19csp705942pxb; Thu, 26 Aug 2021 12:35:17 -0700 (PDT) X-Google-Smtp-Source: ABdhPJwtUA5bOHS8gWcpaEwyHiw6Mm2GzMi2g2OUJ0+IpwQbktNABKs4Hx5EYgcw6ICguf1/Xm0i X-Received: by 2002:a17:906:3a57:: with SMTP id a23mr5986993ejf.469.1630006517134; Thu, 26 Aug 2021 12:35:17 -0700 (PDT) ARC-Seal: i=1; a=rsa-sha256; t=1630006517; cv=none; d=google.com; s=arc-20160816; b=xcOLz4eaPwh0YnfSN1SJ2cM9bh7q/FNMPpwiZoX7H2hjDZoZSklc/zoA83eqzZQVmC MnEe3tGYftGyXI8KuFv/T7P66+D0BeVAhs7GfBwlM+klhBbsUIQ3Q+smHwW10xma4rTM UaqkfgV1+TwoDbA8PifrIhWICxuPpeOv8y/TwsRgl/QsQbZqe6VOjV8P48Pdr/1g/Lj2 aMqoMWiBR2CnGq+oAXJx1vMLIFt4mOHejBGog1Mxii+BgHjm4jiytvUj/2pE7vV2Xan6 vKqXL4/BlDEufDlt2Hx5fyIjq/yuCYFhqrqhytYG/AnUak/r81UIbRrqglUQLDBmlkW8 3kYQ== ARC-Message-Signature: i=1; a=rsa-sha256; c=relaxed/relaxed; d=google.com; s=arc-20160816; h=list-id:precedence:cc:to:subject:message-id:date:from:in-reply-to :references:mime-version:dkim-signature; bh=oMrAHFwTOOxPSr7liVCdsJYpe8w0r6TSIQ3hmL51Bv0=; b=XjUSy22159iOQOB4n/ot+cWbw5ElSH/3pHlIn9O0guJL/Nev8UKENw9iruHm3M/nHv cv4ZzWljVFWpj5yz2AAY6H4xXFYrdFu9+Q7bXaA2I3qjd+Y9KdPUGbl7VdOabW0JRZZE Iu0ENqXFK5j+xQWWY/MTT222+lMLG/dKqZMXNI6EkXpxqTQGeTwe4JPaSqM1vRk+k7ZQ SEdTBVD8uEVd91fmJ45LYEsoN2Kq5Y6KEH4OhZtwedezawM2iRVteceCtXKK80PSKG4P FUOTN9hehy9Y3+J5xGmAVeo8yjJ8RAz6v+e5WrejFKC+Y4d+QJMYt4JDwObC+8FqUEEy 9M0g== ARC-Authentication-Results: i=1; mx.google.com; dkim=pass header.i=@gmail.com header.s=20161025 header.b=NvlM36PK; spf=pass (google.com: domain of linux-nfs-owner@vger.kernel.org designates 23.128.96.18 as permitted sender) smtp.mailfrom=linux-nfs-owner@vger.kernel.org; dmarc=pass (p=NONE sp=QUARANTINE dis=NONE) header.from=gmail.com Return-Path: Received: from vger.kernel.org (vger.kernel.org. [23.128.96.18]) by mx.google.com with ESMTP id q7si4098053edd.61.2021.08.26.12.34.45; Thu, 26 Aug 2021 12:35:17 -0700 (PDT) Received-SPF: pass (google.com: domain of linux-nfs-owner@vger.kernel.org designates 23.128.96.18 as permitted sender) client-ip=23.128.96.18; Authentication-Results: mx.google.com; dkim=pass header.i=@gmail.com header.s=20161025 header.b=NvlM36PK; spf=pass (google.com: domain of linux-nfs-owner@vger.kernel.org designates 23.128.96.18 as permitted sender) smtp.mailfrom=linux-nfs-owner@vger.kernel.org; dmarc=pass (p=NONE sp=QUARANTINE dis=NONE) header.from=gmail.com Received: (majordomo@vger.kernel.org) by vger.kernel.org via listexpand id S243361AbhHZTfa (ORCPT + 99 others); Thu, 26 Aug 2021 15:35:30 -0400 Received: from lindbergh.monkeyblade.net ([23.128.96.19]:46170 "EHLO lindbergh.monkeyblade.net" rhost-flags-OK-OK-OK-OK) by vger.kernel.org with ESMTP id S230008AbhHZTf3 (ORCPT ); Thu, 26 Aug 2021 15:35:29 -0400 Received: from mail-pg1-x52c.google.com (mail-pg1-x52c.google.com [IPv6:2607:f8b0:4864:20::52c]) by lindbergh.monkeyblade.net (Postfix) with ESMTPS id 149FEC061757 for ; Thu, 26 Aug 2021 12:34:42 -0700 (PDT) Received: by mail-pg1-x52c.google.com with SMTP id 17so3935251pgp.4 for ; Thu, 26 Aug 2021 12:34:42 -0700 (PDT) DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=gmail.com; s=20161025; h=mime-version:references:in-reply-to:from:date:message-id:subject:to :cc; bh=oMrAHFwTOOxPSr7liVCdsJYpe8w0r6TSIQ3hmL51Bv0=; b=NvlM36PKZHuLE9g25zMfCQ78ovLs7eaa/xEK3vjIiOUmKw5RKHs67tZdfPlNveyYxz FCstlz1z7Lj8NORlcnotT+2vfVvcSpjRbwN77h4SMG3+pblAcF6pb5ZdkpwzEZsz1tzj DazL6pgcdJztjzzDR28aTEVCZZIy0ntDoxroDO+Ark5URoPZrGxx1K6u5hP50ZutjPHs 5TW8MzJKJCOfLsUFOzKcqtaY9SnUuqJglgTTPJlHhL+ecvbpcPzl3vJ+1vYkWILtK7pm mhLr/Oe859oRVgY2Ac2fnqSMaJPezVgT/Kb6eV8kxcWs6nIT44HbpQ5ToojO4CgSDt5Z MbgQ== X-Google-DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=1e100.net; s=20161025; h=x-gm-message-state:mime-version:references:in-reply-to:from:date :message-id:subject:to:cc; bh=oMrAHFwTOOxPSr7liVCdsJYpe8w0r6TSIQ3hmL51Bv0=; b=EIPnqZIHQJOcmTcOogoXd2cNYCjWcnAHT75OAVzKfkVAJiyUN62kmYEOLhoVpYrtFq u+iTnB26OqW3pJFjqDlr32CTeb7+6uwpNIjXAWxrQj2vaipLoT2cUSi7NVg8HWMZIc5F Gv9niGPdGDGGeKWI+d6bT/qwqsyOwA4OuBxAAMbfdzH360/tuAdt1sh0SwGVpPw0bBSF /s9mdJbPhiidEb3J81I1LhwldTIpz6t96daQ0FrzWS+qdW3IjA1V4Id+lLGQhZaAY1Po VFxyFB3DA4BYvrFeCtHQO+Obpl0BddXJmqePdRxFoYYtbLZFWMqWrBnbRPG5WLUyDPF3 XUOg== X-Gm-Message-State: AOAM5315vCdrZskDtfHlAMZ6z8CAqAwUD4QiMQKOOEiAVnfCWy1WXr1z SQtnRTmQSmygfENOuh84rui6FVwxrikU38DQHSY= X-Received: by 2002:a63:4f03:: with SMTP id d3mr4641875pgb.217.1630006481478; Thu, 26 Aug 2021 12:34:41 -0700 (PDT) MIME-Version: 1.0 References: <162846730406.22632.14734595494457390936@noble.neil.brown.name> <162855893202.12431.3423894387218130632@noble.neil.brown.name> <162882238416.1695.4958036322575947783@noble.neil.brown.name> <162907681945.1695.10796003189432247877@noble.neil.brown.name> <87777C39-BDDA-4E1E-83FA-5B46918A66D3@oracle.com> <162915491276.9892.7049267765583701172@noble.neil.brown.name> <162941948235.9892.6790956894845282568@noble.neil.brown.name> <162960371884.9892.13803244995043191094@noble.neil.brown.name> <162966962721.9892.5962616727949224286@noble.neil.brown.name> In-Reply-To: <162966962721.9892.5962616727949224286@noble.neil.brown.name> From: Mike Javorski Date: Thu, 26 Aug 2021 12:34:29 -0700 Message-ID: Subject: Re: NFS server regression in kernel 5.13 (tested w/ 5.13.9) To: NeilBrown Cc: Chuck Lever III , Linux NFS Mailing List Content-Type: text/plain; charset="UTF-8" Precedence: bulk List-ID: X-Mailing-List: linux-nfs@vger.kernel.org I finally had the chance to test with the offloading disabled on the server. Unfortunately, the issues remain, may even be more frequent, but I can't rule out confirmation bias on that one. Not sure where to go from here. - mike On Sun, Aug 22, 2021 at 3:00 PM NeilBrown wrote: > > On Sun, 22 Aug 2021, Mike Javorski wrote: > > 5.12.15-arch1: > > =============== > > 14723 137 0.00921938 > > > > 5.13.12-arch1: (no freezes) > > =============== > > 15333 206 0.013257 > > > > 5.13.12-arch1: (with freezes) > > =============== > > 9230 299 0.0313779 > > > > So a decent bump w/ 5.13, and even more with the freezes. > > Thanks. 9->13 isn't as much as I was hoping for, but it might still be > significant. > > > This machine is an older Opteron server w/ nforce networking. Not sure > > how much offloading it actually does. > > The tcpdump showed the server sending very large packets, which means > that it is at least doing TCP segmentation offload and checksum offload. > Those are very common these days. > > Thanks, > NeilBrown