Received: by 2002:ad5:474a:0:0:0:0:0 with SMTP id i10csp460064imu;
        Tue, 27 Nov 2018 15:22:14 -0800 (PST)
X-Google-Smtp-Source: AFSGD/WUTBt5/Q/vAtHYc+RILQwndft+KjoLDw9wfM5qjdVavWlPFFLeOFNxOGzS9UFMrkhh0A1L
X-Received: by 2002:a63:f658:: with SMTP id u24mr31312506pgj.267.1543360933986;
        Tue, 27 Nov 2018 15:22:13 -0800 (PST)
ARC-Seal: i=1; a=rsa-sha256; t=1543360933; cv=none;
        d=google.com; s=arc-20160816;
        b=c04H7Yt/S4oo7VRHKw6zH8SzPOwhIYF4x3hQrx1+eOSDJdDk7vOaFzWSHEw8fmI4Cm
         TOOuZZABsitPYs2muItdfpZG9IVtgGyUBnhRtkwSZZ5p7dYlHoPpGFVXDjqxmMV/DVPw
         eOLeku9OjCMFiGP6iESitZcpHxMVHrzxJLYUwVS9V13wj9oR70BHwFRjotwLItmv5aiv
         2ICrZgRX3Xe8ndZFlpCANEu8fkvqERWAyJ3N1bPl5bdb29WNCpyx0yasNyxfiY4/o0/0
         KJI5HekSkedWkAkFdHhIAy1aWWnEFY8P2jPLlvD7VbxI41jsEhOSt3xeF8ZWN86m7g0O
         fCzg==
ARC-Message-Signature: i=1; a=rsa-sha256; c=relaxed/relaxed; d=google.com; s=arc-20160816;
        h=list-id:precedence:sender:mime-version:message-id:references
         :in-reply-to:subject:cc:date:to:from;
        bh=gkxV0smpR1A7pE0hQg1dW61H9EEd5gubBdWGxDKSgGc=;
        b=Bgt6vwIKP5UsS33z5L/pQ5CN54tzV6buB40ORE6Adyfz04E5BCvLi6jI0x2MwgG2bq
         zf2UHX4ci5vHEy4ynOvU/RDT3fzg2CbKcGr0HUx63v3QcZCGJ5XIC6rGxtaM4+v+mf/U
         TZbevNKBP0zoNLZ/mCDXTez4nRZJmU+B0SHIk2xR8cBBylk+Be7xc0iPRSS5eoWTCIbj
         ewdLHpFc+TTr35sjZIbtQoqFBpFhVG2cVeXr06ctaYo8aQ4wbk8JXxZJMEnjUULnAxFm
         NtyQeDS5zKQz1rDQAKPR+Q1iJK6Ww0KXGZH5s79iyLigs7BMuVI7azE1Y07yTINO/prL
         pOeA==
ARC-Authentication-Results: i=1; mx.google.com;
       spf=pass (google.com: best guess record for domain of linux-kernel-owner@vger.kernel.org designates 209.132.180.67 as permitted sender) smtp.mailfrom=linux-kernel-owner@vger.kernel.org
Return-Path: <linux-kernel-owner@vger.kernel.org>
Received: from vger.kernel.org (vger.kernel.org. [209.132.180.67])
        by mx.google.com with ESMTP id l184si4768112pgd.523.2018.11.27.15.21.58;
        Tue, 27 Nov 2018 15:22:13 -0800 (PST)
Received-SPF: pass (google.com: best guess record for domain of linux-kernel-owner@vger.kernel.org designates 209.132.180.67 as permitted sender) client-ip=209.132.180.67;
Authentication-Results: mx.google.com;
       spf=pass (google.com: best guess record for domain of linux-kernel-owner@vger.kernel.org designates 209.132.180.67 as permitted sender) smtp.mailfrom=linux-kernel-owner@vger.kernel.org
Received: (majordomo@vger.kernel.org) by vger.kernel.org via listexpand
        id S1726532AbeK1KUM (ORCPT <rfc822;xiaochao379261131@gmail.com>
        + 99 others); Wed, 28 Nov 2018 05:20:12 -0500
Received: from mx2.suse.de ([195.135.220.15]:35994 "EHLO mx1.suse.de"
        rhost-flags-OK-OK-OK-FAIL) by vger.kernel.org with ESMTP
        id S1726068AbeK1KUM (ORCPT <rfc822;linux-kernel@vger.kernel.org>);
        Wed, 28 Nov 2018 05:20:12 -0500
X-Virus-Scanned: by amavisd-new at test-mx.suse.de
Received: from relay1.suse.de (unknown [195.135.220.254])
        by mx1.suse.de (Postfix) with ESMTP id E13E4B05E;
        Tue, 27 Nov 2018 23:20:40 +0000 (UTC)
From:   NeilBrown <neilb@suse.com>
To:     "J. Bruce Fields" <bfields@redhat.com>,
        kernel test robot <rong.a.chen@intel.com>
Date:   Wed, 28 Nov 2018 10:20:34 +1100
Cc:     Jeff Layton <jlayton@kernel.org>,
        LKML <linux-kernel@vger.kernel.org>,
        Jeff Layton <jlayton@redhat.com>, lkp@01.org,
        linux-nfs@vger.kernel.org
Subject: Re: [LKP] [fs/locks]  83b381078b:  will-it-scale.per_thread_ops -62.5% regression
In-Reply-To: <20181127174315.GA29963@parsley.fieldses.org>
References: <20181127060102.GF6163@shao2-debian> <20181127174315.GA29963@parsley.fieldses.org>
Message-ID: <87mupup0ot.fsf@notabene.neil.brown.name>
MIME-Version: 1.0
Content-Type: multipart/signed; boundary="=-=-=";
        micalg=pgp-sha256; protocol="application/pgp-signature"
Sender: linux-kernel-owner@vger.kernel.org
Precedence: bulk
List-ID: <linux-kernel.vger.kernel.org>
X-Mailing-List: linux-kernel@vger.kernel.org

--=-=-=
Content-Type: text/plain
Content-Transfer-Encoding: quoted-printable

On Tue, Nov 27 2018, J. Bruce Fields wrote:

> Thanks for the report!

Yes, thanks.  I thought I had replied to the previous report of a similar
problem, but I didn't actually send that email - oops.
Though the test is the same and the regression similar, this is a
different patch.  The previous report identified=20
 fs/locks: allow a lock request to block other requests
this one identifies
  fs/locks: always delete_block after waiting.

Both cause blocked_lock_lock to be taken more often.

In one case is it due to locks_move_blocks().  That can probably be
optimised to skip the lock if list_empty(&fl->fl_blocked_requests).
I'd need to double-check, but I think that is safe to check without
locking.

This one causes locks_delete_blocks() to be called more often.  We now
call it even if no waiting happened at all.  I suspect we can test for
that and avoid it.  I'll have a look.

>
> On Tue, Nov 27, 2018 at 02:01:02PM +0800, kernel test robot wrote:
>> FYI, we noticed a -62.5% regression of will-it-scale.per_thread_ops due =
to commit:
>>=20
>>=20
>> commit: 83b381078b5ecab098ebf6bc9548bb32af1dbf31 ("fs/locks: always dele=
te_block after waiting.")
>> https://git.kernel.org/cgit/linux/kernel/git/jlayton/linux.git locks-next
>>=20
>> in testcase: will-it-scale
>> on test machine: 88 threads Intel(R) Xeon(R) CPU E5-2699 v4 @ 2.20GHz wi=
th 64G memory
>> with following parameters:
>>=20
>> 	nr_task: 16
>> 	mode: thread
>> 	test: lock1
>
> So I guess it's doing this, uncontended file lock/unlock?:
>
> 	https://github.com/antonblanchard/will-it-scale/blob/master/tests/lock1.c
>
> Each thread is repeatedly locking and unlocking a file that is only used
> by that thread.

Thanks for identifying that Bruce.
This would certainly be a case where locks_delete_block() is now being
called when it wasn't before.


>
> By the way, what's the X-axis on these graphs?  (Or the y-axis, for that
> matter?)

A key would help.  I think the X-axis is number-of-threads.  y-axis
might be ops-per-second ??.

Thanks,
NeilBrown


>
> --b.
>
>>                             will-it-scale.per_thread_ops=20=20=20=20=20=
=20=20=20=20=20=20=20=20=20=20=20=20=20=20=20=20=20=20=20
>>=20=20=20=20=20=20=20=20=20=20=20=20=20=20=20=20=20=20=20=20=20=20=20=20=
=20=20=20=20=20=20=20=20=20=20=20=20=20=20=20=20=20=20=20=20=20=20=20=20=20=
=20=20=20=20=20=20=20=20=20=20=20=20=20=20=20=20=20=20=20=20=20=20=20=20=20=
=20=20=20=20=20=20=20
>>   450000 +-+------------------------------------------------------------=
----+=20=20=20
>>          |                                                              =
    |=20=20=20
>>   400000 +-+   +..+..  .+..+..  .+..+..+...+..+..+..   +..        .+..  =
  ..|=20=20=20
>>   350000 +-+ ..      +.       +.                     ..         +.    +.=
.+  |=20=20=20
>>          |  +                                       +     +     :       =
    |=20=20=20
>>   300000 +-+                                              :    :        =
    |=20=20=20
>>   250000 +-+                                               :   :        =
    |=20=20=20
>>          |                                                 :   :        =
    |=20=20=20
>>   200000 +-+                                               :   :        =
    |=20=20=20
>>   150000 +-+                                                : :         =
    |=20=20=20
>>          O  O  O  O  O  O  O  O  O  O  O   O  O  O  O  O  O :O: O  O  O =
 O  O=20=20=20
>>   100000 +-+                                                : :         =
    |=20=20=20
>>    50000 +-+                                                : :         =
    |=20=20=20
>>          |                                                   :          =
    |=20=20=20
>>        0 +-+------------------------------------------------------------=
----+=20=20=20
>>=20=20=20=20=20=20=20=20=20=20=20=20=20=20=20=20=20=20=20=20=20=20=20=20=
=20=20=20=20=20=20=20=20=20=20=20=20=20=20=20=20=20=20=20=20=20=20=20=20=20=
=20=20=20=20=20=20=20=20=20=20=20=20=20=20=20=20=20=20=20=20=20=20=20=20=20=
=20=20=20=20=20=20=20
>>=20=20=20=20=20=20=20=20=20=20=20=20=20=20=20=20=20=20=20=20=20=20=20=20=
=20=20=20=20=20=20=20=20=20=20=20=20=20=20=20=20=20=20=20=20=20=20=20=20=20=
=20=20=20=20=20=20=20=20=20=20=20=20=20=20=20=20=20=20=20=20=20=20=20=20=20=
=20=20=20=20=20=20=20=20=20=20=20=20=20=20=20=20=20=20=20=20=20=20=20=20=20=
=20=20=20=20=20=20=20=20=20=20=20=20=20=20=20=20=20=20=20=20=20=20=20=20=20=
=20=20=20=20=20=20=20=20=20=20=20=20=20=20=20=20=20=20=20=20=20=20=20=20=20=
=20=20=20=20=20=20=20=20=20=20=20=20
>>                                will-it-scale.workload=20=20=20=20=20=20=
=20=20=20=20=20=20=20=20=20=20=20=20=20=20=20=20=20=20=20=20=20
>>=20=20=20=20=20=20=20=20=20=20=20=20=20=20=20=20=20=20=20=20=20=20=20=20=
=20=20=20=20=20=20=20=20=20=20=20=20=20=20=20=20=20=20=20=20=20=20=20=20=20=
=20=20=20=20=20=20=20=20=20=20=20=20=20=20=20=20=20=20=20=20=20=20=20=20=20=
=20=20=20=20=20=20=20
>>   7e+06 +-+-------------------------------------------------------------=
----+=20=20=20
>>         |              +...+..           .+..+..+     +            +..  =
    |=20=20=20
>>   6e+06 +-+   +..+.. ..         .+..+..+.        +   + +         ..     =
  ..|=20=20=20
>>         |   ..      +         +.                  + +   +       +     +.=
.+  |=20=20=20
>>   5e+06 +-++                                       +     +      :       =
    |=20=20=20
>>         |                                                :     :        =
    |=20=20=20
>>   4e+06 +-+                                               :    :        =
    |=20=20=20
>>         |                                                 :    :        =
    |=20=20=20
>>   3e+06 +-+                                                :   :        =
    |=20=20=20
>>         |                                       O     O    :  : O  O    =
    |=20=20=20
>>   2e+06 O-+O  O  O  O  O   O  O  O  O  O  O  O     O     O : O:       O =
 O  O=20=20=20
>>         |                                                   : :         =
    |=20=20=20
>>   1e+06 +-+                                                 : :         =
    |=20=20=20
>>         |                                                    :          =
    |=20=20=20
>>       0 +-+-------------------------------------------------------------=
----+=20=20=20
>>=20=20=20=20=20=20=20=20=20=20=20=20=20=20=20=20=20=20=20=20=20=20=20=20=
=20=20=20=20=20=20=20=20=20=20=20=20=20=20=20=20=20=20=20=20=20=20=20=20=20=
=20=20=20=20=20=20=20=20=20=20=20=20=20=20=20=20=20=20=20=20=20=20=20=20=20=
=20=20=20=20=20=20=20
>>=20=20=20=20=20=20=20=20=20=20=20=20=20=20=20=20=20=20=20=20=20=20=20=20=
=20=20=20=20=20=20=20=20=20=20=20=20=20=20=20=20=20=20=20=20=20=20=20=20=20=
=20=20=20=20=20=20=20=20=20=20=20=20=20=20=20=20=20=20=20=20=20=20=20=20=20=
=20=20=20=20=20=20=20=20=20=20=20=20=20=20=20=20=20=20=20=20=20=20=20=20=20=
=20=20=20=20=20=20=20=20=20=20=20=20=20=20=20=20=20=20=20=20=20=20=20=20=20=
=20=20=20=20=20=20=20=20=20=20=20=20=20=20=20=20=20=20=20=20=20=20=20=20=20=
=20=20=20=20=20=20=20=20=20=20=20=20
>>                            will-it-scale.time.user_time=20=20=20=20=20=
=20=20=20=20=20=20=20=20=20=20=20=20=20=20=20=20=20=20=20=20
>>=20=20=20=20=20=20=20=20=20=20=20=20=20=20=20=20=20=20=20=20=20=20=20=20=
=20=20=20=20=20=20=20=20=20=20=20=20=20=20=20=20=20=20=20=20=20=20=20=20=20=
=20=20=20=20=20=20=20=20=20=20=20=20=20=20=20=20=20=20=20=20=20=20=20=20=20=
=20=20=20=20=20=20=20
>>   250 +-+---------------------------------------------------------------=
----+=20=20=20
>>       |                 .+..               .+..                   +..   =
    |=20=20=20
>>       |..   +...+..  .+.      .+...+..+..+.    +..    +..       ..   .  =
  ..|=20=20=20
>>   200 +-+ ..       +.       +.                    . ..         +      +.=
.+  |=20=20=20
>>       |  +                                         +     +     :        =
    |=20=20=20
>>       |                                                  :    :         =
    |=20=20=20
>>   150 +-+                                                 :   :         =
    |=20=20=20
>>       |                                                   :   :         =
    |=20=20=20
>>   100 +-+                                                 :   :         =
    |=20=20=20
>>       |  O      O                                          : :          =
    |=20=20=20
>>       O     O      O  O  O  O  O   O  O  O  O  O   O  O  O :O: O  O   O =
 O  O=20=20=20
>>    50 +-+                                                  : :          =
    |=20=20=20
>>       |                                                    : :          =
    |=20=20=20
>>       |                                                     :           =
    |=20=20=20
>>     0 +-+---------------------------------------------------------------=
----+=20=20=20
>>=20=20=20=20=20=20=20=20=20=20=20=20=20=20=20=20=20=20=20=20=20=20=20=20=
=20=20=20=20=20=20=20=20=20=20=20=20=20=20=20=20=20=20=20=20=20=20=20=20=20=
=20=20=20=20=20=20=20=20=20=20=20=20=20=20=20=20=20=20=20=20=20=20=20=20=20=
=20=20=20=20=20=20=20
>>=20=20=20=20=20=20=20=20=20=20=20=20=20=20=20=20=20=20=20=20=20=20=20=20=
=20=20=20=20=20=20=20=20=20=20=20=20=20=20=20=20=20=20=20=20=20=20=20=20=20=
=20=20=20=20=20=20=20=20=20=20=20=20=20=20=20=20=20=20=20=20=20=20=20=20=20=
=20=20=20=20=20=20=20=20=20=20=20=20=20=20=20=20=20=20=20=20=20=20=20=20=20=
=20=20=20=20=20=20=20=20=20=20=20=20=20=20=20=20=20=20=20=20=20=20=20=20=20=
=20=20=20=20=20=20=20=20=20=20=20=20=20=20=20=20=20=20=20=20=20=20=20=20=20=
=20=20=20=20=20=20=20=20=20=20=20=20
>>                           will-it-scale.time.system_time=20=20=20=20=20=
=20=20=20=20=20=20=20=20=20=20=20=20=20=20=20=20=20=20=20
>>=20=20=20=20=20=20=20=20=20=20=20=20=20=20=20=20=20=20=20=20=20=20=20=20=
=20=20=20=20=20=20=20=20=20=20=20=20=20=20=20=20=20=20=20=20=20=20=20=20=20=
=20=20=20=20=20=20=20=20=20=20=20=20=20=20=20=20=20=20=20=20=20=20=20=20=20=
=20=20=20=20=20=20=20
>>   5000 +-+--------------------------------------------------------------=
----+=20=20=20
>>   4500 O-+O..O..O...O..O..O..O..O..O..O...O..O..O..O..O..O  O  O...O..O.=
.O..O=20=20=20
>>        |                                                 :     :        =
    |=20=20=20
>>   4000 +-+                                               :     :        =
    |=20=20=20
>>   3500 +-+                                                :   :         =
    |=20=20=20
>>        |                                                  :   :         =
    |=20=20=20
>>   3000 +-+                                                :   :         =
    |=20=20=20
>>   2500 +-+                                                :   :         =
    |=20=20=20
>>   2000 +-+                                                 : :          =
    |=20=20=20
>>        |                                                   : :          =
    |=20=20=20
>>   1500 +-+                                                 : :          =
    |=20=20=20
>>   1000 +-+                                                 : :          =
    |=20=20=20
>>        |                                                    :           =
    |=20=20=20
>>    500 +-+                                                  :           =
    |=20=20=20
>>      0 +-+--------------------------------------------------------------=
----+=20=20=20
>>=20=20=20=20=20=20=20=20=20=20=20=20=20=20=20=20=20=20=20=20=20=20=20=20=
=20=20=20=20=20=20=20=20=20=20=20=20=20=20=20=20=20=20=20=20=20=20=20=20=20=
=20=20=20=20=20=20=20=20=20=20=20=20=20=20=20=20=20=20=20=20=20=20=20=20=20=
=20=20=20=20=20=20=20
>>=20=20=20=20=20=20=20=20=20=20=20=20=20=20=20=20=20=20=20=20=20=20=20=20=
=20=20=20=20=20=20=20=20=20=20=20=20=20=20=20=20=20=20=20=20=20=20=20=20=20=
=20=20=20=20=20=20=20=20=20=20=20=20=20=20=20=20=20=20=20=20=20=20=20=20=20=
=20=20=20=20=20=20=20
>> [*] bisect-good sample
>> [O] bisect-bad  sample

--=-=-=
Content-Type: application/pgp-signature; name="signature.asc"

-----BEGIN PGP SIGNATURE-----

iQIzBAEBCAAdFiEEG8Yp69OQ2HB7X0l6Oeye3VZigbkFAlv90UIACgkQOeye3VZi
gbnQiw//dlZQTtoolVqtfDTeStauX3I/+zJGPFjBd80aha39eeFhrDrYnPK0FEXA
HihxxOMWFfDbGccXXmeDKcuGu4+TZbfXquGOVwKFz9cF9OKbUbQj/6vdk7MMBtEK
q7/SBJ+CvEittcOXaC5GWbszTnO9PH01/9ebIkNOOyo/Lx0lALqmgmGCRUTLmoyS
/pHqsVULokqfBycpY4qvEuH8vsfNHxEYAm4nHEkDfBzawsNWGXWGwc/wsdyzd/QD
FLIA9M9aqq6grzJE/4FThu89kCe1jNiVce/m3FLo9MjvK2xnqFQ8Y7auslMpzz9U
m827k+EhQOF1erkz9LCOeIT4LPOTUxF/JwiazEWts4UlnB3yacPnnji2ZSe+qi2r
11+NRCzQLPLym3rErox0BKBDYIfFCyASMM8xY/DTYAt/x1IVwaloImhyHTRTCrTO
8FqcXbU8Q0HTCmJxVoe98/IRIdQgDZm/u6VxAmLU5P6mq78NtVBRMtSi2Mt1RqXB
WHyv4mmfiZUCbnjFk1+wNbqL2XQndBGrk/R29PAEF7zA7xvabvFiTmUTyH3SFtDB
5xUiZCRbbHPPDOgUC7NUzJmVaHCuYeRKiVXL5ruqXkNvyFjcgLUqwDWm2QwIc4ex
IPX3eI3sR5IKIRgvGkPiJ7x/GJop5BtxRTuA6haLHqXBtFL/ph8=
=jf//
-----END PGP SIGNATURE-----
--=-=-=--