Received-SPF: pass (google.com: best guess record for domain of linux-kernel-owner@vger.kernel.org designates 209.132.180.67 as permitted sender) client-ip=209.132.180.67;
MIME-Version: 1.0
References: <20181105050813.GP24195@shao2-debian> <CAHk-=wi8BNNTp+Xt4ROz1TqGgFT-c6-Hmmc9yW0B6p2FTFwJTg@mail.gmail.com>
 <f901e346-6203-371d-a39a-ed9231187570@suse.cz>
In-Reply-To: <f901e346-6203-371d-a39a-ed9231187570@suse.cz>
From:   Linus Torvalds <torvalds@linux-foundation.org>
Date:   Mon, 5 Nov 2018 14:14:33 -0800
Message-ID: <CAHk-=wjZ1C2iL79jOuC9ysvX2oRpUjoqXirvY0NRuLC0eQ8nbg@mail.gmail.com>
Subject: Re: [LKP] [mm] 9bc8039e71: will-it-scale.per_thread_ops -64.1% regression
To:     vbabka@suse.cz, Davidlohr Bueso <dave@stgolabs.net>,
        Waiman Long <longman@redhat.com>
Cc:     rong.a.chen@intel.com, yang.shi@linux.alibaba.com,
        kirill.shutemov@linux.intel.com, mhocko@kernel.org,
        Matthew Wilcox <willy@infradead.org>,
        ldufour@linux.vnet.ibm.com, Colin King <colin.king@canonical.com>,
        Andrew Morton <akpm@linux-foundation.org>,
        Linux Kernel Mailing List <linux-kernel@vger.kernel.org>,
        lkp@01.org
Content-Type: text/plain; charset="UTF-8"
Sender: linux-kernel-owner@vger.kernel.org
Precedence: bulk

On Mon, Nov 5, 2018 at 12:12 PM Vlastimil Babka <vbabka@suse.cz> wrote:
>
> I didn't spot an obvious mistake in the patch itself, so it looks
> like some bad interaction between scheduler and the mmap downgrade?

I'm thinking it's RWSEM_SPIN_ON_OWNER that ends up being confused by
the downgrade.

It looks like the benchmark used to be basically CPU-bound, at about
800% CPU, and now it's somewhere in the 200% CPU region:

                  will-it-scale.time.percent_of_cpu_this_job_got

  800 +-+-------------------------------------------------------------------+
      |.+.+.+.+.+.+.+.  .+.+.+.+.+.+.+.+.+.+.+.+.+.+.+.+.+..+.+.+.+. .+.+.+.|
  700 +-+             +.                                            +       |
      |                                                                     |
  600 +-+                                                                   |
      |                                                                     |
  500 +-+                                                                   |
      |                                                                     |
  400 +-+                                                                   |
      |                                                                     |
  300 +-+                                                                   |
      |                                                                     |
  200 O-O O O O O                O                                          |
      |           O O O  O O O O   O O O O O O O O O O O                    |
  100 +-+-------------------------------------------------------------------+

which sounds like the downgrade really messes with the "spin waiting
for lock" logic.

I'm thinking it's the "wake up waiter" logic that has some bad
interaction with spinning, and breaks that whole optimization.

Adding Waiman and Davidlohr to the participants, because they seem to
be the obvious experts in this area.

                            Linus