Return-Path: Received: (majordomo@vger.kernel.org) by vger.kernel.org via listexpand id S1751716AbdF3AtR (ORCPT ); Thu, 29 Jun 2017 20:49:17 -0400 Received: from mail-qk0-f182.google.com ([209.85.220.182]:34606 "EHLO mail-qk0-f182.google.com" rhost-flags-OK-OK-OK-OK) by vger.kernel.org with ESMTP id S1751625AbdF3AtQ (ORCPT ); Thu, 29 Jun 2017 20:49:16 -0400 Date: Thu, 29 Jun 2017 20:49:13 -0400 From: Josef Bacik To: Joel Fernandes Cc: Mike Galbraith , Peter Zijlstra , LKML , Juri Lelli , Dietmar Eggemann , Patrick Bellasi , Brendan Jackman , Chris Redpath Subject: Re: wake_wide mechanism clarification Message-ID: <20170630004912.GA2457@destiny> References: MIME-Version: 1.0 Content-Type: text/plain; charset=us-ascii Content-Disposition: inline In-Reply-To: User-Agent: Mutt/1.8.0 (2017-02-23) Sender: linux-kernel-owner@vger.kernel.org List-ID: X-Mailing-List: linux-kernel@vger.kernel.org Content-Length: 2602 Lines: 61 On Thu, Jun 29, 2017 at 05:19:14PM -0700, Joel Fernandes wrote: > Dear Mike, > > I wanted your kind help to understand your patch "sched: beef up > wake_wide()"[1] which is a modification to the original patch from > Michael Wang [2]. > > In particular, I didn't following the following comment: > " to shared cache, we look for a minimum 'flip' frequency of llc_size > in one partner, and a factor of lls_size higher frequency in the > other." > > Why are wanting the master's flip frequency to be higher than the > slaves by the factor? (Responding from my personal email as my work email is outlook shit and impossible to use) Because we are trying to detect the case that the master is waking many different processes, and the 'slave' processes are only waking up the master/some other specific processes to determine if we don't care about cache locality. > > The code here is written as: > > if (slave < factor || master < slave * factor) > return 0; > > However I think we should just do (with my current and probably wrong > understanding): > > if (slave < factor || master < factor) > return 0; > Actually I think both are wrong, but I need Mike to weigh in. In my example above we'd return 0, because the 'producer' will definitely have a wakee_flip of ridiculous values, but the 'consumer' could essentially have a wakee_flip of 1, just the master to tell it that it's done. I _suppose_ in practice you have a lock or something so the wakee_flip isn't going to be strictly 1, but some significantly lower value than master. I'm skeptical of the slave < factor test, I think it's too high of a bar in the case where cache locality doesn't really matter, but the master < slave * factor makes sense, as slave is going to be orders of magnitude lower than master. > Basically, I didn't follow why we multiply the slave's flips with > llc_size. That makes it sound like the master has to have way more > flips than the slave to return 0 from wake_wide. Could you maybe give > an example to clarify? Thanks a lot for your help, > It may be worth to try with schedbench and trace it to see how this turns out in practice, as that's the workload that generated all this discussion before. I imagine generally speaking this works out properly. The small regression I reported before was at low RPS, so we wouldn't be waking up as many tasks as often, so we would be returning 0 from wake_wide() and we'd get screwed. This is where I think possibly dropping the slave < factor part of the test would address that, but I'd have to trace it to say for sure. Thanks, Josef