Received: by 2002:a05:6a10:206:0:0:0:0 with SMTP id 6csp286370pxj; Thu, 3 Jun 2021 06:44:21 -0700 (PDT) X-Google-Smtp-Source: ABdhPJzueRY4lyY8rFomNVYYGRxnUYtUYOYHdKilOo2Ri/qAXeYr/Yp7aFyFFv1vrB7cKP3TabAD X-Received: by 2002:a50:fc84:: with SMTP id f4mr24646141edq.252.1622727860809; Thu, 03 Jun 2021 06:44:20 -0700 (PDT) ARC-Seal: i=1; a=rsa-sha256; t=1622727860; cv=none; d=google.com; s=arc-20160816; b=HxLGLTUtRMOYsgBfUPHLxQ9UFPJaxv1h0Bkbj/8nASiBiunZTqazcQEG9p4TYBVxcr UbIhxU0E9qfeRXN+9K+9QbN4p4V2lhZyeTseC1hChSiV4ZMgiNGWFx8OtzaYkspQILXO 4fSS+N6DJy6NujtbA9Wnvt5yKkHatWimy5KnOJ4blpD1B+tUg6WCll9dU0ZzNzY92nxv 3JBlnIhY1qwWzwW6QGGOEZiBR5uCfYGIXsevkiIpERnWFIqN45Mvc0m5nB8CSqfiCZAZ g/h1MSySggNM6YNwzJslreU2PfJMj98biDWa+/Z++c8tvPs2JBCEvWLrq0AYKNwNI7++ LoxA== ARC-Message-Signature: i=1; a=rsa-sha256; c=relaxed/relaxed; d=google.com; s=arc-20160816; h=list-id:precedence:cc:to:subject:message-id:date:from:in-reply-to :references:mime-version:dkim-signature; bh=QTp3vmqt4JhYvd2bJK89SITP9H+81cHGdf+Hvc3N0js=; b=ts5BFGTDcpqwEXmd55sfKASWMTh2fc3YrfXoxQR9Uamdf2hhcb/zdgpvKfP/hbQNwK L2cxAvtp8X0rCipy/FjCV3pedbRo6e9aVzkw81HtQNrIQ0733JTeOt/IXshGBUoPyEdm Xo9t0GrKF+XLwka/YwvlxQmq4hzAQTn9EmVz+XIiFiFnDr4ndn+sHgS6mXAXzC/coz+8 df9WFPO/IBgyu20Td8TEmFz/4/TrNQ58BdHBFZZB6JN7i/rGnTam6toFIdPu/9zf0bet xlS3bL0cNEKxKjwN0ad4M07tlJC3i1WHpkcdezUSaax4sfhFLlhGK89VIsCdBzJt43Fw 3yTA== ARC-Authentication-Results: i=1; mx.google.com; dkim=pass header.i=@linaro.org header.s=google header.b=P11xYanS; spf=pass (google.com: domain of linux-kernel-owner@vger.kernel.org designates 23.128.96.18 as permitted sender) smtp.mailfrom=linux-kernel-owner@vger.kernel.org; dmarc=pass (p=NONE sp=NONE dis=NONE) header.from=linaro.org Return-Path: Received: from vger.kernel.org (vger.kernel.org. [23.128.96.18]) by mx.google.com with ESMTP id x2si2340510ejs.442.2021.06.03.06.43.58; Thu, 03 Jun 2021 06:44:20 -0700 (PDT) Received-SPF: pass (google.com: domain of linux-kernel-owner@vger.kernel.org designates 23.128.96.18 as permitted sender) client-ip=23.128.96.18; Authentication-Results: mx.google.com; dkim=pass header.i=@linaro.org header.s=google header.b=P11xYanS; spf=pass (google.com: domain of linux-kernel-owner@vger.kernel.org designates 23.128.96.18 as permitted sender) smtp.mailfrom=linux-kernel-owner@vger.kernel.org; dmarc=pass (p=NONE sp=NONE dis=NONE) header.from=linaro.org Received: (majordomo@vger.kernel.org) by vger.kernel.org via listexpand id S230114AbhFCNmt (ORCPT + 99 others); Thu, 3 Jun 2021 09:42:49 -0400 Received: from lindbergh.monkeyblade.net ([23.128.96.19]:33158 "EHLO lindbergh.monkeyblade.net" rhost-flags-OK-OK-OK-OK) by vger.kernel.org with ESMTP id S229957AbhFCNms (ORCPT ); Thu, 3 Jun 2021 09:42:48 -0400 Received: from mail-lj1-x233.google.com (mail-lj1-x233.google.com [IPv6:2a00:1450:4864:20::233]) by lindbergh.monkeyblade.net (Postfix) with ESMTPS id 24866C06174A for ; Thu, 3 Jun 2021 06:41:04 -0700 (PDT) Received: by mail-lj1-x233.google.com with SMTP id d2so3022993ljj.11 for ; Thu, 03 Jun 2021 06:41:04 -0700 (PDT) DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=linaro.org; s=google; h=mime-version:references:in-reply-to:from:date:message-id:subject:to :cc; bh=QTp3vmqt4JhYvd2bJK89SITP9H+81cHGdf+Hvc3N0js=; b=P11xYanSpOJL2lIsQLEOdAs2S9eaHOU+2Q8yfObrnHB2QY+uzvNxFgmOhSf+U/tstY EOoJOft+DZa62Leh4t2CadOV/3k0dCoZ5uC0zW9Rf+uKfklUQ7a3EksL2fSgKLD2fMzj ajD3i6vU5D5YjO0fXoUAe8CRf1yde12OZlMBd2rVUNnI0MeYYuUYRTAMRcY4Yv2iioTG 1+Wz/j3lyFM3WYEuVil/ZCMcFXoJw1/yo28xa++blIVd8hZxHGpuIIKuwuzPXOn7x0gy PQtKZImXzjXP50mbDUXG36+PCuIy9/jg6x2FBouv8gs1/gXbWQuo6uYNkvG3DP00K9hL xdwA== X-Google-DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=1e100.net; s=20161025; h=x-gm-message-state:mime-version:references:in-reply-to:from:date :message-id:subject:to:cc; bh=QTp3vmqt4JhYvd2bJK89SITP9H+81cHGdf+Hvc3N0js=; b=sj0kCOKS5qx9zXjvCEbMAoZxt2orH3vK9Wh/XtSl8yBHsc9r2aLnip35wyZpD10qx4 EhaauKfAlz+/08HihuqsNX4Zt+xt/C6ytAq6FmHyvglwuLlxwbZEA3oiIaP6sa6Fhhk9 nWm6rj0miXfXxjEokWmWqMFc5KkHTLw567oRt0JcQDUkqF8NE/64F0efX5weoylyUmSc QVUX1/1mZ5lwGzkunM/glwbWbY2aoolhd1N1Q/gJJwIQxmUTQ8YAoxdxqfrezV7eJhKy IH9vFP8HwQR9X+NMfwVGjt9OXvQfRbUJYneBWjf624xjTuIyxaJFByPwtZVVETKL+GT6 W63Q== X-Gm-Message-State: AOAM533EdZyu8w/O+qOzdPBeWymvONad+2mPGxoo9ZCGCWpe2dThTS/K 1fXYOeeJy8Wf10C/VZCCT/sDU9DBrFPhyDmB+DHUjA== X-Received: by 2002:a2e:9211:: with SMTP id k17mr30170075ljg.284.1622727662425; Thu, 03 Jun 2021 06:41:02 -0700 (PDT) MIME-Version: 1.0 References: <20210603113847.163512-1-odin@uged.al> In-Reply-To: From: Vincent Guittot Date: Thu, 3 Jun 2021 15:40:51 +0200 Message-ID: Subject: Re: [PATCH v2] sched/fair: Correctly insert cfs_rq's to list on unthrottle To: Odin Ugedal Cc: Ingo Molnar , Peter Zijlstra , Juri Lelli , Dietmar Eggemann , Steven Rostedt , Ben Segall , Mel Gorman , Daniel Bristot de Oliveira , "open list:CONTROL GROUP (CGROUP)" , linux-kernel Content-Type: text/plain; charset="UTF-8" Precedence: bulk List-ID: X-Mailing-List: linux-kernel@vger.kernel.org On Thu, 3 Jun 2021 at 15:13, Odin Ugedal wrote: > > Hi, > > > Out of curiosity, why did you decide to use > > cfs_rq->tg_load_avg_contrib instead of !cfs_rq_is_decayed(cfs_rq) > > which is used to delete the cfs_rq from the list when updating blocked > > load ? > > Well, the main reason was that it is currently (without the other in > flight patches) not safe to just use "cfs_rq_is_decayed" directly, > since that could result in > a situation where tg_load_avg_contrib!=0 while > cfs_rq_is_decayed()==true. I guess we can use cfs_rq_is_decayed() if > you prefer that, > and all the other PELT patches are merged. (This was initially why I > thought a new field was a simpler and more elegant solution to make > sure we book-keep correctly, > but when the PELT stuff is fixed properly, that should be no real > issue as long it works as we expect). If it's only a matter of waiting other PELT patches to be merged, we should use cfs_rq_is_decayed(). cfs_rq->tg_load_avg_contrib is used to reduce contention on tg->load_avg but it is outside the scope of PELT and the update of blocked load so we should avoid using it there > > I was also thinking about the cfs_rq->nr_running part; is there a > chance of a situation where a cfs_rq->nr_running==1 and it has no > load, resulting in it being decayed and > removed from the list in "__update_blocked_fair"? I have not looked > properly at it, but just wondering if that is actually possible.. > > > Also, out of curiosity, are there some implications of a situation > where tg_load_avg_contrib=0 while *_load!=0, or would that not cause do you mean all cfs_rq->avg.load_avg with *_load ? > fairness issues? if load_avg!=0, we will update it periodically and sync tg_load_avg_contrib with the former. So it's not a problem. The other way was a problem because we stop updating load_avg and tg_load_avg_contrib when load_avg/load_sum is null so the tg_load_avg_contrib is stalled with a possibly very old value > > Thanks > Odin