Received-SPF: pass (google.com: domain of linux-kernel-owner@vger.kernel.org designates 2620:137:e000::1:20 as permitted sender) client-ip=2620:137:e000::1:20;
Date:   Tue, 22 Nov 2022 13:55:53 +0800
From:   Aaron Lu <aaron.lu@intel.com>
To:     Josh Don <joshdon@google.com>
CC:     Peter Zijlstra <peterz@infradead.org>,
        Ingo Molnar <mingo@redhat.com>,
        Juri Lelli <juri.lelli@redhat.com>,
        Vincent Guittot <vincent.guittot@linaro.org>,
        Dietmar Eggemann <dietmar.eggemann@arm.com>,
        Steven Rostedt <rostedt@goodmis.org>,
        Ben Segall <bsegall@google.com>,
        "Mel Gorman" <mgorman@suse.de>,
        Daniel Bristot de Oliveira <bristot@redhat.com>,
        Valentin Schneider <vschneid@redhat.com>,
        <linux-kernel@vger.kernel.org>, Tejun Heo <tj@kernel.org>,
        Michal =?iso-8859-1?Q?Koutn=FD?= <mkoutny@suse.com>,
        Christian Brauner <brauner@kernel.org>,
        Zefan Li <lizefan.x@bytedance.com>
Subject: Re: [PATCH v3] sched: async unthrottling for cfs bandwidth
Message-ID: <Y3xkaaioFNVQtN8q@ziqianlu-desk2>
References: <20221117005418.3499691-1-joshdon@google.com>
 <Y3d+1a9AEnWaxFwq@hirez.programming.kicks-ass.net>
 <CABk29NtSmXVCvkdpymeam7AYmXhZy2JLYLPFTdKpk5g6AN1-zg@mail.gmail.com>
 <Y3twWYeAW6U2/D92@hirez.programming.kicks-ass.net>
 <CABk29Nsxg2AvUzsvn4kLpE8rzWRJNt-EKwFi1HcK7c6uY_oUUA@mail.gmail.com>
Content-Type: text/plain; charset="us-ascii"
Content-Disposition: inline
In-Reply-To: <CABk29Nsxg2AvUzsvn4kLpE8rzWRJNt-EKwFi1HcK7c6uY_oUUA@mail.gmail.com>
MIME-Version: 1.0
X-MS-Exchange-AntiSpam-MessageData-ChunkCount: 1
X-MS-Exchange-AntiSpam-MessageData-0: =?us-ascii?Q?/yLbo6dUcrQMpi2ZdJir3FMtUSBCvvXK7qBVMEYbFUWIhfzQ2JFs2u0AV+BS?=
 =?us-ascii?Q?570grBEkPRx7UmoFMtgbwkbaRDbroIg2BGLefnQpXssLgN9oRJUM2wD0DR00?=
 =?us-ascii?Q?mJGoRWaJoqlQN1J9IwFtsEuPYErPfRcYGOakb3C0xsKOboyxG4gTkJmyWgX/?=
 =?us-ascii?Q?G3ZC0NxmnCdceHI0ilkDhN9or/yOZjF57uYzN3WG4SDFA14tj3YE/dP2bc8F?=
 =?us-ascii?Q?qUN3keLDXD0PpboT13RyiVbNGzriQgB66sTTvZS8VfS1inZYaneHfe6Su6dz?=
 =?us-ascii?Q?5krTP5FcRcNDUEr9nmLJTYock6oyU41lrcha2r6L6bDEGpGSMXrveBzJSH31?=
 =?us-ascii?Q?DbnWeGRajg3tFKslJAnFw3WdF+vdeJ2v8J0F62LYhk2zerPJ3Su3oFRXScjd?=
 =?us-ascii?Q?XlPXMEAZd6ijZ1/abIbnGRXMtjSEiPyw+koUxx6Xu+T8RerivOZY31KOI3Y0?=
 =?us-ascii?Q?XIJpgJzKlX1cXAi61LpHjdxq8a42BIpT9cAVuf0h4ucIwL59y8hbXCna4Ktb?=
 =?us-ascii?Q?/dZjYZWMfLeCvrGwctkkVY/9GaewN9V1++/0b+gCPq9VTzh9KM9mcLuoP/zV?=
 =?us-ascii?Q?KexZPA81sjfhza8KWH8snsRLRIDbAb/EfsFkvGp4gOr0MQiImjUB6CuxQrlr?=
 =?us-ascii?Q?L7iNSmxmqqX87/GY03s7rM5CsEHBTiIidUwUK4DsnZPHmYojMaKHNBnS3YwW?=
 =?us-ascii?Q?sjaFn15U8lk5o6oiGdCEav3kNRO8cEbUwPVeLyszZGpFtw5/YgP4dS5lDpID?=
 =?us-ascii?Q?AU2B/gIsx5i8IgjlwdSalDFpZ24bxXM8krQdkAMWNQ0ljcIvVR/qjwFv4xRI?=
 =?us-ascii?Q?j6ZotyTUy3M+RRJbG7YIw6uZOux65msyIYK0JmSBo99fuPXV4pILL3Em8hUq?=
 =?us-ascii?Q?r/7tEbsUQ9KxffS0UcuKjTln2YPyUlCodQImfTFWY1FKPfqdnhUcNkKeP3HO?=
 =?us-ascii?Q?DEMkagw8PjcXn6KdGiw0ssffbo/VZm2RdnHna2HbyciMTQaTVoMfxIyzHzTT?=
 =?us-ascii?Q?MZur5t91F9GWSt37kxrRUbjP6FuaFTeNJZAdCD1hoOL9XW1iF2ZQWXcyk8mt?=
 =?us-ascii?Q?W0NvZAD8kFZLLCHJgHPQoUoT47odAWXDmmh4F5TZHb14bflo58Bar7xvJffG?=
 =?us-ascii?Q?apFQPmm3qJ/oQkhPHLJNsZfyquBQFNsCywjX5Cjpwrv+sa59VXYUPnMImssk?=
 =?us-ascii?Q?XjIgNtm6drx3PFudaRtmSIZAeV7b9E7ziiKl2V+/FJgi757g325leGKb2Vaa?=
 =?us-ascii?Q?wLZDuAommGXyUy75G4T9IgzlruTwTT6q0jWY419epPkVvB+a05J1BvGFE/px?=
 =?us-ascii?Q?RZsYmeNTts3diYrsJH06GytYlr88kpMtwiEqDLALYEwY6p0g16XBjGKGllcC?=
 =?us-ascii?Q?O34o8dBZvd6gWSNIeFkqLNNbA+7OSHF/jvnmVUpVlnS/17x+R/oi8BV9AXvV?=
 =?us-ascii?Q?3gSB8xlNipTr0tmAeZMQ0lMS85VXoIPvQ90AqlKDksv8h3Ic+KdsXUD1fTbJ?=
 =?us-ascii?Q?rW6axIhitXE8CGU1ph9MSW1d0OR0zWbWPiqRDCmMbdVj+Qcke4/wMqW/q1S9?=
 =?us-ascii?Q?3JgFRjXAm9Ua1sBJi0Y0KHb5z7eD9h6fj8HwfWtC?=
X-MS-Exchange-CrossTenant-Network-Message-Id: 2861938e-334a-4bf2-3039-08dacc4e4014
X-MS-Exchange-CrossTenant-AuthSource: BYAPR11MB3062.namprd11.prod.outlook.com
X-MS-Exchange-CrossTenant-AuthAs: Internal
X-MS-Exchange-CrossTenant-OriginalArrivalTime: 22 Nov 2022 05:56:08.4207
 (UTC)
X-MS-Exchange-CrossTenant-FromEntityHeader: Hosted
X-MS-Exchange-CrossTenant-Id: 46c98d88-e344-4ed4-8496-4ed7712e255d
X-MS-Exchange-CrossTenant-MailboxType: HOSTED
X-MS-Exchange-CrossTenant-UserPrincipalName: iNdewDzTcS4CQrCn0R9CZEuNapxhdzQ+1yhBYGirGDm/GpkBD7n4zcRV8jnjtw1cdUxOxryJoo9PkVdPIycGiA==
X-MS-Exchange-Transport-CrossTenantHeadersStamped: PH0PR11MB4808
X-OriginatorOrg: intel.com
X-Spam-Status: No, score=-7.1 required=5.0 tests=BAYES_00,DKIMWL_WL_HIGH,
        DKIM_SIGNED,DKIM_VALID,DKIM_VALID_AU,DKIM_VALID_EF,RCVD_IN_DNSWL_HI,
        SPF_HELO_NONE,SPF_NONE autolearn=ham autolearn_force=no version=3.4.6
X-Spam-Checker-Version: SpamAssassin 3.4.6 (2021-04-09) on
        lindbergh.monkeyblade.net
Precedence: bulk
List-ID: <linux-kernel.vger.kernel.org>
X-Mailing-List: linux-kernel@vger.kernel.org

On Mon, Nov 21, 2022 at 11:31:20AM -0800, Josh Don wrote:
> > Please test the final version as found here:
> >
> >   https://git.kernel.org/pub/scm/linux/kernel/git/peterz/queue.git/commit/?h=sched/core&id=4e3c1b7b489e218dfa576cd6af0680b975b8743e
> 
> Thanks Peter, the patch looks good to me and continues to work
> correctly. Note that I needed to make two edits to get it to build
> though on sched/core:
> 
> - local_cfq_rq doesn't match variable name used in function (local_unthrottle)
> - rq_lock_irqrestore -> rq_unlock_irqrestore

I also tested it on a desktop bare metal with 800 cgroups under 2
cgroups that have quota set. Before this patch, the majority of durations
for sched_cfs_period_timer() are in the range of 512us-1ms, with outliers
in the range of 1ms-4ms; after this patch, the majority are in the range
of 128us-512us with no outliers above 512us.