Return-Path: Received: (majordomo@vger.kernel.org) by vger.kernel.org via listexpand id S1751417AbdGQNY4 (ORCPT ); Mon, 17 Jul 2017 09:24:56 -0400 Received: from mga11.intel.com ([192.55.52.93]:64654 "EHLO mga11.intel.com" rhost-flags-OK-OK-OK-OK) by vger.kernel.org with ESMTP id S1751289AbdGQNYz (ORCPT ); Mon, 17 Jul 2017 09:24:55 -0400 X-ExtLoop1: 1 X-IronPort-AV: E=Sophos;i="5.40,374,1496127600"; d="scan'208";a="126028210" Subject: Re: [RFC PATCH v1 00/11] Create fast idle path for short idle periods To: paulmck@linux.vnet.ibm.com Cc: Peter Zijlstra , Andi Kleen , Frederic Weisbecker , Christoph Lameter , Aubrey Li , tglx@linutronix.de, len.brown@intel.com, rjw@rjwysocki.net, tim.c.chen@linux.intel.com, arjan@linux.intel.com, yang.zhang.wz@gmail.com, x86@kernel.org, linux-kernel@vger.kernel.org References: <20170711160926.GA18805@lerouge> <20170711163422.etydkhhtgfthpfi5@hirez.programming.kicks-ass.net> <496d4921-5768-cd1e-654b-38630b7d2e13@linux.intel.com> <20170712083410.ualmvnvzoohyami5@hirez.programming.kicks-ass.net> <20170712213240.GE3441@tassilo.jf.intel.com> <20170713083649.febfflfl5hafkko5@hirez.programming.kicks-ass.net> <16e12e23-6b28-f174-7c4b-4d719225cd3b@linux.intel.com> <20170713145311.z4zxlyd2dospeoqg@hirez.programming.kicks-ass.net> <20170713152032.GM2393@linux.vnet.ibm.com> <58a312d3-bd3d-9427-109b-5d81a5fd7c76@linux.intel.com> <20170714040553.GR2393@linux.vnet.ibm.com> From: "Li, Aubrey" Message-ID: Date: Mon, 17 Jul 2017 21:24:51 +0800 User-Agent: Mozilla/5.0 (Windows NT 10.0; WOW64; rv:52.0) Gecko/20100101 Thunderbird/52.2.1 MIME-Version: 1.0 In-Reply-To: <20170714040553.GR2393@linux.vnet.ibm.com> Content-Type: text/plain; charset=utf-8 Content-Language: en-US Content-Transfer-Encoding: 7bit Sender: linux-kernel-owner@vger.kernel.org List-ID: X-Mailing-List: linux-kernel@vger.kernel.org Content-Length: 5010 Lines: 137 On 2017/7/14 12:05, Paul E. McKenney wrote: > > More specifically: rcu_needs_cpu(), rcu_prepare_for_idle(), > rcu_cleanup_after_idle(), rcu_eqs_enter(), rcu_eqs_enter_common(), > rcu_dynticks_eqs_enter(), do_nocb_deferred_wakeup(), > rcu_dynticks_task_enter(), rcu_eqs_exit(), rcu_eqs_exit_common(), > rcu_dynticks_task_exit(), rcu_dynticks_eqs_exit(). > > The first three (rcu_needs_cpu(), rcu_prepare_for_idle(), and > rcu_cleanup_after_idle()) should not be significant unless you have > CONFIG_RCU_FAST_NO_HZ=y. If you do, it would be interesting to learn > how often invoke_rcu_core() is invoked from rcu_prepare_for_idle() > and rcu_cleanup_after_idle(), as this can raise softirq. Also > rcu_accelerate_cbs() and rcu_try_advance_all_cbs(). > > Knowing which of these is causing the most trouble might help me > reduce the overhead in the current idle path. I measured two cases, nothing notable found. The one is CONFIG_NO_HZ_IDLE=y, so the following function is just empty. rcu_prepare_for_idle(): NULL rcu_cleanup_after_idle(): NULL do_nocb_deferred_wakeup(): NULL rcu_dynticks_task_enter(): NULL rcu_dynticks_task_exit(): NULL And the following functions are traced separately, for each function I traced 3 times by intel_PT, for each time the sampling period is 1-second. num means the times the function is invoked in 1-second. (min, max, avg) is the function duration, the unit is nano-second. rcu_needs_cpu(): 1) num: 6110 min: 3 max: 564 avg: 17.0 2) num: 16535 min: 3 max: 683 avg: 18.0 3) num: 8815 min: 3 max: 394 avg: 20.0 rcu_eqs_enter(): 1) num: 7956 min: 17 max: 656 avg: 32.0 2) num: 9170 min: 17 max: 1075 avg: 35.0 3) num: 8604 min: 17 max: 859 avg: 29.0 rcu_eqs_enter_common(): 1) num: 14676 min: 15 max: 620 avg: 28.0 2) num: 11180 min: 15 max: 795 avg: 30.0 3) num: 11484 min: 15 max: 725 avg: 29.0 rcu_dynticks_eqs_enter(): 1) num: 11035 min: 10 max: 580 avg: 17.0 2) num: 15518 min: 10 max: 456 avg: 16.0 3) num: 15320 min: 10 max: 454 avg: 19.0 rcu_eqs_exit(): 1) num: 11080 min: 14 max: 893 avg: 23.0 2) num: 13526 min: 14 max: 640 avg: 23.0 3) num: 12534 min: 14 max: 630 avg: 22.0 rcu_eqs_exit_common(): 1) num: 18002 min: 12 max: 553 avg: 17.0 2) num: 10570 min: 11 max: 485 avg: 17.0 3) num: 13628 min: 11 max: 567 avg: 16.0 rcu_dynticks_eqs_exit(): 1) num: 11195 min: 11 max: 436 avg: 16.0 2) num: 11808 min: 10 max: 506 avg: 16.0 3) num: 8132 min: 10 max: 546 avg: 15.0 ============================================================================== The other case is CONFIG_NO_HZ_FULL, I also enabled the required config to make all the functions not empty. rcu_needs_cpu(): 1) num: 8530 min: 5 max: 770 avg: 13.0 2) num: 9965 min: 5 max: 518 avg: 12.0 3) num: 12503 min: 5 max: 755 avg: 16.0 rcu_prepare_for_idle(): 1) num: 11662 min: 5 max: 684 avg: 9.0 2) num: 15294 min: 5 max: 676 avg: 9.0 3) num: 14332 min: 5 max: 524 avg: 9.0 rcu_cleanup_after_idle(): 1) num: 13584 min: 4 max: 657 avg: 6.0 2) num: 9102 min: 4 max: 529 avg: 5.0 3) num: 10648 min: 4 max: 471 avg: 6.0 rcu_eqs_enter(): 1) num: 14222 min: 26 max: 745 avg: 54.0 2) num: 12502 min: 26 max: 650 avg: 53.0 3) num: 11834 min: 26 max: 863 avg: 52.0 rcu_eqs_enter_common(): 1) num: 16792 min: 24 max: 973 avg: 43.0 2) num: 19755 min: 24 max: 898 avg: 45.0 3) num: 8167 min: 24 max: 722 avg: 42.0 rcu_dynticks_eqs_enter(): 1) num: 11605 min: 10 max: 532 avg: 14.0 2) num: 10438 min: 9 max: 554 avg: 14.0 3) num: 19816 min: 9 max: 701 avg: 14.0 do_nocb_deferred_wakeup(): 1) num: 15348 min: 1 max: 459 avg: 3.0 2) num: 12822 min: 1 max: 564 avg: 4.0 3) num: 8272 min: 0 max: 458 avg: 3.0 rcu_dynticks_task_enter(): 1) num: 6358 min: 1 max: 268 avg: 1.0 2) num: 11128 min: 1 max: 360 avg: 1.0 3) num: 20516 min: 1 max: 214 avg: 1.0 rcu_eqs_exit(): 1) num: 16117 min: 20 max: 782 avg: 43.0 2) num: 11042 min: 20 max: 775 avg: 47.0 3) num: 16499 min: 20 max: 752 avg: 46.0 rcu_eqs_exit_common(): 1) num: 12584 min: 17 max: 703 avg: 28.0 2) num: 17412 min: 17 max: 759 avg: 28.0 3) num: 16733 min: 17 max: 798 avg: 29.0 rcu_dynticks_task_exit(): 1) num: 11730 min: 1 max: 528 avg: 4.0 2) num: 18840 min: 1 max: 581 avg: 5.0 3) num: 9815 min: 1 max: 381 avg: 4.0 rcu_dynticks_eqs_exit(): 1) num: 10902 min: 9 max: 557 avg: 13.0 2) num: 19474 min: 9 max: 563 avg: 13.0 3) num: 11865 min: 9 max: 672 avg: 12.0 Please let me know if there is some data not reasonable, I can revisit again. Thanks, -Aubrey