Received-SPF: pass (google.com: domain of linux-kernel-owner@vger.kernel.org designates 23.128.96.34 as permitted sender) client-ip=23.128.96.34;
Date:   Thu, 26 Oct 2023 15:02:42 +0800
From:   Chen Yu <yu.c.chen@intel.com>
To:     Thomas Gleixner <tglx@linutronix.de>
CC:     Juergen Gross <jgross@suse.com>, Len Brown <len.brown@intel.com>,
        "Rafael J . Wysocki" <rafael.j.wysocki@intel.com>,
        Dan Williams <dan.j.williams@intel.com>,
        <linux-kernel@vger.kernel.org>, Chen Yu <yu.chen.surf@gmail.com>,
        Wendy Wang <wendy.wang@intel.com>
Subject: Re: [RFC PATCH] genirq: Exclude managed irq during irq migration
Message-ID: <ZToPEihOjvttyh5q@chenyu5-mobl2.ccr.corp.intel.com>
References: <20231020072522.557846-1-yu.c.chen@intel.com>
 <87v8au6a98.ffs@tglx>
Content-Type: text/plain; charset="us-ascii"
Content-Disposition: inline
In-Reply-To: <87v8au6a98.ffs@tglx>
MIME-Version: 1.0
X-MS-Exchange-AntiSpam-MessageData-ChunkCount: 1
X-MS-Exchange-AntiSpam-MessageData-0: =?us-ascii?Q?E2F7APcObg2LF7GC4MZz0lsgvjg84nQVd4aRNX/WmUIex/StE1bwFgE/DUzq?=
 =?us-ascii?Q?HlX8uLjhsHe+hHZPt2PgEElYvN49uKCJqc/rrLosqyPPmepPCCdaCuw5X6JC?=
 =?us-ascii?Q?g2nRVt5UlHPhfhiOICCisEkKdKVij40UD5kHs8XuaGruBmjffDXzg/XFQnFt?=
 =?us-ascii?Q?6mC/LZMYLpXlzztxw/9xvr8aUYxGwS3zJ75UpdLUb1+CE18OGJCz6fwXW8J+?=
 =?us-ascii?Q?N8pW7oZTUb2OFYb5XPDmYBWpvXi38yyXXB6IFYNXjm4JNKBMKFHrV0osyTBF?=
 =?us-ascii?Q?pxm8U9zIlPCO3wcG1+t5tm3NFLf+16P4xQOYrJonH6Bvnt9C4B8xpD4F6vpy?=
 =?us-ascii?Q?Fa4j7tC29Q+HavNHSLF5nBQGp16korgOiybCOM9Yc+z/E5fML/cSU9HZbpGT?=
 =?us-ascii?Q?nTRu2Pey0i66hQRbJLRL9d1IPAR5mmyJ+eUg5yVYMUrauQGohtfC5YFfna8Y?=
 =?us-ascii?Q?DjhfbPfK+fCNjqh+HK8GPRAuyNascvZmjTMNBt95jnEiUnvtwI2aKEpqLVZo?=
 =?us-ascii?Q?oOItE6SaDW1PhXhhDed5xm6nsT8rCUSgAFRuhE40KD2f9Opz6ZD5zb3zsOEB?=
 =?us-ascii?Q?s8jtFprnhC2YWJ9vGvOT+W7bl+q137DR1grrqN2x9vQQWIe05NwNB7NoV4S4?=
 =?us-ascii?Q?9M8GjdBe99gGcEb0rTnxBHAEbZfRnwxQG8e/Hd5BeI3POG1FK3i/VKfb1oW6?=
 =?us-ascii?Q?f7Auq+23C8aSqdTbSdiNBi3qK7Kky4f4Vk/NmVE/3exJ3tdtsqzqg950adsB?=
 =?us-ascii?Q?pzJbqGsv0RAqirdyow8jSZY+vGNHyeNOCFGTv7P9p5/JUOBiJWyX5Jvt2IqR?=
 =?us-ascii?Q?G7lLMV2VWL1N9yaRopHoY0zeZNXsER/0J/LwDAzLw5Rmm/u0NRltMI1v9b8R?=
 =?us-ascii?Q?JsAZhGKst154klpLIJ4Nm9d+uWIt04a7HNYnBq4cLdwPgAixrZT93BwtjAdC?=
 =?us-ascii?Q?nARVt36F3PE0Ck9CNywKI8fmqGFfH/UD0Lf+EoFXH4+P6ujnRMik2ohqDlGr?=
 =?us-ascii?Q?BoZYHC8KbOtlZ5fyl1bqdaAb5ujeLqqO4tP0iKFrDr0Bc/r0fF9b2Rrxy1IS?=
 =?us-ascii?Q?7h5VdANkKEKUcnranHgOSrltWfMvRJNlMDF+3+wx6k6kXlYJPUXrfTBooH+G?=
 =?us-ascii?Q?C8BVoJqCet06AEWKGGGVrhmaI4sSJnNgvBt9lI6rGEzNPm+K13uSo22FBvLI?=
 =?us-ascii?Q?YffFxPAKA7o4HRNNQwpehMYByLMjDB1OInc9yEUYv9mKpiNRgVmrchDWpLxt?=
 =?us-ascii?Q?QszMuFBbxRLgaR3OmATvpetY0V4UL063FhdDkY3fyDSnoDmMHzvHhuy3Yajz?=
 =?us-ascii?Q?OwupWMb6AIhPIGHJltE15yw/dEJy0jhBFC/N5/m6dQrP1Po9hEFQTf2nk2LD?=
 =?us-ascii?Q?G790aY2ELEZY0nBl9qx0iJFVTspCklKV2CXvmLl5b/nZV1N0+gCS8r4cfQGC?=
 =?us-ascii?Q?Kqzg66Mp/g0uK00UguKNH0zEj7Fem6z3nt3b24JAcRHUjC5EKFG2ThrFDkcM?=
 =?us-ascii?Q?J8oDZ0KHj0CIwSXMN2+slqfkPmabEHoeebdI0OAo2P6a9ZZFFtgIKR8oSueq?=
 =?us-ascii?Q?9wGpsQK83Prj7v2R9Q9nuO8RtFRi2ud5uPmSgYWL?=
X-MS-Exchange-CrossTenant-Network-Message-Id: 3ce0c203-4a05-48d1-71b4-08dbd5f1946c
X-MS-Exchange-CrossTenant-AuthSource: MN0PR11MB6206.namprd11.prod.outlook.com
X-MS-Exchange-CrossTenant-AuthAs: Internal
X-MS-Exchange-CrossTenant-OriginalArrivalTime: 26 Oct 2023 07:02:56.1181
 (UTC)
X-MS-Exchange-CrossTenant-FromEntityHeader: Hosted
X-MS-Exchange-CrossTenant-Id: 46c98d88-e344-4ed4-8496-4ed7712e255d
X-MS-Exchange-CrossTenant-MailboxType: HOSTED
X-MS-Exchange-CrossTenant-UserPrincipalName: 2tTq5bM5uIfdr8XI7tm8r+c8YvEVClRbhjvGmCLEb9yuy/Yn5VkKO+9L443pS/gZclolv55q9s+cSS3nMicShg==
X-MS-Exchange-Transport-CrossTenantHeadersStamped: DM4PR11MB5344
X-OriginatorOrg: intel.com
X-Spam-Status: No, score=-0.9 required=5.0 tests=DKIMWL_WL_HIGH,DKIM_SIGNED,
	DKIM_VALID,DKIM_VALID_AU,HEADER_FROM_DIFFERENT_DOMAINS,
	MAILING_LIST_MULTI,SPF_HELO_NONE,SPF_PASS autolearn=unavailable
	autolearn_force=no version=3.4.6
X-Spam-Checker-Version: SpamAssassin 3.4.6 (2021-04-09) on howler.vger.email
Precedence: bulk
List-ID: <linux-kernel.vger.kernel.org>
X-Mailing-List: linux-kernel@vger.kernel.org
X-Greylist: Sender passed SPF test, not delayed by milter-greylist-4.6.4 (howler.vger.email [0.0.0.0]); Thu, 26 Oct 2023 00:03:25 -0700 (PDT)

Hi Thomas,

On 2023-10-25 at 16:34:59 +0200, Thomas Gleixner wrote:
> Chen!
> 
> On Fri, Oct 20 2023 at 15:25, Chen Yu wrote:
> > The managed IRQ will be shutdown and not be migrated to
> 
> Please write out interrupts in change logs, this is not twitter.
> 
> > other CPUs during CPU offline. Later when the CPU is online,
> > the managed IRQ will be re-enabled on this CPU. The managed
> > IRQ can be used to reduce the IRQ migration during CPU hotplug.
> >
> > Before putting the CPU offline, the number of the already allocated
> > IRQs on this offlining CPU will be compared to the total number
> 
> The usage of IRQs and vectors is slightly confusing all over the
> place.
> 
> > of available IRQ vectors on the remaining online CPUs. If there is
> > not enough slot for these IRQs to be migrated to, the CPU offline
> > will be terminated. However, currently the code treats the managed
> > IRQ as migratable, which is not true, and brings false negative
> > during CPU hotplug and hibernation stress test.
> 
> Your assumption that managed interrupts cannot be migrated is only
> correct when the managed interrupts affinity mask has exactly one online
> target CPU. Otherwise the interrupt is migrated to one of the other
> online CPUs in the affinity mask.
> 
> Though that does not affect the migrateability calculation because in
> case that a managed interrupt has an affinity mask with more than one
> target CPU set, the vectors on the currently not targeted CPUs are
> already reserved and accounted for in matrix->global_available. IOW,
> migrateability for such managed interrupts is already guaranteed.
>

Got it, the percpu cm->managed has been pre-reserved already and is
excluded from m->global_available. So we still should substract the
number of allocated managed interrupts to avoid duplicated calculation.
 
> I'll amend the changelog to make this clear.
> 

Thanks for helping on this.

thanks,
Chenyu