Received-SPF: pass (google.com: domain of linux-kernel+bounces-123658-linux.lists.archive=gmail.com@vger.kernel.org designates 2604:1380:40f1:3f00::1 as permitted sender) client-ip=2604:1380:40f1:3f00::1;
Message-ID: <e1fe68a5-d088-4e4d-bac4-bbb7793dab6d@intel.com>
Date: Thu, 28 Mar 2024 14:11:01 -0700
User-Agent: Mozilla Thunderbird
Subject: Re: [PATCH] x86/resctrl: Fix mbm_setup_overflow_handler() when last
 CPU goes offline
To: "Luck, Tony" <tony.luck@intel.com>, Borislav Petkov <bp@alien8.de>
CC: "Yu, Fenghua" <fenghua.yu@intel.com>, "Wieczor-Retman, Maciej"
	<maciej.wieczor-retman@intel.com>, Peter Newman <peternewman@google.com>,
	James Morse <james.morse@arm.com>, Babu Moger <babu.moger@amd.com>, "Drew
 Fustini" <dfustini@baylibre.com>, "x86@kernel.org" <x86@kernel.org>,
	"linux-kernel@vger.kernel.org" <linux-kernel@vger.kernel.org>,
	"patches@lists.linux.dev" <patches@lists.linux.dev>
References: <20240327184619.236057-1-tony.luck@intel.com>
 <c53d6abd-5dbe-4edf-89c0-abbea0df1f1c@intel.com>
 <SJ1PR11MB6083F3AB8069CCDC7DB5004BFC342@SJ1PR11MB6083.namprd11.prod.outlook.com>
Content-Language: en-US
From: Reinette Chatre <reinette.chatre@intel.com>
In-Reply-To: <SJ1PR11MB6083F3AB8069CCDC7DB5004BFC342@SJ1PR11MB6083.namprd11.prod.outlook.com>
Content-Type: text/plain; charset="UTF-8"
Content-Transfer-Encoding: 7bit
Precedence: bulk
MIME-Version: 1.0
X-MS-Exchange-AntiSpam-MessageData-ChunkCount: 1
X-MS-Exchange-AntiSpam-MessageData-0: =?utf-8?B?MjFoc3Q2em9xWEhRV0Rnck1HcTJyS1BuMUFpQmxGVkVxMllwaVhJYWlqdloy?=
 =?utf-8?B?c3JEZVRyUVpHL3o5RGNRcllqYWQ1Ti9HdnduUlNjMXF1dEkyRFFmWktEbXd5?=
 =?utf-8?B?R1hRRkViQkRXNlBQQ3RNL1pYSHphMUVQV3BTMHVxTU9qdmw1R3p2QkhxU0Q1?=
 =?utf-8?B?VFF1eVhxMWV5WWNqYmhOdU51Njd1cGJmbWZPb21oTnF0QVc1SzE1Nzc4VGli?=
 =?utf-8?B?WWorKzlsWUY4cHhSS1FoczZJMHYwRklMbnhKbkswN2pCaHBYTjNJZnNad2xi?=
 =?utf-8?B?WmFXK2Z0ZElsdWQrMHJDemFLZUdJMWdGeXpxemRiV2QyRHlvaGxoTWZENGVx?=
 =?utf-8?B?cFdDemtoU1JtRis0WFkyYVIrdDdQY1pFQU9kazA1b0Z0dllvdThrVTgxK0N4?=
 =?utf-8?B?enI2SjR3WExBNXJGTnlGbHJNLzNUZlVLaHpNK0lFRzA4aHBJNDFFejR2WTdE?=
 =?utf-8?B?MkdLK2g4N0ZmRXo3Y2YvaHBXMUJhQnorZDFtOGpjUW5TZ1RpQXM3TmhpM3FM?=
 =?utf-8?B?THM0YTZqdEgwdzc0OEw3R2dpaWRrcHcyVGg2UGp5OHU2cm41b0ZnZEFNQ1dM?=
 =?utf-8?B?RjZUUitqKytGZlF2Nk5PZnZoU3VWNGtmeVBIZ1BvYXlINGc1TVlFMklPdGgz?=
 =?utf-8?B?eWd2bmV5ZnYrZGloeWdmTFhIVGIxZG45Q0U0a1BLVVpFN3o0ZXFpWmZrNjlJ?=
 =?utf-8?B?Z2JZck0wT2VObzJ0QllORXpDTkJBSmJ3YjFicGlCdHRheTVXUG1wZXJRZSs4?=
 =?utf-8?B?MGtYckQwRkUrdDJvb3ZpTXBVT0xHN1NTSlhpNjFPUFdrVzZuMFpHM1g4dUJ4?=
 =?utf-8?B?Rnh1RUIzWm9lSUlHMllPOVBTdk5VWkJkS3puQUZ3Y3FibUtvdXloTTFSR243?=
 =?utf-8?B?WmpGbG9BZG9lVkNWbHRKNGN4YXJ5Z0k1YlBkcU5MckNOM1ZrNG1BVDkxcVF0?=
 =?utf-8?B?QW1KZXRzZ0N4YlhaVzVyMEdFZFZEem93Q0hzM3NsVUdkVGFmOWRLV1htdXVo?=
 =?utf-8?B?OTBQRkdjM2dHaEpyQUhPS0diV1ViYS9tMUY1Y0EyZzBLdW1ySTJkV2VrZEs4?=
 =?utf-8?B?Qkp6TlVjakRFNHowckYyL2EzMklUK243bCsvNllBaGJTMmtjRXdhTDZOUEhM?=
 =?utf-8?B?cVNKK1BmYkl3eE1iV1F1WE9rdTZGMURQUWJrU1lINFdWakM0MG43OU1rR2Y0?=
 =?utf-8?B?VEEzZFE3R3orYS9yVVc2dHhhRmMxZEp5UnpOYzAzUUREVWxxMG5HZTZUZURl?=
 =?utf-8?B?Snpla1QzVnVkWi9RQmcxMXErdlN2ME9OSzcyeUhDSEh6QVhHTDVXQUp0Q1ZE?=
 =?utf-8?B?RW96OXE3Y1VpSldZSXJOVXlsUzZYbjFzdFkwR3FmRjY3dlJxOUQ3YlBhWWJo?=
 =?utf-8?B?THJVeFNoT2NJK01ZMC9VaG9aajBYNzgzOUFLL2FNTGZjNWpkMytQZFFUVTBO?=
 =?utf-8?B?QXRYV3hVdlYxRmVwT1NGbDYrY2RFVkcwenBvNnNMUlh2Q3RsN1oweGRYL1pU?=
 =?utf-8?B?RjJlUGZjQzc0TzVocFFDQ2FxWEN1dis1YnFwR3NEbU5uWThBaHdWTTVPcmRh?=
 =?utf-8?B?Ump6TmZ4UENLYjZOZXNQcm1kT1NvMnZzMWY5NHRROW96TWhhcHhLR2ppN3R3?=
 =?utf-8?B?YnJwTGVrOXJPZzZwZGlycFhEL1JJMWdQb1hWK3Q1eGdjeG85NkIzYmxpaC9n?=
 =?utf-8?B?Zld0bGlCTzFwZmh6RFBZckw4ZEc1ejJrQWFHSFhDY1plamlOK0hYdWYrSkdh?=
 =?utf-8?B?NlE1M01YUWtLVlNSbGJ6Y1hUenNQVFRSdWh1UVhHUkNOVEZjMWNvcVV1QXJV?=
 =?utf-8?B?SmM1S2h3aTBGc25JbUhWMHJDcXpJYnl2REJ4UDdOMUZ6UkJZNXhkSmxVeWRM?=
 =?utf-8?B?cnJ3dCt0eXQ5eCtuK09TTjNlMkJFYW9IUUFLZjVPNHZQNkY0cnI1WTl4ci94?=
 =?utf-8?B?aVMrTTNNbXRvb1U5SHJyNWxlcVpqMU9aRUU3T2taV0R5SklwQWhzK1dIKzIz?=
 =?utf-8?B?aVBQUGhjSTI1c1JyRml4RFQ5MDlzbUF4ck43SkpyWDBUZGU3WmRrMHNBbUpn?=
 =?utf-8?B?NEo3MDZFYWpNV2lWcVJkaFZKajhUVVF0c3d0TmZ4d25FZlRrdGd1NFpHREI2?=
 =?utf-8?B?WUZUdHJjTEhlRUhKYmo3SldGbU1oeXN1RmV4dEtYOXpxRUozcjJEZjBaZUU0?=
 =?utf-8?B?YUE9PQ==?=
X-MS-Exchange-CrossTenant-Network-Message-Id: 0ace8876-834c-43bf-5974-08dc4f6b9542
X-MS-Exchange-CrossTenant-AuthSource: SJ2PR11MB7573.namprd11.prod.outlook.com
X-MS-Exchange-CrossTenant-AuthAs: Internal
X-MS-Exchange-CrossTenant-OriginalArrivalTime: 28 Mar 2024 21:11:06.5572
 (UTC)
X-MS-Exchange-CrossTenant-FromEntityHeader: Hosted
X-MS-Exchange-CrossTenant-Id: 46c98d88-e344-4ed4-8496-4ed7712e255d
X-MS-Exchange-CrossTenant-MailboxType: HOSTED
X-MS-Exchange-CrossTenant-UserPrincipalName: RiqhX3copYN4N6GMC+m0DdUWOGs4zWfhFYXxCdJ2yZcjUFkJ2ucKFoEjuO2IdmZrlO90MyUhC0SA6bun5Vp6fWGMS9k9J5pY7Kp4IaM0Yuk=
X-MS-Exchange-Transport-CrossTenantHeadersStamped: SJ2PR11MB7547
X-OriginatorOrg: intel.com

Hi Tony,

On 3/27/2024 4:01 PM, Luck, Tony wrote:
>> There seems to be two issues here (although I am not familiar with these flows). First,
>> it seems that tick_nohz_full_mask is not actually allocated unless the user boots
>> with a "nohz_full=". This means that any attempt to access bits within tick_nohz_full_mask
>> will cause this OOPS. If that is allocated then the second issue seems that the  
>> buried __ffs() call requires that it not be called with 0 and this checking is not done.
>>
>> To me it seems most appropriate to fix this at the central place to ensure all scenarios
>> are handled instead of scattering checks.
> 
> Good analysis.
> 
>> To that end, what do you think of something like below? It uses tick_nohz_full_enabled() check
>> to ensure that tick_nohz_full_mask is actually allocated while the other changes aim to
>> avoid __ffs() on 0.

I studied the flows some more and I no longer believe that there is a risk of __ffs() being
called on 0. Looking at the flows starting with cpumask_nth_andnot() the value provided to
__ffs() is ensured to be non-zero via either an explicit check or hweight_long().

To confirm this I tested a CONFIG_NO_HZ_FULL kernel by booting with "nohz_full=" set to the
highest numbered CPU in the domain. When all the CPUs are offlined the behavior for the
"second to last" and "last" CPU are most interesting and worked as expected when using 
cpumask_nth_andnot(). 

> 
> Looks good.
> 
> Tested-by: Tony Luck <tony.luck@intel.com>
> Reviewed-by: Tont Luck <tony.luck@intel.com>

Considering the motivation above I will submit a change that only adds the 
tick_nohz_full_enabled() check. Since it is such a big change from the snippet you reviewed
and tested here I dropped your tags from it and hope you can reconsider the fix after considering
the information above.

Reinette