Received-SPF: pass (google.com: domain of linux-kernel+bounces-58699-linux.lists.archive=gmail.com@vger.kernel.org designates 147.75.199.223 as permitted sender) client-ip=147.75.199.223;
Message-ID: <75f48901-fbfa-4ef4-99b9-312800d20896@intel.com>
Date: Thu, 8 Feb 2024 13:09:23 -0800
User-Agent: Mozilla Thunderbird
Subject: Re: [PATCH 1/2] x86/MCE: Extend size of the MCE Records pool
To: Avadhut Naik <avadhut.naik@amd.com>, <x86@kernel.org>,
	<linux-edac@vger.kernel.org>
CC: <bp@alien8.de>, <tony.luck@intel.com>, <linux-kernel@vger.kernel.org>,
	<yazen.ghannam@amd.com>, <avadnaik@amd.com>
References: <20240207225632.159276-1-avadhut.naik@amd.com>
 <20240207225632.159276-2-avadhut.naik@amd.com>
Content-Language: en-US
From: Sohil Mehta <sohil.mehta@intel.com>
In-Reply-To: <20240207225632.159276-2-avadhut.naik@amd.com>
Content-Type: text/plain; charset="UTF-8"
Content-Transfer-Encoding: 7bit
Precedence: bulk
MIME-Version: 1.0
X-MS-Exchange-AntiSpam-MessageData-ChunkCount: 1
X-MS-Exchange-AntiSpam-MessageData-0: =?utf-8?B?dzM5QjdETnZvdFlWL1F3WnMwbXNqcys5WUxFcW1lTmRnSDJBSFcraDFhS2Mx?=
 =?utf-8?B?ci94NzFYT1UrdDVRUkpOSGxWMjVKMEtuYVFvMTZMeWhOcGVJdEUrTFdmZmVO?=
 =?utf-8?B?TEpzQmFYNnN1Z2REZ25oYWpDNWFNMWFTNC9zekwydGJDc2dpelY0a1B1MDhi?=
 =?utf-8?B?YzFzdVljWnNGTWNiZS8rSS8zaDRtZ3RTbHMwb2E1Njh0TEd3SXREbGdLWXEr?=
 =?utf-8?B?N3pFS1NzZFFyek9jUVhsbXBXRWp6dWRLV0ZuQWFRd3AzNDlrUFFycFZMSjVU?=
 =?utf-8?B?Uzk4TU5KRzNvalp3a1paSDlubkZrWDB0ZnoyWm5WWDhiRFJWRlk2VmRlVnV0?=
 =?utf-8?B?eXZVa2RuUFk3T0NtR2xQeTFsMjVUTUozL0FuYVhVVjhQV3FOdWlYMTNTaktG?=
 =?utf-8?B?ZFI4bmJrZXhid1hKTnYrbWtTRlZ4YTd4YnhaczBGRmFlMml1R2Y4M0J3Ukc0?=
 =?utf-8?B?bWNlcm03WU5KVkVRRTdXRE9Cenh4aVpxMEZyQnB3ZUtXMnF0UWIyK0RlU1h1?=
 =?utf-8?B?SWVqRUhvSGNjUWdlOEtRZ2NjMDB1cThXU3ZNM2luL0ZyV3huS01EMUk1TUxU?=
 =?utf-8?B?QzdtUGlRd2lLbHMzZmpnd0RkWEtOcDY4dll2REVXYkpneWdPemV2Rnk3cUpn?=
 =?utf-8?B?UVdwWHFGclN6NGxtUFRWSXEzQllwb2ZmcEp2RW9yK2daT242NG9iZHZWN1di?=
 =?utf-8?B?Z3Zpb2FBSzJjVjJZWmwvRTF3Qks5YUJLdTlLQ0VOU0t0RitRcFNKRytmcUxs?=
 =?utf-8?B?R0h2cUI4SW54SStnb2RVWjVIS0hOcTFvbXg2dW5yTDZ2VktaeENla1FRVHI5?=
 =?utf-8?B?NU5wV3pnZDF4b2k2VWRXNGRQcWlPanMwMVd0L3dtUDBjcjlHbU5Rdk1yVm1a?=
 =?utf-8?B?M1hOMmdhZDB3d0FISGFuYUZycGpjUVM4ZTMvY0NGVWFrSVk2TmdUWXhMdGcr?=
 =?utf-8?B?akRBR3RNcldmR1VCSm5wZHljT1VyZlQ5YUFWbVFaeG05S2hMSEpzRFQ5S0Mv?=
 =?utf-8?B?Rnp4ZVIyMFhrWGZ5YTd0ckxjRUdpeCs4TTdVUUllRExLRGZrbFV3dE8vaUJW?=
 =?utf-8?B?N2RqbmN6OWhZcHhoNmdaNHIyamhibnhuV2RMRkcvaEhIQSsyTWR1ZmJSbU9Q?=
 =?utf-8?B?NDd3ZFdPaW1LVmFvNUludExWOVA0a1hWV2wremJNcEpVVzhqWVdoQnpraFhH?=
 =?utf-8?B?d1F3VjArZWs5M2Z4djRMczFyWis0aGY0WURDMVZiM2FJVERnbnRsTUxnandE?=
 =?utf-8?B?N0c4NFo4K2lxUGJBQWNrUnQvMDdQOWRCbzJjUnUwaEJ0QitIckF0RWZVRnpV?=
 =?utf-8?B?Z2tpVnVic0tJKzhaODlWU3ZiMWlMMXNzKzVaQU1SVVhBU0FpT215NnZOSGxG?=
 =?utf-8?B?eXpvZGUyY3FNRlJ5UkVod3kyOWVjb3E3MTRKc2J2WDFFQTFaTktWVGYzQnRU?=
 =?utf-8?B?SnJadVB5bVRBdDJMZHFSSE9rR0lVQ3dHNlFua3g2SzdXeitGc0xXU3dCMFBq?=
 =?utf-8?B?Tkg5bG5ROXM5Vlc3Ky9KR0RZTnZURUFSaTlIUm5Oa3paUndWRVFIVzlpdnhE?=
 =?utf-8?B?UjNaYzI4S1pjbU1FKzZHK1Z1OEtGMVcxbVRYcGNoeFNSZ0VMT05rOUx2Z3JH?=
 =?utf-8?B?dDVIYThTeE12MVZhQTJ0N2Q0eWR6cWR4OG54MXVMMEE1TUFIN1JXSDRBWEdm?=
 =?utf-8?B?dDE0UEZhVXVXUXVwS21yTU5uVG9qc1VCaDk4cEdiVjBSeTlVM3JTT05XUmdH?=
 =?utf-8?B?RUlZSVEwbDgxNVZaek9hS3IrbEFyQmc3Yk45Ynd4OG5OeStndE11c0tudmhm?=
 =?utf-8?B?RWY4UFBMcmtZbkJCOGx1eWZFb05MQ25HNitoRysvaVo2MTBJZ3JvYVhYU2J0?=
 =?utf-8?B?akwyL3pqZ3A0MEpHRmgyUkFNTjdGMXg5N0dvSkNQdmlzTkJqOVJLcFlmSlZB?=
 =?utf-8?B?cThGTEh5OVBON3NHazltbnlFaDIwWDZvN3NNUlYyVDduRFE1b0JodWZJckRx?=
 =?utf-8?B?R0hsdm8zNzVwUkgrMHpzK1NKeWl4TzdvcStYOVk1SDBjVGlzK1ZNZyt6UWlB?=
 =?utf-8?B?d2l1dUdHL3hxSUdpYTNUYzhsNHVVdmRPVS9FOEpQYmYwbU1lK3lMdWF2eWtq?=
 =?utf-8?Q?hYAfLtEVmFGpraxYXVAdEDXcW?=
X-MS-Exchange-CrossTenant-Network-Message-Id: 27531e4e-b79d-46b4-4d65-08dc28ea3a81
X-MS-Exchange-CrossTenant-AuthSource: BYAPR11MB3320.namprd11.prod.outlook.com
X-MS-Exchange-CrossTenant-AuthAs: Internal
X-MS-Exchange-CrossTenant-OriginalArrivalTime: 08 Feb 2024 21:09:24.9660
 (UTC)
X-MS-Exchange-CrossTenant-FromEntityHeader: Hosted
X-MS-Exchange-CrossTenant-Id: 46c98d88-e344-4ed4-8496-4ed7712e255d
X-MS-Exchange-CrossTenant-MailboxType: HOSTED
X-MS-Exchange-CrossTenant-UserPrincipalName: UT+1RfXCR299RkYywRDwNslSyBixdEXss+pPnZJ75hBaqIZQbjqlimqRIRfn+NyXHuKrE5hrUuyD3X+IhbeMyQ==
X-MS-Exchange-Transport-CrossTenantHeadersStamped: MW5PR11MB5881
X-OriginatorOrg: intel.com

On 2/7/2024 2:56 PM, Avadhut Naik wrote:

> Extend the size of MCE Records pool to better serve modern systems. The
> increase in size depends on the CPU count of the system. Currently, since
> size of struct mce is 124 bytes, each logical CPU of the system will have
> space for at least 2 MCE records available in the pool. To get around the
> allocation woes during early boot time, the same is undertaken using
> late_initcall().
> 

I guess making this proportional to the number of CPUs is probably fine
assuming CPUs and memory capacity *would* generally increase in sync.

But, is there some logic to having 2 MCE records per logical cpu or it
is just a heuristic approach? In practice, the pool is shared amongst
all MCE sources and can be filled by anyone, right?

> Signed-off-by: Avadhut Naik <avadhut.naik@amd.com>
> ---
>  arch/x86/kernel/cpu/mce/core.c     |  3 +++
>  arch/x86/kernel/cpu/mce/genpool.c  | 22 ++++++++++++++++++++++
>  arch/x86/kernel/cpu/mce/internal.h |  1 +
>  3 files changed, 26 insertions(+)
> 
> diff --git a/arch/x86/kernel/cpu/mce/core.c b/arch/x86/kernel/cpu/mce/core.c
> index b5cc557cfc37..5d6d7994d549 100644
> --- a/arch/x86/kernel/cpu/mce/core.c
> +++ b/arch/x86/kernel/cpu/mce/core.c
> @@ -2901,6 +2901,9 @@ static int __init mcheck_late_init(void)
>  	if (mca_cfg.recovery)
>  		enable_copy_mc_fragile();
>  
> +	if (mce_gen_pool_extend())
> +		pr_info("Couldn't extend MCE records pool!\n");
> +

Why do this unconditionally? For a vast majority of low core-count, low
memory systems the default 2 pages would be good enough.

Should there be a threshold beyond which the extension becomes active?
Let's say, for example, a check for num_present_cpus() > 32 (Roughly
based on 8Kb memory and 124b*2 estimate per logical CPU).

Whatever you choose, a comment above the code would be helpful
describing when the extension is expected to be useful.

>  	mcheck_debugfs_init();
>  
>  	/*
> diff --git a/arch/x86/kernel/cpu/mce/genpool.c b/arch/x86/kernel/cpu/mce/genpool.c
> index fbe8b61c3413..aed01612d342 100644
> --- a/arch/x86/kernel/cpu/mce/genpool.c
> +++ b/arch/x86/kernel/cpu/mce/genpool.c
> @@ -20,6 +20,7 @@
>   * 2 pages to save MCE events for now (~80 MCE records at most).
>   */
>  #define MCE_POOLSZ	(2 * PAGE_SIZE)
> +#define CPU_GEN_MEMSZ	256
>  

The comment above MCE_POOLSZ probably needs a complete re-write. Right
now, it reads as follows:

* This memory pool is only to be used to save MCE records in MCE context.
* MCE events are rare, so a fixed size memory pool should be enough. Use
* 2 pages to save MCE events for now (~80 MCE records at most).

Apart from the numbers being incorrect since sizeof(struct mce) has
increased, this patch is based on the assumption that the current MCE
memory pool is no longer enough in certain cases.

>  static struct gen_pool *mce_evt_pool;
>  static LLIST_HEAD(mce_event_llist);
> @@ -116,6 +117,27 @@ int mce_gen_pool_add(struct mce *mce)
>  	return 0;
>  }
>  
> +int mce_gen_pool_extend(void)
> +{
> +	unsigned long addr, len;

s/len/size/

> +	int ret = -ENOMEM;
> +	u32 num_threads;
> +
> +	num_threads = num_present_cpus();
> +	len = PAGE_ALIGN(num_threads * CPU_GEN_MEMSZ);

Nit: Can the use of the num_threads variable be avoided?
How about:

	size = PAGE_ALIGN(num_present_cpus() * CPU_GEN_MEMSZ);


> +	addr = (unsigned long)kzalloc(len, GFP_KERNEL);

Also, shouldn't the new allocation be incremental to the 2 pages already
present?

Let's say, for example, that you have a 40-cpu system and the calculated
size in this case comes out to 40 * 2 * 128b = 9920bytes  i.e. 3 pages.
You only need to allocate 1 additional page to add to mce_evt_pool
instead of the 3 pages that the current code does.

Sohil

> +
> +	if (!addr)
> +		goto out;
> +
> +	ret = gen_pool_add(mce_evt_pool, addr, len, -1);
> +	if (ret)
> +		kfree((void *)addr);
> +
> +out:
> +	return ret;
> +}
> +
>  static int mce_gen_pool_create(void)
>  {
>  	struct gen_pool *tmpp;