Received-SPF: pass (google.com: domain of linux-kernel-owner@vger.kernel.org designates 2620:137:e000::1:18 as permitted sender) client-ip=2620:137:e000::1:18;
Message-ID: <300a485c-4bc5-c58a-d34a-bacdb1fa4510@oracle.com>
Date:   Tue, 26 Apr 2022 09:39:55 -0500
User-Agent: Mozilla/5.0 (X11; Linux x86_64; rv:91.0) Gecko/20100101
 Thunderbird/91.3.1
Subject: Re: [PATCH v7 2/8] x86/crash: Introduce new options to support cpu
 and memory hotplug
Content-Language: en-US
To:     Sourabh Jain <sourabhjain@linux.ibm.com>,
        linux-kernel@vger.kernel.org, x86@kernel.org,
        kexec@lists.infradead.org, ebiederm@xmission.com,
        dyoung@redhat.com, bhe@redhat.com, vgoyal@redhat.com
Cc:     tglx@linutronix.de, mingo@redhat.com, bp@alien8.de,
        dave.hansen@linux.intel.com, hpa@zytor.com,
        nramas@linux.microsoft.com, thomas.lendacky@amd.com,
        robh@kernel.org, efault@gmx.de, rppt@kernel.org, david@redhat.com,
        konrad.wilk@oracle.com, boris.ostrovsky@oracle.com
References: <20220413164237.20845-1-eric.devolder@oracle.com>
 <20220413164237.20845-3-eric.devolder@oracle.com>
 <d90f4299-e821-f5d5-957a-1e84f16816e0@linux.ibm.com>
From:   Eric DeVolder <eric.devolder@oracle.com>
In-Reply-To: <d90f4299-e821-f5d5-957a-1e84f16816e0@linux.ibm.com>
Content-Type: text/plain; charset=UTF-8; format=flowed
Content-Transfer-Encoding: 8bit
MIME-Version: 1.0
X-MS-Exchange-AntiSpam-MessageData-ChunkCount: 2
X-MS-Exchange-AntiSpam-MessageData-0: =?utf-8?B?ZUllUXZKcVZJcmVJRnZxU09GN2MxSFRkTUlCWGt4NmZtY2dacTVjdWdDUzU1?=
 =?utf-8?B?Yk5VRytnYXhDTDVRYmNxWlkwK0wvcXQraWJ0Yy9jVmtReE9hS1FuZU0wMnp4?=
 =?utf-8?B?bzFyOThBS3loSVJ5cDFacm5sREhqQnpKMFFOVFUyb2RhUjVCWTRzVlFzS0h3?=
 =?utf-8?B?RnV3dFkzMnFaMDUwNHQrMVZwQlhqY0s2WkJQYXZ1SE90b1Q4OXU1YkJtSEQw?=
 =?utf-8?B?VFRIL1NxeVRmaXdKYStMNDdFMGsySDNvQmh6MDd4azhZem1DUUdUSFZNOSt0?=
 =?utf-8?B?ZkNLSVViUUxIWngyMFpmVUx4NHhobVhMRlRtY0JkTVVSdE1Edy9jS3dQT3ht?=
 =?utf-8?B?UkJ1dXNHS0c2N2xFd0VaUkxaNFEzM04yQS9oQW5SMUFMOUFFK0t6b0V4Z2ZK?=
 =?utf-8?B?amtzVVNFYUlYSWxGM3g4VlExMW9odWZsdk1lcWNkOUExTU11SGpsMzQ2RG1V?=
 =?utf-8?B?S1RWWHppL3czVkU2aEVjeTM0R0xka2pPb3NsTCswekd1a1R2WEdvZmYzLzNW?=
 =?utf-8?B?UjZwdmN4TzBJTzcxdGRXbHR4K1NNaFRzTzhBWkJIdXA4YmMwZ2FoZk1MN0lT?=
 =?utf-8?B?aExFcVRnY3VNWFBVRmQ3anNUemFjRGFjVzVtUGttRGhab2VxMnF2aDBkbGk2?=
 =?utf-8?B?elRpY1c3MFpuUTN6ZXR3d0JVaVZxQkdVNC9vK0ZzMmJyRDdsTW01RUdGS2hi?=
 =?utf-8?B?MlRNaWt2U3dJTldvSk5jN01XaDVuclFMQm5relR5dXJaY1U2TmVtZTd0bEIw?=
 =?utf-8?B?ZldEOVp5N1lEc0N3ekVHWTlSQTJ3ZzA2QTRxRk9IemVYMEg2TjJ0U0IxMWtQ?=
 =?utf-8?B?N2hyOWdCSGQ5T1g2aThSWnVhUEJuQzlheWRrelJJQzU1Q0dlYk10dFJ3dUJO?=
 =?utf-8?B?R0VxWHJIcG8wT0lON3VLWnZsZ3FYZC9zbmhDS3dOSXQwUytvV08wL3kwV3ow?=
 =?utf-8?B?dmgzQTNkOEdSVDNlaDNiQmJoTEgxN3lhOGJlYUF2MEEzSlViMU5ydUxaNFk2?=
 =?utf-8?B?dThpMEFabGhTOXRwQWZUR1JSU2hzT05JeUdVaHFPazRNdHdncDIvUnpvTlpo?=
 =?utf-8?B?V2E3bTFxTEwzcmNoQXJYUW1PRzk4Mk40TWloREQwdTVwU2lpT1VqMzhncGpl?=
 =?utf-8?B?cjB4Y0JnZjRtNzBTMTBTcnN2b1pUTmxSWkFmcEpRTUVxU08rRWNBS1JnSkZH?=
 =?utf-8?B?elByUkdRL1I4MitmVkF3cnlGbzJaWmwyOXVSOE14NVUxQ2dwZ0ZweU1WUmdC?=
 =?utf-8?B?L3IweVArNWN4K3E3ZVkrWmFpb2FvSkU1SndMR3h5aEhEbTVIdXR0UGN6b3lQ?=
 =?utf-8?B?T2dueUhiNGc0ZjNVcnRlK0JWTVFvRHFOeG1MZXpoTFBqaVBQbWlzTGE2bDJv?=
 =?utf-8?B?Sms0bTZnMFZxZEtsN21Bbll0Mm5DSGtRZHJyN3JaOHNDRDZ0eWRDRzE1UmJ5?=
 =?utf-8?B?RVk4SEVjczlsM1ZEWVg4Wm1VZmNLRHJwNEcrVXRlYXF6R2VjU2FueTRDT3U4?=
 =?utf-8?B?R0N5QlhwR0hwNTNpRFo5WTFWSHV6N1VETFBBMDNmcTIrdjdBVWszK09BeUxi?=
 =?utf-8?B?T25mNWE3RytCMnN2Y0hzcW5Cb3hRTE9DaXhZTE1lWUdUb0FHcGxZMmxYVWUz?=
 =?utf-8?B?NHJUb2hKRmRUNFp4c25hVTVIWG5QOStITzVMSWhRL0RtRU03OUdpQ09HRURU?=
 =?utf-8?B?OXI1Znl3YWluUG5qN2Z0UnlTejcycC9McHI3WXZmTit4REs3d202aXBxaHND?=
 =?utf-8?B?OUllanRWWFdEcmpNdjI5L2tGMEIwRUtUNzQ5SjNTTWdlQkd3UTY1UGJkSUhT?=
 =?utf-8?B?dnRjVHFWMnEvZy9oNTZZeEJVWXFxdW1JZ0FVMElNNHMxNGJwZDIydEJ1QUNL?=
 =?utf-8?B?L0J6d2RXell6bG9IWFBoV2pnODVXb3ZlM0pteWRMbHBqU2dZTE1oMmhNeFNj?=
 =?utf-8?B?aWhXSmo4L0NwK2xxZDNhQjh4ZlNPU0xKbkIyd0Y3RytuVFFZWDlySjhscjd5?=
 =?utf-8?B?YlY3TWEzQmxyRUU4S0c2bGtTbEFQN01VN1RQbGFscDI4SUpJc0VNQXpmRTRO?=
 =?utf-8?B?OWExU0xNTlhwT1U4eVkwcDhNVjViWVVaTWlMOVc5TWpyOTR4YVFIbXJnRHdz?=
 =?utf-8?B?ZDVWQWxRdk40WG1hbW4xa3VSTGZvSjN4dDdSYzdsS2UxcWFIY2loWHcxQ1d3?=
 =?utf-8?B?R2VMTW8yZ2JiVjFLMXh6L2lWQisvQ1dWckdVeTA0RTBRRmY0NmxUWDRJcGdD?=
 =?utf-8?B?YkU5N1VQcjc0dzVOS1Y5UGZLbm5XUXlMMTk2Y2RwdFVVamo2MVM4RHZwUEFl?=
 =?utf-8?B?WUVrN1BJdG45MEJESVpsY09YbTRFV2FpQzV4RVAyMTM2eE81dVRpL01yODBw?=
 =?utf-8?Q?076fRCxaNdifdv9gX9DPrln00/AsBkBCsNipsKNsqhLzd?=
X-MS-Exchange-AntiSpam-MessageData-1: z0Hxwg7aRW2eIw==
X-OriginatorOrg: oracle.com
X-MS-Exchange-CrossTenant-Network-Message-Id: 1ee4621c-8be4-4288-09ea-08da2792a4de
X-MS-Exchange-CrossTenant-AuthSource: CO1PR10MB4531.namprd10.prod.outlook.com
X-MS-Exchange-CrossTenant-AuthAs: Internal
X-MS-Exchange-CrossTenant-OriginalArrivalTime: 26 Apr 2022 14:40:01.3725
 (UTC)
X-MS-Exchange-CrossTenant-FromEntityHeader: Hosted
X-MS-Exchange-CrossTenant-Id: 4e2c6054-71cb-48f1-bd6c-3a9705aca71b
X-MS-Exchange-CrossTenant-MailboxType: HOSTED
X-MS-Exchange-CrossTenant-UserPrincipalName: mvSTcAYCXsFPt8AZxzRiN3MAfkMkc1sY/i4JM4w+KtWOrDcoYRiAEgAcu+XS5tQbdxg3Ul73BBZyUFvCGAE/M+qTYN/AfwOZIFj9ednuH6M=
X-MS-Exchange-Transport-CrossTenantHeadersStamped: DS7PR10MB5069
X-Proofpoint-Virus-Version: vendor=fsecure engine=2.50.10434:6.0.486,18.0.858
 definitions=2022-04-26_04:2022-04-26,2022-04-26 signatures=0
X-Proofpoint-Spam-Details: rule=notspam policy=default score=0 phishscore=0 mlxlogscore=999 mlxscore=0
 suspectscore=0 malwarescore=0 spamscore=0 adultscore=0 bulkscore=0
 classifier=spam adjust=0 reason=mlx scancount=1 engine=8.12.0-2202240000
 definitions=main-2204260092
X-Proofpoint-GUID: Foy2yvD9KbZDlBtdknZGmUhFPnVkyhb2
X-Proofpoint-ORIG-GUID: Foy2yvD9KbZDlBtdknZGmUhFPnVkyhb2
X-Spam-Status: No, score=-3.8 required=5.0 tests=BAYES_00,DKIM_SIGNED,
	DKIM_VALID,DKIM_VALID_AU,HEADER_FROM_DIFFERENT_DOMAINS,
	MAILING_LIST_MULTI,NICE_REPLY_A,RDNS_NONE,SPF_HELO_NONE
	autolearn=unavailable autolearn_force=no version=3.4.6
X-Spam-Checker-Version: SpamAssassin 3.4.6 (2021-04-09) on
	lindbergh.monkeyblade.net
Precedence: bulk
List-ID: <linux-kernel.vger.kernel.org>
X-Mailing-List: linux-kernel@vger.kernel.org


On 4/25/22 23:21, Sourabh Jain wrote:
> 
> On 13/04/22 22:12, Eric DeVolder wrote:
>> CRASH_HOTPLUG is to enable cpu and memory hotplug support of crash.
>>
>> CRASH_HOTPLUG_ELFCOREHDR_SZ is used to specify the maximum size of
>> the elfcorehdr buffer/segment.
>>
>> This is a preparation for later usage.
>>
>> Signed-off-by: Eric DeVolder <eric.devolder@oracle.com>
>> Acked-by: Baoquan He <bhe@redhat.com>
>> ---
>>   arch/x86/Kconfig | 26 ++++++++++++++++++++++++++
>>   1 file changed, 26 insertions(+)
>>
>> diff --git a/arch/x86/Kconfig b/arch/x86/Kconfig
>> index b0142e01002e..f7b92ee1bcc7 100644
>> --- a/arch/x86/Kconfig
>> +++ b/arch/x86/Kconfig
>> @@ -2072,6 +2072,32 @@ config CRASH_DUMP
>>         (CONFIG_RELOCATABLE=y).
>>         For more details see Documentation/admin-guide/kdump/kdump.rst
>> +config CRASH_HOTPLUG
>> +    bool "kernel updates of crash elfcorehdr"
>> +    depends on CRASH_DUMP && (HOTPLUG_CPU || MEMORY_HOTPLUG) && KEXEC_FILE
>> +    help
>> +      Enable the kernel to update the crash elfcorehdr (which contains
>> +      the list of CPUs and memory regions) directly when hot plug/unplug
>> +      of CPUs or memory. Otherwise userspace must monitor these hot
>> +      plug/unplug change notifications via udev in order to
>> +      unload-then-reload the crash kernel so that the list of CPUs and
>> +      memory regions is kept up-to-date. Note that the udev CPU and
>> +      memory change notifications still occur (however, userspace is not
>> +      required to monitor for crash dump purposes).
>> +
>> +config CRASH_HOTPLUG_ELFCOREHDR_SZ
>> +    depends on CRASH_HOTPLUG
>> +    int
>> +    default 131072
>> +    help
>> +      Specify the maximum size of the elfcorehdr buffer/segment.
>> +      The 128KiB default is sized so that it can accommodate 2048
>> +      Elf64_Phdr, where each Phdr represents either a CPU or a
>> +      region of memory.
>> +      For example, this size can accommodate a machine with up to 1024
>> +      CPUs and up to 1024 memory regions, eg. as represented by the
>> +      'System RAM' entries in /proc/iomem.
> 
> Is it possible to get rid of CRASH_HOTPLUG_ELFCOREHDR_SZ?
At the moment, I do not think so. The idea behind this value is to represent the largest number of 
CPUs and memory regions possible in the system. Today there is NR_CPUS which could be used for CPUs, 
but there isn't a similar value for memory. I also am not aware of a kernel variable that could be 
utilized to represent the maximum number of memory regions. If there is, please let me know!
> 
> How about finding the additional buffer space needed for future CPU and memory
> add during the kdump load? Not sure about the feasibility of doing this in
> kexec tool (userspace).

I may not understand what you are asking, but the x86 code, for kexec_file_load, does in fact 
allocate all the space needed (currently via CRASH_HOTPLUG_ELFCOREHDR_SZ) upon kdump load.

For kexec_load, I've had no problem asking the kexec tool to allocate a larger piece of memory for 
the elfcorehdr. But it is the same problem as CRASH_HOTPLUG_ELFCOREHDR_SZ; how big? In my workspace 
I tell kexec tool how big. If there are sysfs visible values for NR_CPU and memory, then we could 
have kexec pull those and compute.

I do think the important thing is that this allocation needs to happen once (for either kexec_load 
or kexec_file_load), so that the buffer is always in the same spot and thus the pointer to that 
buffer does not change; else boot_params cmdline would need to change. I once had this coded this 
way, but Baoquan pointed out this simpler way.

Regards,
eric

> 
> Thanks,
> Sourabh Jain
> 
> 
>