Received-SPF: pass (google.com: domain of linux-kernel-owner@vger.kernel.org designates 2620:137:e000::1:20 as permitted sender) client-ip=2620:137:e000::1:20;
Message-ID: <b4c5348d-c8a1-2923-6d91-d10c359adbd1@oracle.com>
Date:   Mon, 1 May 2023 13:33:47 -0500
User-Agent: Mozilla/5.0 (X11; Linux x86_64; rv:91.0) Gecko/20100101
 Thunderbird/91.5.1
Subject: Re: [PATCH v21 5/7] x86/crash: add x86 crash hotplug support
Content-Language: en-US
To:     Hari Bathini <hbathini@linux.ibm.com>, Baoquan He <bhe@redhat.com>
Cc:     linux-kernel@vger.kernel.org, x86@kernel.org,
        kexec@lists.infradead.org, ebiederm@xmission.com,
        dyoung@redhat.com, vgoyal@redhat.com, tglx@linutronix.de,
        mingo@redhat.com, bp@alien8.de, dave.hansen@linux.intel.com,
        hpa@zytor.com, nramas@linux.microsoft.com, thomas.lendacky@amd.com,
        robh@kernel.org, efault@gmx.de, rppt@kernel.org, david@redhat.com,
        sourabhjain@linux.ibm.com, konrad.wilk@oracle.com,
        boris.ostrovsky@oracle.com
References: <20230404180326.6890-1-eric.devolder@oracle.com>
 <20230404180326.6890-6-eric.devolder@oracle.com>
 <da95e01d-24bd-676b-3a48-bafda1b36cda@linux.ibm.com>
 <ZEo3NK/9Nkc7ctrc@bhe.users.ipa.redhat.com>
 <409c8253-49b9-6993-f79e-8e6203ce4b31@linux.ibm.com>
 <ZEuQ/hxmZey+eFZs@MiWiFi-R3L-srv>
 <97d18ae8-468d-cff5-0a32-4f07b050899e@linux.ibm.com>
From:   Eric DeVolder <eric.devolder@oracle.com>
In-Reply-To: <97d18ae8-468d-cff5-0a32-4f07b050899e@linux.ibm.com>
Content-Type: text/plain; charset=UTF-8; format=flowed
Content-Transfer-Encoding: 8bit
MIME-Version: 1.0
X-MS-Exchange-AntiSpam-MessageData-ChunkCount: 1
X-MS-Exchange-AntiSpam-MessageData-0: =?utf-8?B?cHA5cEZpcWhxY2pTNEFteG11ZVB2NkpreDdZWnhJRFNoRUwwRVF1OFo1dnQz?=
 =?utf-8?B?K0VWN0JCZDZzRjJDWmk0SHZoVnp6dXdWSHM4MkhqbzdMRDZJczNiK1Z1Qy9r?=
 =?utf-8?B?MEQ5S211c2ZtZUI4MmZrSDR1KzJwMFJ3UTNJOGR2MmdQeDBhSzFkc2c2Zk5H?=
 =?utf-8?B?Q21uRTRhTWtSazFGMDBrQ0xjcmhqZGw4VGJnWnYxMUlNMHJmMTg4SjhBejBh?=
 =?utf-8?B?OUQ2ditDQXhodEZZTmR5UVNHYTVKWHJBTTZQYVF1b005Q3ViMll4aGNjQk1w?=
 =?utf-8?B?cDBKOWQrRmRVYjNwSjBqcGxHVkw1MWJzUklrNGVrRm1POHJFVlVELzJQelN1?=
 =?utf-8?B?WnFma0dFRVBRZ2lzQjJUcGtTUmVZN1NYNC9lM0dIaFYwcmd3TTBaRXE2V0o0?=
 =?utf-8?B?dHh0QVJRZlhvbzM4YjRGYVFGM2dGQnBDQldHMXBHdE5kZWh5SnUyb1MzZVdQ?=
 =?utf-8?B?bkhRR3Jadk5MUm9ieDh3bkRkTUdvQkJjK05aOTc5dXIxWVhFc1hYZHhMNndO?=
 =?utf-8?B?OGh0dEdVb3hmUVUvSEZscGRpRkhUZWk2TGNUQVFINlc0STNaeDdtVllGbGZQ?=
 =?utf-8?B?bzl1bHptSGtjb0JOUGRsQitvOC82MjI3QXdiMi9BS2IrTEhjUS8zSmR5eXhW?=
 =?utf-8?B?Skd5YVBlcHJZb0FYRVhTaE9WZjN5bHczejNOaUZNUVloRG8zUUFmL1NKQjVv?=
 =?utf-8?B?SnpUeHV2dmYxMUpvWW5mWFpDU2tlNW1oY3JmNzArZ0Nsc0UvOGlvblB4Ym5h?=
 =?utf-8?B?cjlPbWtPdTVUNnVibmdoS05XTTJSb1dPYUI4T1d4RG5vQnpEZUZvN2tINlRT?=
 =?utf-8?B?M3RTQ1B1V0dqOEE5RnpIQXg4STVCQWU4OGZ5VFVkaVhEZWFrVTgreDA4ZjJa?=
 =?utf-8?B?b1pISEE0a0dZb3VSYnVYeHFoU1F6YmJpcUh2dTRPM2FUR1JEN29KSlp5S1NK?=
 =?utf-8?B?M3M0RkpqS00xaE5xNVFvZ2ZwNFpRQ1pKMG5uRmdFajV0Sk5kWG53ZVU1WHdi?=
 =?utf-8?B?aXpXalpPanV0TDNhcytjWFJ3YzMwYnMrdW4zV0xrcnB0T2UvSE5qak5KeWZV?=
 =?utf-8?B?bDB0Smd0aCthZ0tONWx3d2lqazlpd3VYWldPRElDaCtaMFdSWDc4c0d4NXFM?=
 =?utf-8?B?VUs0cjJmbHFxME1xVGRlMHNlZm5wZTZiTlZHVjB6aUdTclZSMTNEY2NheXVw?=
 =?utf-8?B?aDZjcENZZGdwMTZOTHllaldYME1ZdzFmYWVhMVpVemRtcDdZM3NEVEZzcng0?=
 =?utf-8?B?VzNqdEJTdjBub1RPL2F0ZVNIU2h6QmcwWHppWVc0M1pVamkwK1VJOVpEendS?=
 =?utf-8?B?dFY5OC9UcTZoeU15akFkem9YQjlRS3l1bWRmRlllK3hlVWY0RktwU3RPU1Iz?=
 =?utf-8?B?WUpIL0ZEUjNqOG5EMVZsSEFNR2FqMGpqNG4wZk5DSTJxWjB4OFRod05iYnRm?=
 =?utf-8?B?dG1uQmRwMU9ibDNwYklYWlpPSXVoRk5kaGs2WWllUlVXUGlnMytYb2phdzE1?=
 =?utf-8?B?L24xaG91dVBBM3RyK2V2RjVyRER5UHgvU1cxTXljNkkrY20vZ3FsQ0VhNkIy?=
 =?utf-8?B?RWNMM2ZzOHhqWENzV28rMTBDclowQUtvZ1l5cHY5alFIRnJYUzhnZWJtbEM2?=
 =?utf-8?B?VnB5SlZFUy9KT1l3Y0prbFpHejU2bDFhQUkxV2Uyc2RiQ3ZwWm9Gb1l0U1p3?=
 =?utf-8?B?RHgzMzlXaWZhaXBGZ09BaGxheEFsTXp1cTRHaWZ6TURmc2luVDE4UUlqdlVG?=
 =?utf-8?B?eC9RRVlHcitxMjFTRjdRM09IdjlndjBLMmZMN2RjUHNCMjBxOGQ4dUpKT29n?=
 =?utf-8?B?RFBaaVJqNk9SZUxuQjdrZEFhd1AraXFTanlsenFHZXpvNjJQZXNQQmg1Y0F5?=
 =?utf-8?B?WUpKWU81eHFDbmRHS2xBVjJ1TG5nTnJHK3p2L1pFMkI0aHV3c2N3bEVxRkd3?=
 =?utf-8?B?VkQrRGtidUE0Q3V3dkE2OTJ6clhidUEwZ0RYTUtyeC9scWRCczlaYWZmemJY?=
 =?utf-8?B?MzhrNTlPUFhERTFMYUsremExdEx0YmtWMVVPdjRGZzdiTWhxNUxkeHpIckRU?=
 =?utf-8?B?VU9ONkU2bSszejBDZ293VWFkZHlvNXE1TFpaTDFMSG80YWlkVDRBM1NESXQ1?=
 =?utf-8?Q?ltbKd/dpsGNB0ich08ld6Pg+1?=
X-MS-Exchange-AntiSpam-ExternalHop-MessageData-ChunkCount: 1
X-MS-Exchange-AntiSpam-ExternalHop-MessageData-0: =?utf-8?B?TUJKN28xVlJFKzlwbUl3ZmhpdWlERjNOK0tQSFZOUlNoeDdXTkQwMXlSVGx6?=
 =?utf-8?B?aDYvbHcvRGxjR1YrVWhnakNtOXU4bkx5bERaZlI3VHpLT0Y1ZzJIYzBycWhv?=
 =?utf-8?B?cWh4bEZiSjNBTDIxZStTUDBUQnFVZ2ZTd21mZ1BEUmVnYzV0akh5YTRtZ0h5?=
 =?utf-8?B?SmRBYmZ3WWVJVHVsK1RiQ3dCLzQ4cWQ3TGNnM1d4SDNXRDFBSFZTekdvUTk4?=
 =?utf-8?B?dGhVZGtUSC9ZWFJXNnB1UEdpZHRsZ1F2ZVFVVGs5ZmV6dG5PU1RCeFh3Q1Vv?=
 =?utf-8?B?L2F3NFErVW9iVHQvbHk5V0JvSE1aRElRWms0SEREZXhVa1Y4aXJQSFJGSE1B?=
 =?utf-8?B?amplVmtQTnZJQjZZRGdxZjV2elFXekZaKzUyQXRKbUQ2M3crRENhdFBZVTdn?=
 =?utf-8?B?ellacXJoaVNybTQxZUh2YzErTFE5YVlKQW0wSlIrZElkN281NmNVVHJyZGty?=
 =?utf-8?B?SXZlWlJXOURid1REMXRhVTFhWFFWVnFJL0hkdTZtWlNkVXhjbVNxSVcxOXN2?=
 =?utf-8?B?dVdSNmoyN21NNXpEK2w3c3VVNndRRGlId2gzc3lUem5mNXpsM2xpYk5yUmVy?=
 =?utf-8?B?enNaN1daOVl0U0ZyelZycVF4VXhQaVltTFNLUW96THFrOUluMkhDMlZGSlNC?=
 =?utf-8?B?M28rZ2JvUWdJQlFRa2RQUEd0dUFDSUVHMGdkejNpVWY3NnB0bllOT1hJRWd5?=
 =?utf-8?B?TGFlSjRvbnA2RzBEbGFkQmlUUWRNbk9maDJiajBML3V4eG9HOGFUWkNrc05X?=
 =?utf-8?B?Z3kvalFJVWcweTYvcUYzSVlLaGVwQlRDLzN1ZnlPcUJiL21BMnRFV1BRc1d5?=
 =?utf-8?B?V2Y5WTQ3YXVPN0psWldxTFc4WjNqRGJKcktzeU1wTEhSMFVYYU9lTWRBTzBi?=
 =?utf-8?B?cWpJUnI4Rk4yMXFIeTJDbE9CSjBwT0tYRzFzNXl2SGV6ZGY0amdPQ3JkRk96?=
 =?utf-8?B?dGc4TTdFbzAvcDNMRWQvNEpPZ0VmR1Y5VW82ZWdRUnJoN3duMmRObDdWNWNx?=
 =?utf-8?B?QVBGbE1jTHhmL25FeS9KVEkvSW5YcXBrRmlvS3RMYVdBSENUTXRWcnlGUDhF?=
 =?utf-8?B?RUZTL0FVUDhlTC9vdlFJTUtwVVMvdjM0ZzVWdlVmVjMvQ0FxVGxsMkpIQkE0?=
 =?utf-8?B?ZmRuUFRvRW1KM2NlZ1Nzbm1xeEcwbERvNStMVW45cVIxVFhHTFI4UzRHOER6?=
 =?utf-8?B?NXp0QmI4ZUgxTWJoQ2JnT0JDNGRsTFlWRzB1K2U4RjhSRFdYSElJUmZUMzFH?=
 =?utf-8?B?dDdHendDTUJDcjF3T3ppczlZMUxtbXFiVGNiVVZ3UzBvUURKWDNic0hOYVgv?=
 =?utf-8?B?Z0RabW9sWms4VktuYjdTczkxR0UyaHRQTmRoWW5mUDNhcHVpckx3b2dMdFlj?=
 =?utf-8?B?NzJlQjJPQlFZY01FTlhlaGM2STZ2NGI1SWJUZkx0WEZadGYzQitQTlo1VU80?=
 =?utf-8?B?dDBOKzVzWEUzbW9XNmM2c2lDK3BKOVNOT0VGVkExSk9FZ2ViczR4OVpPeHM5?=
 =?utf-8?B?QWc4cWZranpaMGtDVUJJTzlVZEZHMEdxWVNSOTNyOG91QWd3R1JVWDVVSW9F?=
 =?utf-8?Q?xCl5I9VWT7vBkFss+POa0wA28=3D?=
X-OriginatorOrg: oracle.com
X-MS-Exchange-CrossTenant-Network-Message-Id: 297304b5-e03f-4a5d-c8bc-08db4a72aee9
X-MS-Exchange-CrossTenant-AuthSource: CO1PR10MB4531.namprd10.prod.outlook.com
X-MS-Exchange-CrossTenant-AuthAs: Internal
X-MS-Exchange-CrossTenant-OriginalArrivalTime: 01 May 2023 18:34:22.8662
 (UTC)
X-MS-Exchange-CrossTenant-FromEntityHeader: Hosted
X-MS-Exchange-CrossTenant-Id: 4e2c6054-71cb-48f1-bd6c-3a9705aca71b
X-MS-Exchange-CrossTenant-MailboxType: HOSTED
X-MS-Exchange-CrossTenant-UserPrincipalName: QHv7RLmPx8mPdApSSl4HHb9X1/1GQn4yBWM1QWBksijz2VFhVkP3GCqg9YJGZ8e0dQeeHLr4adjRHf53K1dA2iP8pd2C2PsuboPUGt/+4Pc=
X-MS-Exchange-Transport-CrossTenantHeadersStamped: SA1PR10MB5844
X-Proofpoint-Virus-Version: vendor=baseguard
 engine=ICAP:2.0.254,Aquarius:18.0.942,Hydra:6.0.573,FMLib:17.11.170.22
 definitions=2023-05-01_11,2023-04-27_01,2023-02-09_01
X-Proofpoint-Spam-Details: rule=notspam policy=default score=0 malwarescore=0 mlxlogscore=999
 phishscore=0 bulkscore=0 suspectscore=0 mlxscore=0 spamscore=0
 adultscore=0 classifier=spam adjust=0 reason=mlx scancount=1
 engine=8.12.0-2303200000 definitions=main-2305010152
X-Proofpoint-GUID: AAsiutRZy5QT-yVB_VmZwFivokAFNEWw
X-Proofpoint-ORIG-GUID: AAsiutRZy5QT-yVB_VmZwFivokAFNEWw
X-Spam-Status: No, score=-4.2 required=5.0 tests=BAYES_00,DKIM_SIGNED,
        DKIM_VALID,DKIM_VALID_AU,DKIM_VALID_EF,NICE_REPLY_A,RCVD_IN_DNSWL_LOW,
        RCVD_IN_MSPIKE_H2,SPF_HELO_NONE,SPF_NONE,T_SCC_BODY_TEXT_LINE
        autolearn=ham autolearn_force=no version=3.4.6
X-Spam-Checker-Version: SpamAssassin 3.4.6 (2021-04-09) on
        lindbergh.monkeyblade.net
Precedence: bulk
List-ID: <linux-kernel.vger.kernel.org>
X-Mailing-List: linux-kernel@vger.kernel.org


On 4/28/23 13:31, Hari Bathini wrote:
> 
> On 28/04/23 2:55 pm, Baoquan He wrote:
>> On 04/27/23 at 10:26pm, Hari Bathini wrote:
>>> On 27/04/23 2:19 pm, Baoquan He wrote:
>>>> On 04/27/23 at 12:39pm, Hari Bathini wrote:
>>>>> Hi Eric,
>>>>>
>>>>> On 04/04/23 11:33 pm, Eric DeVolder wrote:
>>>>>> When CPU or memory is hot un/plugged, or off/onlined, the crash
>>>>>> elfcorehdr, which describes the CPUs and memory in the system,
>>>>>> must also be updated.
>>>>>>
>>>>>> The segment containing the elfcorehdr is identified at run-time
>>>>>> in crash_core:crash_handle_hotplug_event(), which works for both
>>>>>> the kexec_load() and kexec_file_load() syscalls. A new elfcorehdr
>>>>>> is generated from the available CPUs and memory into a buffer,
>>>>>> and then installed over the top of the existing elfcorehdr.
>>>>>>
>>>>>> In the patch 'kexec: exclude elfcorehdr from the segment digest'
>>>>>> the need to update purgatory due to the change in elfcorehdr was
>>>>>> eliminated.  As a result, no changes to purgatory or boot_params
>>>>>> (as the elfcorehdr= kernel command line parameter pointer
>>>>>> remains unchanged and correct) are needed, just elfcorehdr.
>>>>>>
>>>>>> To accommodate a growing number of resources via hotplug, the
>>>>>> elfcorehdr segment must be sufficiently large enough to accommodate
>>>>>> changes, see the CRASH_MAX_MEMORY_RANGES description. This is used
>>>>>> only on the kexec_file_load() syscall; for kexec_load() userspace
>>>>>> will need to size the segment similarly.
>>>>>>
>>>>>> To accommodate kexec_load() syscall in the absence of
>>>>>
>>>>> Firstly, thanks! This series is a nice improvement to kdump support
>>>>> in hotplug environment.
Thank you!

>>>>>
>>>>> One concern though is that this change assumes corresponding support
>>>>> in kexec-tools. Without that support kexec_load would fail to boot
>>>>> with digest verification failure, iiuc.

Yes, you've correctly identified that if a hotplug change occurs following kexec_load
(made with kexec-tools unaltered for hotplug), then a subsequent panic would in fact
fail the purgatory digest verification, and  kdump would not happen.

>>>>
>>>> Eric has posted patchset to modify kexec_tools to support that, please
>>>> see the link Eric pasted in the cover letter.
>>>>
>>>> http://lists.infradead.org/pipermail/kexec/2022-October/026032.html
>>>
>>> Right, Baoquan.
>>>
>>> I did see that and if I read the code correctly, without that patchset
>>> kexec_load would fail. Not with an explicit error that hotplug support
>>> is missing or such but it would simply fail to boot into capture kernel
>>> with digest verification failure.
This is correct.

>>>
>>> My suggestion was to avoid that userspace tool breakage for older
>>> kexec-tools version by introducing a new kexec flag that can tell
>>> kernel that kexec-tools is ready to use this in-kernel update support.
>>> So, if kexec_load happens without the flag, avoid doing an in-kernel
>>> update on hotplug. I hope that clears the confusion.
>>
>> Yeah, sounds like a good idea. It may be extended in later patch.
> 
> Fixing it in this series itself would be a cleaner way, I guess.

You're suggestion of using a flag makes alot of sense; it is an indication
to the kernel that it is valid/okay to modify the kexec_load elfcorehdr.
Only kexec-tools that understands this (meaning the elfcorehdr buffer is
appropriately sized *and* excludes the elfcorehdr from the purgatory check)
would set that flag.

The roll-out of this feature needs to be coordinated, no doubt. There are three
pieces to this puzzle: this kernel series, the udev rule changes, and the changes
to kexec-tools for kexec_load.

I consider the udev rule changes critical to making this feature work efficiently.
I also think that deploying the udev rules immediately is doable since nothing
references them, yet; they would be NOPs. And they would be in place when the
kernel and/or kexec-tool changes deploy.

However, your point about supporting kexec_load with and without this new flag
means the sysfs nodes upon which the udev rule change rely need to be a bit
smarter now. (I'm assuming these udev rules will be generally accepted as-is,
as they are simple and efficient.)

The sysfs crash_hotplug nodes need to take into account kexec_file_load vs
(kexec_load && new_flag). Generally speaking these crash_hotplug sysfs nodes we
want to be 1 going forward, but where kexec_load/kexec-tools is older and/or no new_flag,
it needs to be 0. In this way the udev rules can remain as proposed and work properly
for kexec_file_load and both flavors of kexec_load.

Good catch! I'll post v22 soon.

Eric

> 
> Thanks
> Hari