The ipmi communication is not restored after a specific version of BMC is
upgraded on our server.
The ipmi driver does not respond after printing the following log:
ipmi_ssif: Invalid response getting flags: 1c 1
I found that after entering this branch, ssif_info->ssif_state always
holds SSIF_GETTING_FLAGS and never return to IDLE.
As a result, the driver cannot be loaded, because the driver status is
checked during the unload process and must be IDLE in shutdown_ssif():
while (ssif_info->ssif_state != SSIF_IDLE)
schedule_timeout(1);
The process trigger this problem is:
1. One msg timeout and next msg start send, and call
ssif_set_need_watch().
2. ssif_set_need_watch()->watch_timeout()->start_flag_fetch() change
ssif_state to SSIF_GETTING_FLAGS.
3. In msg_done_handler() ssif_state == SSIF_GETTING_FLAGS, if an error
message is received, the second branch does not modify the ssif_state.
4. All retry action need IS_SSIF_IDLE() == True. Include retry action in
watch_timeout(), msg_done_handler(). Sending msg does not work either.
SSIF_IDLE is also checked in start_next_msg().
5. The only thing that can be triggered in the SSIF driver is
watch_timeout(), after destory_user(), this timer will stop too.
So, if enter this branch, the ssif_state will remain SSIF_GETTING_FLAGS
and can't send msg, no timer started, can't unload.
We did a comparative test before and after adding this patch, and the
result is effective.
Fixes: 259307074bfc ("ipmi: Add SMBus interface driver (SSIF)")
Signed-off-by: Zhang Yuchen <[email protected]>
---
drivers/char/ipmi/ipmi_ssif.c | 4 ++--
1 file changed, 2 insertions(+), 2 deletions(-)
diff --git a/drivers/char/ipmi/ipmi_ssif.c b/drivers/char/ipmi/ipmi_ssif.c
index a5ddebb1edea..48be3694fa64 100644
--- a/drivers/char/ipmi/ipmi_ssif.c
+++ b/drivers/char/ipmi/ipmi_ssif.c
@@ -784,9 +784,9 @@ static void msg_done_handler(struct ssif_info *ssif_info, int result,
} else if (data[0] != (IPMI_NETFN_APP_REQUEST | 1) << 2
|| data[1] != IPMI_GET_MSG_FLAGS_CMD) {
/*
- * Don't abort here, maybe it was a queued
- * response to a previous command.
+ * Recv error response, give up.
*/
+ ssif_info->ssif_state = SSIF_IDLE;
ipmi_ssif_unlock_cond(ssif_info, flags);
dev_warn(&ssif_info->client->dev,
"Invalid response getting flags: %x %x\n",
--
2.20.1
On Wed, Apr 12, 2023 at 03:49:07PM +0800, Zhang Yuchen wrote:
> The ipmi communication is not restored after a specific version of BMC is
> upgraded on our server.
> The ipmi driver does not respond after printing the following log:
Excellent explaination, this is queued in my branch for the next release
and marked for backport.
Thank you,
-corey
>
> ipmi_ssif: Invalid response getting flags: 1c 1
>
> I found that after entering this branch, ssif_info->ssif_state always
> holds SSIF_GETTING_FLAGS and never return to IDLE.
>
> As a result, the driver cannot be loaded, because the driver status is
> checked during the unload process and must be IDLE in shutdown_ssif():
>
> while (ssif_info->ssif_state != SSIF_IDLE)
> schedule_timeout(1);
>
> The process trigger this problem is:
>
> 1. One msg timeout and next msg start send, and call
> ssif_set_need_watch().
>
> 2. ssif_set_need_watch()->watch_timeout()->start_flag_fetch() change
> ssif_state to SSIF_GETTING_FLAGS.
>
> 3. In msg_done_handler() ssif_state == SSIF_GETTING_FLAGS, if an error
> message is received, the second branch does not modify the ssif_state.
>
> 4. All retry action need IS_SSIF_IDLE() == True. Include retry action in
> watch_timeout(), msg_done_handler(). Sending msg does not work either.
> SSIF_IDLE is also checked in start_next_msg().
>
> 5. The only thing that can be triggered in the SSIF driver is
> watch_timeout(), after destory_user(), this timer will stop too.
>
> So, if enter this branch, the ssif_state will remain SSIF_GETTING_FLAGS
> and can't send msg, no timer started, can't unload.
>
> We did a comparative test before and after adding this patch, and the
> result is effective.
>
> Fixes: 259307074bfc ("ipmi: Add SMBus interface driver (SSIF)")
>
> Signed-off-by: Zhang Yuchen <[email protected]>
> ---
> drivers/char/ipmi/ipmi_ssif.c | 4 ++--
> 1 file changed, 2 insertions(+), 2 deletions(-)
>
> diff --git a/drivers/char/ipmi/ipmi_ssif.c b/drivers/char/ipmi/ipmi_ssif.c
> index a5ddebb1edea..48be3694fa64 100644
> --- a/drivers/char/ipmi/ipmi_ssif.c
> +++ b/drivers/char/ipmi/ipmi_ssif.c
> @@ -784,9 +784,9 @@ static void msg_done_handler(struct ssif_info *ssif_info, int result,
> } else if (data[0] != (IPMI_NETFN_APP_REQUEST | 1) << 2
> || data[1] != IPMI_GET_MSG_FLAGS_CMD) {
> /*
> - * Don't abort here, maybe it was a queued
> - * response to a previous command.
> + * Recv error response, give up.
> */
> + ssif_info->ssif_state = SSIF_IDLE;
> ipmi_ssif_unlock_cond(ssif_info, flags);
> dev_warn(&ssif_info->client->dev,
> "Invalid response getting flags: %x %x\n",
> --
> 2.20.1
>