SMC connections might fail to be registered to a link group due to
things like unable to find a link to assign to in its creation. As
a result, connection creation will return a failure and most
resources related to the connection won't be applied or initialized,
such as conn->abort_work or conn->lnk.
If smc_conn_free() is invoked later, it will try to access the
resources related to the connection, which wasn't initialized, thus
causing a panic.
Here is an example, a SMC-R connection failed to be registered
to a link group and conn->lnk is NULL. The following crash will
happen if smc_conn_free() tries to access conn->lnk in
smc_cdc_tx_dismiss_slots().
BUG: kernel NULL pointer dereference, address: 0000000000000168
#PF: supervisor read access in kernel mode
#PF: error_code(0x0000) - not-present page
PGD 0 P4D 0
Oops: 0000 [#1] PREEMPT SMP PTI
CPU: 4 PID: 68 Comm: kworker/4:1 Kdump: loaded Tainted: G E 5.16.0-rc5+ #52
Workqueue: smc_hs_wq smc_listen_work [smc]
RIP: 0010:smc_wr_tx_dismiss_slots+0x1e/0xc0 [smc]
Call Trace:
<TASK>
smc_conn_free+0xd8/0x100 [smc]
smc_lgr_cleanup_early+0x15/0x90 [smc]
smc_listen_work+0x302/0x1230 [smc]
? process_one_work+0x25c/0x600
process_one_work+0x25c/0x600
worker_thread+0x4f/0x3a0
? process_one_work+0x600/0x600
kthread+0x15d/0x1a0
? set_kthread_struct+0x40/0x40
ret_from_fork+0x1f/0x30
</TASK>
This patch tries to fix this by resetting conn->lgr to NULL if an
abnormal exit occurs in smc_lgr_register_conn(), thus avoiding the
crash caused by accessing the uninitialized resources in smc_conn_free().
And the new created link group will be terminated if smc connections
can't be registered to it.
Fixes: 56bc3b2094b4 ("net/smc: assign link to a new connection")
Signed-off-by: Wen Gu <[email protected]>
---
v1->v2:
- Reset conn->lgr to NULL in smc_lgr_register_conn().
- Only free new created link group.
v2->v3:
- Using __smc_lgr_terminate() instead of smc_lgr_schedule_free_work()
for an immediate free.
---
net/smc/smc_core.c | 12 ++++++++++--
1 file changed, 10 insertions(+), 2 deletions(-)
diff --git a/net/smc/smc_core.c b/net/smc/smc_core.c
index 412bc85..0201f99 100644
--- a/net/smc/smc_core.c
+++ b/net/smc/smc_core.c
@@ -171,8 +171,10 @@ static int smc_lgr_register_conn(struct smc_connection *conn, bool first)
if (!conn->lgr->is_smcd) {
rc = smcr_lgr_conn_assign_link(conn, first);
- if (rc)
+ if (rc) {
+ conn->lgr = NULL;
return rc;
+ }
}
/* find a new alert_token_local value not yet used by some connection
* in this link group
@@ -1835,8 +1837,14 @@ int smc_conn_create(struct smc_sock *smc, struct smc_init_info *ini)
write_lock_bh(&lgr->conns_lock);
rc = smc_lgr_register_conn(conn, true);
write_unlock_bh(&lgr->conns_lock);
- if (rc)
+ if (rc) {
+ spin_lock_bh(lgr_lock);
+ if (!list_empty(&lgr->list))
+ list_del_init(&lgr->list);
+ spin_unlock_bh(lgr_lock);
+ __smc_lgr_terminate(lgr, true);
goto out;
+ }
}
conn->local_tx_ctrl.common.type = SMC_CDC_MSG_TYPE;
conn->local_tx_ctrl.len = SMC_WR_TX_SIZE;
--
1.8.3.1
On Wed, Jan 05, 2022 at 02:28:53PM +0800, Wen Gu wrote:
>SMC connections might fail to be registered to a link group due to
>things like unable to find a link to assign to in its creation. As
>a result, connection creation will return a failure and most
>resources related to the connection won't be applied or initialized,
>such as conn->abort_work or conn->lnk.
>
>If smc_conn_free() is invoked later, it will try to access the
>resources related to the connection, which wasn't initialized, thus
>causing a panic.
>
>Here is an example, a SMC-R connection failed to be registered
>to a link group and conn->lnk is NULL. The following crash will
>happen if smc_conn_free() tries to access conn->lnk in
>smc_cdc_tx_dismiss_slots().
>
> BUG: kernel NULL pointer dereference, address: 0000000000000168
> #PF: supervisor read access in kernel mode
> #PF: error_code(0x0000) - not-present page
> PGD 0 P4D 0
> Oops: 0000 [#1] PREEMPT SMP PTI
> CPU: 4 PID: 68 Comm: kworker/4:1 Kdump: loaded Tainted: G E 5.16.0-rc5+ #52
> Workqueue: smc_hs_wq smc_listen_work [smc]
> RIP: 0010:smc_wr_tx_dismiss_slots+0x1e/0xc0 [smc]
> Call Trace:
> <TASK>
> smc_conn_free+0xd8/0x100 [smc]
> smc_lgr_cleanup_early+0x15/0x90 [smc]
> smc_listen_work+0x302/0x1230 [smc]
> ? process_one_work+0x25c/0x600
> process_one_work+0x25c/0x600
> worker_thread+0x4f/0x3a0
> ? process_one_work+0x600/0x600
> kthread+0x15d/0x1a0
> ? set_kthread_struct+0x40/0x40
> ret_from_fork+0x1f/0x30
> </TASK>
>
>This patch tries to fix this by resetting conn->lgr to NULL if an
>abnormal exit occurs in smc_lgr_register_conn(), thus avoiding the
>crash caused by accessing the uninitialized resources in smc_conn_free().
>And the new created link group will be terminated if smc connections
>can't be registered to it.
>
>Fixes: 56bc3b2094b4 ("net/smc: assign link to a new connection")
>Signed-off-by: Wen Gu <[email protected]>
>---
>v1->v2:
>- Reset conn->lgr to NULL in smc_lgr_register_conn().
>- Only free new created link group.
>v2->v3:
>- Using __smc_lgr_terminate() instead of smc_lgr_schedule_free_work()
> for an immediate free.
>---
> net/smc/smc_core.c | 12 ++++++++++--
> 1 file changed, 10 insertions(+), 2 deletions(-)
>
>diff --git a/net/smc/smc_core.c b/net/smc/smc_core.c
>index 412bc85..0201f99 100644
>--- a/net/smc/smc_core.c
>+++ b/net/smc/smc_core.c
>@@ -171,8 +171,10 @@ static int smc_lgr_register_conn(struct smc_connection *conn, bool first)
>
> if (!conn->lgr->is_smcd) {
> rc = smcr_lgr_conn_assign_link(conn, first);
>- if (rc)
>+ if (rc) {
>+ conn->lgr = NULL;
> return rc;
>+ }
> }
> /* find a new alert_token_local value not yet used by some connection
> * in this link group
>@@ -1835,8 +1837,14 @@ int smc_conn_create(struct smc_sock *smc, struct smc_init_info *ini)
> write_lock_bh(&lgr->conns_lock);
> rc = smc_lgr_register_conn(conn, true);
> write_unlock_bh(&lgr->conns_lock);
>- if (rc)
>+ if (rc) {
>+ spin_lock_bh(lgr_lock);
>+ if (!list_empty(&lgr->list))
>+ list_del_init(&lgr->list);
>+ spin_unlock_bh(lgr_lock);
>+ __smc_lgr_terminate(lgr, true);
What about adding a smc_lgr_terminate() wrapper and put list_del_init()
and __smc_lgr_terminate() into it ?
> goto out;
>+ }
> }
> conn->local_tx_ctrl.common.type = SMC_CDC_MSG_TYPE;
> conn->local_tx_ctrl.len = SMC_WR_TX_SIZE;
>--
>1.8.3.1
Thanks for your suggestion.
On 2022/1/5 3:54 pm, dust.li wrote:
>> - if (rc)
>> + if (rc) {
>> + spin_lock_bh(lgr_lock);
>> + if (!list_empty(&lgr->list))
>> + list_del_init(&lgr->list);
>> + spin_unlock_bh(lgr_lock);
>> + __smc_lgr_terminate(lgr, true);
>
> What about adding a smc_lgr_terminate() wrapper and put list_del_init()
> and __smc_lgr_terminate() into it ?
Adding a new wrapper is a good idea. But I think the logic here is relatively simple.
So instead of wrapping them, I coded them like what smc_lgr_cleanup_early() does.
Thanks,
Wen Gu
>
>> goto out;
>> + }
>> }
>> conn->local_tx_ctrl.common.type = SMC_CDC_MSG_TYPE;
>> conn->local_tx_ctrl.len = SMC_WR_TX_SIZE;
>> --
>> 1.8.3.1
On 05/01/2022 09:55, Wen Gu wrote:
> On 2022/1/5 3:54 pm, dust.li wrote:
>
>>> - if (rc)
>>> + if (rc) {
>>> + spin_lock_bh(lgr_lock);
>>> + if (!list_empty(&lgr->list))
>>> + list_del_init(&lgr->list);
>>> + spin_unlock_bh(lgr_lock);
>>> + __smc_lgr_terminate(lgr, true);
>>
>> What about adding a smc_lgr_terminate() wrapper and put list_del_init()
>> and __smc_lgr_terminate() into it ?
>
> Adding a new wrapper is a good idea. But I think the logic here is relatively simple.
> So instead of wrapping them, I coded them like what smc_lgr_cleanup_early() does.
It might look cleaner with the following changes:
- adopt smc_lgr_cleanup_early() to take only an lgr as parameter and remove the call to smc_conn_free()
- change smc_conn_abort() (which is the only caller of smc_lgr_cleanup_early() right now), always
call smc_conn_free() and if (local_first) additionally call smc_lgr_cleanup_early()
(hold a local copy of the lgr for this call)
- finally call smc_lgr_cleanup_early(lgr) from smc_conn_create()
This should be the same processing, but the smc_conn_free() is moved to smc_conn_abort() where
it looks to be a better place for this call. And smc_lgr_cleanup_early() takes only care of an lgr.
What do you think? Did I miss something?
Thanks for your suggestion.
On 2022/1/5 9:25 pm, Karsten Graul wrote:
> It might look cleaner with the following changes:
> - adopt smc_lgr_cleanup_early() to take only an lgr as parameter and remove the call to smc_conn_free()
> - change smc_conn_abort() (which is the only caller of smc_lgr_cleanup_early() right now), always
> call smc_conn_free() and if (local_first) additionally call smc_lgr_cleanup_early()
> (hold a local copy of the lgr for this call)
> - finally call smc_lgr_cleanup_early(lgr) from smc_conn_create()
>
> This should be the same processing, but the smc_conn_free() is moved to smc_conn_abort() where
> it looks to be a better place for this call. And smc_lgr_cleanup_early() takes only care of an lgr.
>
I think those are very good changes, making smc_lgr_cleanup_early() processing link group only and
more reusable.
> What do you think? Did I miss something?
I think it is better and complete. I will improve the patch and test it, then send a v4.
Thanks,
Wen Gu