BUG: KASAN: double-free or invalid-free in nvme_rdma_free_queue+0xf6/0x110 [nvme_rdma]
Workqueue: nvme-reset-wq nvme_rdma_reset_ctrl_work [nvme_rdma]
Call Trace:
dump_stack+0x91/0xeb
print_address_description+0x6b/0x290
kasan_report_invalid_free+0x55/0x80
__kasan_slab_free+0x176/0x190
kfree+0xeb/0x310
nvme_rdma_free_queue+0xf6/0x110 [nvme_rdma]
nvme_rdma_configure_admin_queue+0x1a3/0x4d0 [nvme_rdma]
nvme_rdma_reset_ctrl_work+0x4e/0xd0 [nvme_rdma]
process_one_work+0x3ca/0xaa0
worker_thread+0x4e2/0x6c0
kthread+0x18d/0x1e0
ret_from_fork+0x24/0x30
The double free is on ctrl->async_event_sqe.
If any case fails before ctrl->async_event_sqe is allocated in
nvme_rdma_configure_admin_queue, nvme_rdma_free_queue will be
invoked. However, at the moment, the ctrl->async_event_sqe has
not been allocated because it has been freed in
nvme_rdma_reset_ctrl_work
-> nvme_rdma_shutdown_ctrl
->nvme_rdma_destroy_admin_queue
-> nvme_rdma_free_queue
Signed-off-by: Jianchao Wang <[email protected]>
---
V2:
handle it in nvme_rdma_free_queue and add some comment to explain it.
drivers/nvme/host/rdma.c | 15 ++++++++++++---
1 file changed, 12 insertions(+), 3 deletions(-)
diff --git a/drivers/nvme/host/rdma.c b/drivers/nvme/host/rdma.c
index 966e0dd..fa5cf87 100644
--- a/drivers/nvme/host/rdma.c
+++ b/drivers/nvme/host/rdma.c
@@ -561,9 +561,18 @@ static void nvme_rdma_free_queue(struct nvme_rdma_queue *queue)
return;
if (nvme_rdma_queue_idx(queue) == 0) {
- nvme_rdma_free_qe(queue->device->dev,
- &queue->ctrl->async_event_sqe,
- sizeof(struct nvme_command), DMA_TO_DEVICE);
+ /*
+ * async_event_sqe is not allocated in nvme_rdma_alloc_queue.
+ * so there are cases where NVME_RDMA_Q_ALLOCATED is set, but
+ * async_event_sqe is not allocated. To avoid double free, set
+ * async_event_sqe.data to NULL to indicate it has been freed.
+ */
+ if (queue->ctrl->async_event_sqe.data) {
+ nvme_rdma_free_qe(queue->device->dev,
+ &queue->ctrl->async_event_sqe,
+ sizeof(struct nvme_command), DMA_TO_DEVICE);
+ queue->ctrl->async_event_sqe.data = NULL;
+ }
}
nvme_rdma_destroy_queue_ib(queue);
--
2.7.4
Hi Jianchao,
On 5/10/2018 12:42 PM, Jianchao Wang wrote:
> BUG: KASAN: double-free or invalid-free in nvme_rdma_free_queue+0xf6/0x110 [nvme_rdma]
> Workqueue: nvme-reset-wq nvme_rdma_reset_ctrl_work [nvme_rdma]
> Call Trace:
> dump_stack+0x91/0xeb
> print_address_description+0x6b/0x290
> kasan_report_invalid_free+0x55/0x80
> __kasan_slab_free+0x176/0x190
> kfree+0xeb/0x310
> nvme_rdma_free_queue+0xf6/0x110 [nvme_rdma]
> nvme_rdma_configure_admin_queue+0x1a3/0x4d0 [nvme_rdma]
> nvme_rdma_reset_ctrl_work+0x4e/0xd0 [nvme_rdma]
> process_one_work+0x3ca/0xaa0
> worker_thread+0x4e2/0x6c0
> kthread+0x18d/0x1e0
> ret_from_fork+0x24/0x30
>
> The double free is on ctrl->async_event_sqe.
> If any case fails before ctrl->async_event_sqe is allocated in
> nvme_rdma_configure_admin_queue, nvme_rdma_free_queue will be
> invoked. However, at the moment, the ctrl->async_event_sqe has
> not been allocated because it has been freed in
> nvme_rdma_reset_ctrl_work
> -> nvme_rdma_shutdown_ctrl
> ->nvme_rdma_destroy_admin_queue
> -> nvme_rdma_free_queue
>
> Signed-off-by: Jianchao Wang <[email protected]>
> ---
>
> V2:
> handle it in nvme_rdma_free_queue and add some comment to explain it.
I don't know exactly what Christoph meant but IMO the best place to
allocate it is in nvme_rdma_alloc_queue just before calling
"set_bit(NVME_RDMA_Q_ALLOCATED, &queue->flags);"
then you will never get to double free since we clear the
NVME_RDMA_Q_ALLOCATED bit in the beginning of nvme_rdma_free_queue.
>
> drivers/nvme/host/rdma.c | 15 ++++++++++++---
> 1 file changed, 12 insertions(+), 3 deletions(-)
>
> diff --git a/drivers/nvme/host/rdma.c b/drivers/nvme/host/rdma.c
> index 966e0dd..fa5cf87 100644
> --- a/drivers/nvme/host/rdma.c
> +++ b/drivers/nvme/host/rdma.c
> @@ -561,9 +561,18 @@ static void nvme_rdma_free_queue(struct nvme_rdma_queue *queue)
> return;
>
> if (nvme_rdma_queue_idx(queue) == 0) {
> - nvme_rdma_free_qe(queue->device->dev,
> - &queue->ctrl->async_event_sqe,
> - sizeof(struct nvme_command), DMA_TO_DEVICE);
> + /*
> + * async_event_sqe is not allocated in nvme_rdma_alloc_queue.
> + * so there are cases where NVME_RDMA_Q_ALLOCATED is set, but
> + * async_event_sqe is not allocated. To avoid double free, set
> + * async_event_sqe.data to NULL to indicate it has been freed.
> + */
> + if (queue->ctrl->async_event_sqe.data) {
> + nvme_rdma_free_qe(queue->device->dev,
> + &queue->ctrl->async_event_sqe,
> + sizeof(struct nvme_command), DMA_TO_DEVICE);
> + queue->ctrl->async_event_sqe.data = NULL;
> + }
> }
>
> nvme_rdma_destroy_queue_ib(queue);
>
-Max.
Hi Max
Thanks for kindly review and suggestion for this.
On 05/16/2018 08:18 PM, Max Gurtovoy wrote:
> I don't know exactly what Christoph meant but IMO the best place to allocate it is in nvme_rdma_alloc_queue just before calling
>
> "set_bit(NVME_RDMA_Q_ALLOCATED, &queue->flags);"
>
> then you will never get to double free since we clear the NVME_RDMA_Q_ALLOCATED bit in the beginning of nvme_rdma_free_queue.
Yes, I will investigate the whether there is some special reason for async_event_sqe is not allocated in nvme_rdma_alloc_queue next.
Thanks
Jianchao