Received-SPF: pass (google.com: domain of linux-kernel-owner@vger.kernel.org designates 2620:137:e000::1:20 as permitted sender) client-ip=2620:137:e000::1:20;
From:   "Marc Hartmayer" <mhartmay@linux.ibm.com>
To:     Yu Kuai <yukuai1@huaweicloud.com>
Cc:     linux-scsi@vger.kernel.org, linux-kernel@vger.kernel.org,
        yi.zhang@huawei.com, yangerkun@huawei.com, hch@lst.de,
        chaitanyak@nvidia.com, shinichiro.kawasaki@wdc.com,
        dgilbert@interlog.com, jejb@linux.ibm.com,
        martin.petersen@oracle.com, axboe@kernel.dk,
        Benjamin Block <bblock@linux.ibm.com>,
        "yukuai (C)" <yukuai3@huawei.com>
Subject: Re: [PATCH] scsi/sg: don't grab scsi host module reference
In-Reply-To: <1efb3d4d-a05b-2eb2-a140-a76981b662a4@huaweicloud.com>
References: <20230621160111.1433521-1-yukuai1@huaweicloud.com>
 <87lefv622n.fsf@linux.ibm.com>
 <70432d91-3909-ac3c-9c36-5f7484c1fdf1@huaweicloud.com>
 <1efb3d4d-a05b-2eb2-a140-a76981b662a4@huaweicloud.com>
Date:   Wed, 05 Jul 2023 08:13:08 +0200
Message-ID: <87bkgqkhsb.fsf@linux.ibm.com>
Content-Type: text/plain; charset=utf-8
Content-Transfer-Encoding: quoted-printable
MIME-Version: 1.0
Precedence: bulk

On Wed, Jul 05, 2023 at 10:28 AM +0800, Yu Kuai <yukuai1@huaweicloud.com> w=
rote:
> Hi,
>
> =E5=9C=A8 2023/07/05 9:43, Yu Kuai =E5=86=99=E9=81=93:
>> Hi,
>>=20
>> =E5=9C=A8 2023/07/05 1:04, Marc Hartmayer =E5=86=99=E9=81=93:
>>> On Thu, Jun 22, 2023 at 12:01 AM +0800, Yu Kuai=20
>>> <yukuai1@huaweicloud.com> wrote:
>>>> From: Yu Kuai <yukuai3@huawei.com>
>>>>
>>>> In order to prevent request_queue to be freed before cleaning up
>>>> blktrace debugfs entries, commit db59133e9279 ("scsi: sg: fix blktrace
>>>> debugfs entries leakage") use scsi_device_get(), however,
>>>> scsi_device_get() will also grab scsi module reference and scsi module
>>>> can't be removed.
>>>>
>>>> It's reported that blktests can't unload scsi_debug after block/001:
>>>>
>>>> blktests (master) # ./check block
>>>> block/001 (stress device hotplugging) [failed]
>>>> =C2=A0=C2=A0=C2=A0=C2=A0=C2=A0 +++ /root/blktests/results/nodev/block/=
001.out.bad 2023-06-19
>>>> =C2=A0=C2=A0=C2=A0=C2=A0=C2=A0=C2=A0 Running block/001
>>>> =C2=A0=C2=A0=C2=A0=C2=A0=C2=A0=C2=A0 Stressing sd
>>>> =C2=A0=C2=A0=C2=A0=C2=A0=C2=A0 +modprobe: FATAL: Module scsi_debug is =
in use.
>>>>
>>>> Fix this problem by grabbing request_queue reference directly, so that
>>>> scsi host module can still be unloaded while request_queue will be
>>>> pinged by sg device.
>>>>
>>>> Reported-by: Chaitanya Kulkarni <chaitanyak@nvidia.com>
>>>> Link:=20
>>>> https://lore.kernel.org/all/1760da91-876d-fc9c-ab51-999a6f66ad50@nvidi=
a.com/=20
>>>>
>>>> Fixes: db59133e9279 ("scsi: sg: fix blktrace debugfs entries leakage")
>>>> Signed-off-by: Yu Kuai <yukuai3@huawei.com>
>>>> ---
>>>> =C2=A0 drivers/scsi/sg.c | 6 +++---
>>>> =C2=A0 1 file changed, 3 insertions(+), 3 deletions(-)
>>>>
>>>> diff --git a/drivers/scsi/sg.c b/drivers/scsi/sg.c
>>>> index 2433eeef042a..dcb73787c29d 100644
>>>> --- a/drivers/scsi/sg.c
>>>> +++ b/drivers/scsi/sg.c
>>>> @@ -1497,7 +1497,7 @@ sg_add_device(struct device *cl_dev)
>>>> =C2=A0=C2=A0=C2=A0=C2=A0=C2=A0 int error;
>>>> =C2=A0=C2=A0=C2=A0=C2=A0=C2=A0 unsigned long iflags;
>>>> -=C2=A0=C2=A0=C2=A0 error =3D scsi_device_get(scsidp);
>>>> +=C2=A0=C2=A0=C2=A0 error =3D blk_get_queue(scsidp->request_queue);
>>>> =C2=A0=C2=A0=C2=A0=C2=A0=C2=A0 if (error)
>>>> =C2=A0=C2=A0=C2=A0=C2=A0=C2=A0=C2=A0=C2=A0=C2=A0=C2=A0 return error;
>>>> @@ -1558,7 +1558,7 @@ sg_add_device(struct device *cl_dev)
>>>> =C2=A0 out:
>>>> =C2=A0=C2=A0=C2=A0=C2=A0=C2=A0 if (cdev)
>>>> =C2=A0=C2=A0=C2=A0=C2=A0=C2=A0=C2=A0=C2=A0=C2=A0=C2=A0 cdev_del(cdev);
>>>> -=C2=A0=C2=A0=C2=A0 scsi_device_put(scsidp);
>>>> +=C2=A0=C2=A0=C2=A0 blk_put_queue(scsidp->request_queue);
>>>> =C2=A0=C2=A0=C2=A0=C2=A0=C2=A0 return error;
>>>> =C2=A0 }
>>>> @@ -1575,7 +1575,7 @@ sg_device_destroy(struct kref *kref)
>>>> =C2=A0=C2=A0=C2=A0=C2=A0=C2=A0=C2=A0 */
>>>> =C2=A0=C2=A0=C2=A0=C2=A0=C2=A0 blk_trace_remove(q);
>>>> -=C2=A0=C2=A0=C2=A0 scsi_device_put(sdp->device);
>>>> +=C2=A0=C2=A0=C2=A0 blk_put_queue(q);
>>>> =C2=A0=C2=A0=C2=A0=C2=A0=C2=A0 write_lock_irqsave(&sg_index_lock, flag=
s);
>>>> =C2=A0=C2=A0=C2=A0=C2=A0=C2=A0 idr_remove(&sg_index_idr, sdp->index);
>>>> --=20
>>>> 2.39.2
>>>
>>> Hi,
>>>
>>> This change (bisected) triggers a regression in our KVM on s390x CI. The
>>> symptom is that a =E2=80=9Cscsi_debug device=E2=80=9D does not bind to =
the scsi_generic
>>> driver. On s390x you can reproduce the problem as follows (I have not
>>> tested on x86):
>>>
>>> With this patch applied:
>>>
>>> $ sudo modprobe scsi_debug
>>> $ # Get the 'scsi_host,channel,target_number,LUN' tuple for the=20
>>> scsi_debug device
>>> $ lsscsi |grep scsi_debug |awk '{ print $1 }'
>>> [0:0:0:0]
>>> $ sudo stat /sys/bus/scsi/devices/0:0:0:0/scsi_generic
>>> stat: cannot statx '/sys/bus/scsi/devices/0:0:0:0/scsi_generic': No=20
>>> such file or directory
>>>
>>>
>>> Patch reverted:
>>>
>>=20
>> I didn't figure out the root cause, howver, have you tried to reviert
>> this patch as well?
>>=20
>> db59133e9279 ("scsi: sg: fix blktrace debugfs entries leakage"
>
> Never mind this, root cause is that the checking of return value of
> blk_get_queue() is wrong.
>
> This shoud be fixed by following patch:
>
> diff --git a/drivers/scsi/sg.c b/drivers/scsi/sg.c
> index 89fa046c7158..0d8afffd1683 100644
> --- a/drivers/scsi/sg.c
> +++ b/drivers/scsi/sg.c
> @@ -1497,9 +1497,10 @@ sg_add_device(struct device *cl_dev)
>          int error;
>          unsigned long iflags;
>
> -       error =3D blk_get_queue(scsidp->request_queue);
> -       if (error)
> -               return error;
> +       if (!blk_get_queue(scsidp->request_queue)) {
> +               pr_warn("%s: get scsi_device queue failed\n", __func__);
> +               return -ENODEV;
> +       }

Hi Kuai,

I just tried your fix and it works - thanks.

Tested-by: Marc Hartmayer <mhartmay@linux.ibm.com>

 Marc

[=E2=80=A6snip]