A virtio_blk block device can still be referenced after hot unplug by
userspace processes that hold the file descriptor. In this case
virtblk_getgeo() can be invoked after virtblk_remove() was called. For
example, a program that has /dev/vdb open can call ioctl(HDIO_GETGEO)
after hot unplug.
Fix this by clearing vblk->disk->private_data and checking that the
virtio_blk driver instance is still around in virtblk_getgeo().
Note that the virtblk_getgeo() function itself is guaranteed to remain
in memory after hot unplug because the virtio_blk module refcount is
still held while a block device reference exists.
Originally-by: Lance Digby <[email protected]>
Signed-off-by: Stefan Hajnoczi <[email protected]>
---
drivers/block/virtio_blk.c | 5 +++++
1 file changed, 5 insertions(+)
diff --git a/drivers/block/virtio_blk.c b/drivers/block/virtio_blk.c
index 93468b7c6701..b50cdf37a6f7 100644
--- a/drivers/block/virtio_blk.c
+++ b/drivers/block/virtio_blk.c
@@ -300,6 +300,10 @@ static int virtblk_getgeo(struct block_device *bd, struct hd_geometry *geo)
{
struct virtio_blk *vblk = bd->bd_disk->private_data;
+ /* Driver instance has been removed */
+ if (!vblk)
+ return -ENOTTY;
+
/* see if the host passed in geometry config */
if (virtio_has_feature(vblk->vdev, VIRTIO_BLK_F_GEOMETRY)) {
virtio_cread(vblk->vdev, struct virtio_blk_config,
@@ -835,6 +839,7 @@ static void virtblk_remove(struct virtio_device *vdev)
vdev->config->reset(vdev);
refc = kref_read(&disk_to_dev(vblk->disk)->kobj.kref);
+ vblk->disk->private_data = NULL;
put_disk(vblk->disk);
vdev->config->del_vqs(vdev);
kfree(vblk->vqs);
--
2.25.1
On Thu, Apr 23, 2020 at 01:37:17PM +0100, Stefan Hajnoczi wrote:
> A virtio_blk block device can still be referenced after hot unplug by
> userspace processes that hold the file descriptor. In this case
> virtblk_getgeo() can be invoked after virtblk_remove() was called. For
> example, a program that has /dev/vdb open can call ioctl(HDIO_GETGEO)
> after hot unplug.
>
> Fix this by clearing vblk->disk->private_data and checking that the
> virtio_blk driver instance is still around in virtblk_getgeo().
>
> Note that the virtblk_getgeo() function itself is guaranteed to remain
> in memory after hot unplug because the virtio_blk module refcount is
> still held while a block device reference exists.
>
> Originally-by: Lance Digby <[email protected]>
> Signed-off-by: Stefan Hajnoczi <[email protected]>
> ---
> drivers/block/virtio_blk.c | 5 +++++
> 1 file changed, 5 insertions(+)
>
> diff --git a/drivers/block/virtio_blk.c b/drivers/block/virtio_blk.c
> index 93468b7c6701..b50cdf37a6f7 100644
> --- a/drivers/block/virtio_blk.c
> +++ b/drivers/block/virtio_blk.c
> @@ -300,6 +300,10 @@ static int virtblk_getgeo(struct block_device *bd, struct hd_geometry *geo)
> {
> struct virtio_blk *vblk = bd->bd_disk->private_data;
>
> + /* Driver instance has been removed */
> + if (!vblk)
> + return -ENOTTY;
> +
> /* see if the host passed in geometry config */
> if (virtio_has_feature(vblk->vdev, VIRTIO_BLK_F_GEOMETRY)) {
> virtio_cread(vblk->vdev, struct virtio_blk_config,
Just so I understand, what serializes this access?
See below for what looks like a race condition ...
> @@ -835,6 +839,7 @@ static void virtblk_remove(struct virtio_device *vdev)
> vdev->config->reset(vdev);
>
> refc = kref_read(&disk_to_dev(vblk->disk)->kobj.kref);
So what if private_data is tested at this time ...
> + vblk->disk->private_data = NULL;
> put_disk(vblk->disk);
> vdev->config->del_vqs(vdev);
> kfree(vblk->vqs);
... and then used at this time?
What prevents this?
> --
> 2.25.1
>
On Thu, Apr 23, 2020 at 01:37:17PM +0100, Stefan Hajnoczi wrote:
> A virtio_blk block device can still be referenced after hot unplug by
> userspace processes that hold the file descriptor. In this case
> virtblk_getgeo() can be invoked after virtblk_remove() was called. For
> example, a program that has /dev/vdb open can call ioctl(HDIO_GETGEO)
> after hot unplug.
>
> Fix this by clearing vblk->disk->private_data and checking that the
> virtio_blk driver instance is still around in virtblk_getgeo().
>
> Note that the virtblk_getgeo() function itself is guaranteed to remain
> in memory after hot unplug because the virtio_blk module refcount is
> still held while a block device reference exists.
>
> Originally-by: Lance Digby <[email protected]>
> Signed-off-by: Stefan Hajnoczi <[email protected]>
> ---
> drivers/block/virtio_blk.c | 5 +++++
> 1 file changed, 5 insertions(+)
>
> diff --git a/drivers/block/virtio_blk.c b/drivers/block/virtio_blk.c
> index 93468b7c6701..b50cdf37a6f7 100644
> --- a/drivers/block/virtio_blk.c
> +++ b/drivers/block/virtio_blk.c
> @@ -300,6 +300,10 @@ static int virtblk_getgeo(struct block_device *bd, struct hd_geometry *geo)
> {
> struct virtio_blk *vblk = bd->bd_disk->private_data;
>
> + /* Driver instance has been removed */
> + if (!vblk)
> + return -ENOTTY;
If this ever hits you have a nasty race condition and this is not
going to fix it, as it could be removed just after this check as well.
On Thu, Apr 23, 2020 at 01:37:17PM +0100, Stefan Hajnoczi wrote:
> A virtio_blk block device can still be referenced after hot unplug by
> userspace processes that hold the file descriptor. In this case
> virtblk_getgeo() can be invoked after virtblk_remove() was called. For
> example, a program that has /dev/vdb open can call ioctl(HDIO_GETGEO)
> after hot unplug.
>
> Fix this by clearing vblk->disk->private_data and checking that the
> virtio_blk driver instance is still around in virtblk_getgeo().
>
> Note that the virtblk_getgeo() function itself is guaranteed to remain
> in memory after hot unplug because the virtio_blk module refcount is
> still held while a block device reference exists.
>
> Originally-by: Lance Digby <[email protected]>
> Signed-off-by: Stefan Hajnoczi <[email protected]>
> ---
> drivers/block/virtio_blk.c | 5 +++++
> 1 file changed, 5 insertions(+)
>
> diff --git a/drivers/block/virtio_blk.c b/drivers/block/virtio_blk.c
> index 93468b7c6701..b50cdf37a6f7 100644
> --- a/drivers/block/virtio_blk.c
> +++ b/drivers/block/virtio_blk.c
> @@ -300,6 +300,10 @@ static int virtblk_getgeo(struct block_device *bd, struct hd_geometry *geo)
> {
> struct virtio_blk *vblk = bd->bd_disk->private_data;
>
> + /* Driver instance has been removed */
> + if (!vblk)
> + return -ENOTTY;
> +
> /* see if the host passed in geometry config */
> if (virtio_has_feature(vblk->vdev, VIRTIO_BLK_F_GEOMETRY)) {
> virtio_cread(vblk->vdev, struct virtio_blk_config,
> @@ -835,6 +839,7 @@ static void virtblk_remove(struct virtio_device *vdev)
> vdev->config->reset(vdev);
>
> refc = kref_read(&disk_to_dev(vblk->disk)->kobj.kref);
> + vblk->disk->private_data = NULL;
> put_disk(vblk->disk);
> vdev->config->del_vqs(vdev);
> kfree(vblk->vqs);
As pointed out, can be a race. We had a very similar issue in
virtio-vsock, and we solved using RCU to assign and get the pointer [1],
maybe the same solution can work here.
Cheers,
Stefano
[1] 0deab087b16a vsock/virtio: use RCU to avoid use-after-free on the_virtio_vsock
On Thu, Apr 23, 2020 at 07:13:38AM -0700, Christoph Hellwig wrote:
> On Thu, Apr 23, 2020 at 01:37:17PM +0100, Stefan Hajnoczi wrote:
> > A virtio_blk block device can still be referenced after hot unplug by
> > userspace processes that hold the file descriptor. In this case
> > virtblk_getgeo() can be invoked after virtblk_remove() was called. For
> > example, a program that has /dev/vdb open can call ioctl(HDIO_GETGEO)
> > after hot unplug.
> >
> > Fix this by clearing vblk->disk->private_data and checking that the
> > virtio_blk driver instance is still around in virtblk_getgeo().
> >
> > Note that the virtblk_getgeo() function itself is guaranteed to remain
> > in memory after hot unplug because the virtio_blk module refcount is
> > still held while a block device reference exists.
> >
> > Originally-by: Lance Digby <[email protected]>
> > Signed-off-by: Stefan Hajnoczi <[email protected]>
> > ---
> > drivers/block/virtio_blk.c | 5 +++++
> > 1 file changed, 5 insertions(+)
> >
> > diff --git a/drivers/block/virtio_blk.c b/drivers/block/virtio_blk.c
> > index 93468b7c6701..b50cdf37a6f7 100644
> > --- a/drivers/block/virtio_blk.c
> > +++ b/drivers/block/virtio_blk.c
> > @@ -300,6 +300,10 @@ static int virtblk_getgeo(struct block_device *bd, struct hd_geometry *geo)
> > {
> > struct virtio_blk *vblk = bd->bd_disk->private_data;
> >
> > + /* Driver instance has been removed */
> > + if (!vblk)
> > + return -ENOTTY;
>
> If this ever hits you have a nasty race condition and this is not
> going to fix it, as it could be removed just after this check as well.
Perhaps a better fix is to keep a refcount in struct virtio_blk and
implement block_device_operations->release() to decrement the refcount.
That way the struct virtio_blk stays around until all block device
references are dropped. This is similar to nbd_put() in
drivers/block/nbd.c.
Stefan