Objects allocated by xen_blkif_alloc come from the 'blkif_cache' kmem
cache. This cache is destoyed when xen-blkif is unloaded so it is
necessary to wait for the deferred free routine used for such objects to
complete. This necessity was missed in commit 14855954f636 "xen-blkback:
allow module to be cleanly unloaded". This patch fixes the problem by
taking/releasing extra module references in xen_blkif_alloc/free()
respectively.
Signed-off-by: Paul Durrant <[email protected]>
---
Cc: Konrad Rzeszutek Wilk <[email protected]>
Cc: "Roger Pau Monné" <[email protected]>
Cc: Jens Axboe <[email protected]>
---
drivers/block/xen-blkback/xenbus.c | 10 ++++++++++
1 file changed, 10 insertions(+)
diff --git a/drivers/block/xen-blkback/xenbus.c b/drivers/block/xen-blkback/xenbus.c
index e8c5c54e1d26..59d576d27ca7 100644
--- a/drivers/block/xen-blkback/xenbus.c
+++ b/drivers/block/xen-blkback/xenbus.c
@@ -171,6 +171,15 @@ static struct xen_blkif *xen_blkif_alloc(domid_t domid)
blkif->domid = domid;
atomic_set(&blkif->refcnt, 1);
init_completion(&blkif->drain_complete);
+
+ /*
+ * Because freeing back to the cache may be deferred, it is not
+ * safe to unload the module (and hence destroy the cache) until
+ * this has completed. To prevent premature unloading, take an
+ * extra module reference here and release only when the object
+ * has been free back to the cache.
+ */
+ __module_get(THIS_MODULE);
INIT_WORK(&blkif->free_work, xen_blkif_deferred_free);
return blkif;
@@ -320,6 +329,7 @@ static void xen_blkif_free(struct xen_blkif *blkif)
/* Make sure everything is drained before shutting down */
kmem_cache_free(xen_blkif_cachep, blkif);
+ module_put(THIS_MODULE);
}
int __init xen_blkif_interface_init(void)
--
2.20.1
On Tue, Dec 10, 2019 at 02:53:05PM +0000, Paul Durrant wrote:
> Objects allocated by xen_blkif_alloc come from the 'blkif_cache' kmem
> cache. This cache is destoyed when xen-blkif is unloaded so it is
> necessary to wait for the deferred free routine used for such objects to
> complete. This necessity was missed in commit 14855954f636 "xen-blkback:
> allow module to be cleanly unloaded". This patch fixes the problem by
> taking/releasing extra module references in xen_blkif_alloc/free()
> respectively.
>
> Signed-off-by: Paul Durrant <[email protected]>
Reviewed-by: Roger Pau Monn? <[email protected]>
One nit below.
> ---
> Cc: Konrad Rzeszutek Wilk <[email protected]>
> Cc: "Roger Pau Monn?" <[email protected]>
> Cc: Jens Axboe <[email protected]>
> ---
> drivers/block/xen-blkback/xenbus.c | 10 ++++++++++
> 1 file changed, 10 insertions(+)
>
> diff --git a/drivers/block/xen-blkback/xenbus.c b/drivers/block/xen-blkback/xenbus.c
> index e8c5c54e1d26..59d576d27ca7 100644
> --- a/drivers/block/xen-blkback/xenbus.c
> +++ b/drivers/block/xen-blkback/xenbus.c
> @@ -171,6 +171,15 @@ static struct xen_blkif *xen_blkif_alloc(domid_t domid)
> blkif->domid = domid;
> atomic_set(&blkif->refcnt, 1);
> init_completion(&blkif->drain_complete);
> +
> + /*
> + * Because freeing back to the cache may be deferred, it is not
> + * safe to unload the module (and hence destroy the cache) until
> + * this has completed. To prevent premature unloading, take an
> + * extra module reference here and release only when the object
> + * has been free back to the cache.
^ freed
> + */
> + __module_get(THIS_MODULE);
> INIT_WORK(&blkif->free_work, xen_blkif_deferred_free);
>
> return blkif;
> @@ -320,6 +329,7 @@ static void xen_blkif_free(struct xen_blkif *blkif)
>
> /* Make sure everything is drained before shutting down */
> kmem_cache_free(xen_blkif_cachep, blkif);
> + module_put(THIS_MODULE);
> }
>
> int __init xen_blkif_interface_init(void)
> --
> 2.20.1
>
> -----Original Message-----
> From: Roger Pau Monn? <[email protected]>
> Sent: 11 December 2019 11:29
> To: Durrant, Paul <[email protected]>
> Cc: [email protected]; [email protected]; linux-
> [email protected]; Konrad Rzeszutek Wilk <[email protected]>;
> Jens Axboe <[email protected]>
> Subject: Re: [PATCH] xen-blkback: prevent premature module unload
>
> On Tue, Dec 10, 2019 at 02:53:05PM +0000, Paul Durrant wrote:
> > Objects allocated by xen_blkif_alloc come from the 'blkif_cache' kmem
> > cache. This cache is destoyed when xen-blkif is unloaded so it is
> > necessary to wait for the deferred free routine used for such objects to
> > complete. This necessity was missed in commit 14855954f636 "xen-blkback:
> > allow module to be cleanly unloaded". This patch fixes the problem by
> > taking/releasing extra module references in xen_blkif_alloc/free()
> > respectively.
> >
> > Signed-off-by: Paul Durrant <[email protected]>
>
> Reviewed-by: Roger Pau Monn? <[email protected]>
>
> One nit below.
>
> > ---
> > Cc: Konrad Rzeszutek Wilk <[email protected]>
> > Cc: "Roger Pau Monn?" <[email protected]>
> > Cc: Jens Axboe <[email protected]>
> > ---
> > drivers/block/xen-blkback/xenbus.c | 10 ++++++++++
> > 1 file changed, 10 insertions(+)
> >
> > diff --git a/drivers/block/xen-blkback/xenbus.c b/drivers/block/xen-
> blkback/xenbus.c
> > index e8c5c54e1d26..59d576d27ca7 100644
> > --- a/drivers/block/xen-blkback/xenbus.c
> > +++ b/drivers/block/xen-blkback/xenbus.c
> > @@ -171,6 +171,15 @@ static struct xen_blkif *xen_blkif_alloc(domid_t
> domid)
> > blkif->domid = domid;
> > atomic_set(&blkif->refcnt, 1);
> > init_completion(&blkif->drain_complete);
> > +
> > + /*
> > + * Because freeing back to the cache may be deferred, it is not
> > + * safe to unload the module (and hence destroy the cache) until
> > + * this has completed. To prevent premature unloading, take an
> > + * extra module reference here and release only when the object
> > + * has been free back to the cache.
> ^ freed
Oh yes. Can this be done on commit, or would you like me to send a v2?
Paul
> > + */
> > + __module_get(THIS_MODULE);
> > INIT_WORK(&blkif->free_work, xen_blkif_deferred_free);
> >
> > return blkif;
> > @@ -320,6 +329,7 @@ static void xen_blkif_free(struct xen_blkif *blkif)
> >
> > /* Make sure everything is drained before shutting down */
> > kmem_cache_free(xen_blkif_cachep, blkif);
> > + module_put(THIS_MODULE);
> > }
> >
> > int __init xen_blkif_interface_init(void)
> > --
> > 2.20.1
> >
On Wed, Dec 11, 2019 at 01:27:42PM +0000, Durrant, Paul wrote:
> > -----Original Message-----
> > From: Roger Pau Monn? <[email protected]>
> > Sent: 11 December 2019 11:29
> > To: Durrant, Paul <[email protected]>
> > Cc: [email protected]; [email protected]; linux-
> > [email protected]; Konrad Rzeszutek Wilk <[email protected]>;
> > Jens Axboe <[email protected]>
> > Subject: Re: [PATCH] xen-blkback: prevent premature module unload
> >
> > On Tue, Dec 10, 2019 at 02:53:05PM +0000, Paul Durrant wrote:
> > > Objects allocated by xen_blkif_alloc come from the 'blkif_cache' kmem
> > > cache. This cache is destoyed when xen-blkif is unloaded so it is
> > > necessary to wait for the deferred free routine used for such objects to
> > > complete. This necessity was missed in commit 14855954f636 "xen-blkback:
> > > allow module to be cleanly unloaded". This patch fixes the problem by
> > > taking/releasing extra module references in xen_blkif_alloc/free()
> > > respectively.
> > >
> > > Signed-off-by: Paul Durrant <[email protected]>
> >
> > Reviewed-by: Roger Pau Monn? <[email protected]>
> >
> > One nit below.
> >
> > > ---
> > > Cc: Konrad Rzeszutek Wilk <[email protected]>
> > > Cc: "Roger Pau Monn?" <[email protected]>
> > > Cc: Jens Axboe <[email protected]>
> > > ---
> > > drivers/block/xen-blkback/xenbus.c | 10 ++++++++++
> > > 1 file changed, 10 insertions(+)
> > >
> > > diff --git a/drivers/block/xen-blkback/xenbus.c b/drivers/block/xen-
> > blkback/xenbus.c
> > > index e8c5c54e1d26..59d576d27ca7 100644
> > > --- a/drivers/block/xen-blkback/xenbus.c
> > > +++ b/drivers/block/xen-blkback/xenbus.c
> > > @@ -171,6 +171,15 @@ static struct xen_blkif *xen_blkif_alloc(domid_t
> > domid)
> > > blkif->domid = domid;
> > > atomic_set(&blkif->refcnt, 1);
> > > init_completion(&blkif->drain_complete);
> > > +
> > > + /*
> > > + * Because freeing back to the cache may be deferred, it is not
> > > + * safe to unload the module (and hence destroy the cache) until
> > > + * this has completed. To prevent premature unloading, take an
> > > + * extra module reference here and release only when the object
> > > + * has been free back to the cache.
> > ^ freed
>
> Oh yes. Can this be done on commit, or would you like me to send a v2?
Adjusting on commit would be fine for me, but it's up to Juergen since
he is the one that will pick this up. IIRC the module unload patches
didn't go through the block subsystem.
Thanks, Roger.
On 11.12.19 14:55, Roger Pau Monné wrote:
> On Wed, Dec 11, 2019 at 01:27:42PM +0000, Durrant, Paul wrote:
>>> -----Original Message-----
>>> From: Roger Pau Monné <[email protected]>
>>> Sent: 11 December 2019 11:29
>>> To: Durrant, Paul <[email protected]>
>>> Cc: [email protected]; [email protected]; linux-
>>> [email protected]; Konrad Rzeszutek Wilk <[email protected]>;
>>> Jens Axboe <[email protected]>
>>> Subject: Re: [PATCH] xen-blkback: prevent premature module unload
>>>
>>> On Tue, Dec 10, 2019 at 02:53:05PM +0000, Paul Durrant wrote:
>>>> Objects allocated by xen_blkif_alloc come from the 'blkif_cache' kmem
>>>> cache. This cache is destoyed when xen-blkif is unloaded so it is
>>>> necessary to wait for the deferred free routine used for such objects to
>>>> complete. This necessity was missed in commit 14855954f636 "xen-blkback:
>>>> allow module to be cleanly unloaded". This patch fixes the problem by
>>>> taking/releasing extra module references in xen_blkif_alloc/free()
>>>> respectively.
>>>>
>>>> Signed-off-by: Paul Durrant <[email protected]>
>>>
>>> Reviewed-by: Roger Pau Monné <[email protected]>
>>>
>>> One nit below.
>>>
>>>> ---
>>>> Cc: Konrad Rzeszutek Wilk <[email protected]>
>>>> Cc: "Roger Pau Monné" <[email protected]>
>>>> Cc: Jens Axboe <[email protected]>
>>>> ---
>>>> drivers/block/xen-blkback/xenbus.c | 10 ++++++++++
>>>> 1 file changed, 10 insertions(+)
>>>>
>>>> diff --git a/drivers/block/xen-blkback/xenbus.c b/drivers/block/xen-
>>> blkback/xenbus.c
>>>> index e8c5c54e1d26..59d576d27ca7 100644
>>>> --- a/drivers/block/xen-blkback/xenbus.c
>>>> +++ b/drivers/block/xen-blkback/xenbus.c
>>>> @@ -171,6 +171,15 @@ static struct xen_blkif *xen_blkif_alloc(domid_t
>>> domid)
>>>> blkif->domid = domid;
>>>> atomic_set(&blkif->refcnt, 1);
>>>> init_completion(&blkif->drain_complete);
>>>> +
>>>> + /*
>>>> + * Because freeing back to the cache may be deferred, it is not
>>>> + * safe to unload the module (and hence destroy the cache) until
>>>> + * this has completed. To prevent premature unloading, take an
>>>> + * extra module reference here and release only when the object
>>>> + * has been free back to the cache.
>>> ^ freed
>>
>> Oh yes. Can this be done on commit, or would you like me to send a v2?
>
> Adjusting on commit would be fine for me, but it's up to Juergen since
> he is the one that will pick this up. IIRC the module unload patches
> didn't go through the block subsystem.
Oh, right. Yes, will fix this when committing.
Juergen
> -----Original Message-----
> From: Roger Pau Monn? <[email protected]>
> Sent: 11 December 2019 13:55
> To: Durrant, Paul <[email protected]>; Juergen Gross <[email protected]>
> Cc: [email protected]; [email protected]; linux-
> [email protected]; Konrad Rzeszutek Wilk <[email protected]>;
> Jens Axboe <[email protected]>
> Subject: Re: [PATCH] xen-blkback: prevent premature module unload
>
> On Wed, Dec 11, 2019 at 01:27:42PM +0000, Durrant, Paul wrote:
> > > -----Original Message-----
> > > From: Roger Pau Monn? <[email protected]>
> > > Sent: 11 December 2019 11:29
> > > To: Durrant, Paul <[email protected]>
> > > Cc: [email protected]; [email protected];
> linux-
> > > [email protected]; Konrad Rzeszutek Wilk
> <[email protected]>;
> > > Jens Axboe <[email protected]>
> > > Subject: Re: [PATCH] xen-blkback: prevent premature module unload
> > >
> > > On Tue, Dec 10, 2019 at 02:53:05PM +0000, Paul Durrant wrote:
> > > > Objects allocated by xen_blkif_alloc come from the 'blkif_cache'
> kmem
> > > > cache. This cache is destoyed when xen-blkif is unloaded so it is
> > > > necessary to wait for the deferred free routine used for such
> objects to
> > > > complete. This necessity was missed in commit 14855954f636 "xen-
> blkback:
> > > > allow module to be cleanly unloaded". This patch fixes the problem
> by
> > > > taking/releasing extra module references in xen_blkif_alloc/free()
> > > > respectively.
> > > >
> > > > Signed-off-by: Paul Durrant <[email protected]>
> > >
> > > Reviewed-by: Roger Pau Monn? <[email protected]>
> > >
> > > One nit below.
> > >
> > > > ---
> > > > Cc: Konrad Rzeszutek Wilk <[email protected]>
> > > > Cc: "Roger Pau Monn?" <[email protected]>
> > > > Cc: Jens Axboe <[email protected]>
> > > > ---
> > > > drivers/block/xen-blkback/xenbus.c | 10 ++++++++++
> > > > 1 file changed, 10 insertions(+)
> > > >
> > > > diff --git a/drivers/block/xen-blkback/xenbus.c b/drivers/block/xen-
> > > blkback/xenbus.c
> > > > index e8c5c54e1d26..59d576d27ca7 100644
> > > > --- a/drivers/block/xen-blkback/xenbus.c
> > > > +++ b/drivers/block/xen-blkback/xenbus.c
> > > > @@ -171,6 +171,15 @@ static struct xen_blkif
> *xen_blkif_alloc(domid_t
> > > domid)
> > > > blkif->domid = domid;
> > > > atomic_set(&blkif->refcnt, 1);
> > > > init_completion(&blkif->drain_complete);
> > > > +
> > > > + /*
> > > > + * Because freeing back to the cache may be deferred, it is
> not
> > > > + * safe to unload the module (and hence destroy the cache)
> until
> > > > + * this has completed. To prevent premature unloading, take an
> > > > + * extra module reference here and release only when the
> object
> > > > + * has been free back to the cache.
> > > ^ freed
> >
> > Oh yes. Can this be done on commit, or would you like me to send a v2?
>
> Adjusting on commit would be fine for me, but it's up to Juergen since
> he is the one that will pick this up. IIRC the module unload patches
> didn't go through the block subsystem.
True. I forgot manually add Juergen cc list.
Paul
>
> Thanks, Roger.
On 10.12.19 15:53, Paul Durrant wrote:
> Objects allocated by xen_blkif_alloc come from the 'blkif_cache' kmem
> cache. This cache is destoyed when xen-blkif is unloaded so it is
> necessary to wait for the deferred free routine used for such objects to
> complete. This necessity was missed in commit 14855954f636 "xen-blkback:
> allow module to be cleanly unloaded". This patch fixes the problem by
> taking/releasing extra module references in xen_blkif_alloc/free()
> respectively.
>
> Signed-off-by: Paul Durrant <[email protected]>
Pushed to xen/tip.git for-linus-5.5b
Juergen