Received-SPF: pass (google.com: domain of linux-kernel-owner@vger.kernel.org designates 23.128.96.18 as permitted sender) client-ip=23.128.96.18;
IronPort-SDR: rrdr9IygSTSEzZ5XAnPYRq+lc9vkc0tVrPOE8QvNusbYRwLDq4933BNHo4qzJdPbOVCfwx/gLn
 jMcYmsOHw6iwCYfTxLR7weMP783jZ3Wy18GUuZiUTPVJYdok0neRt97DpgQ/CESfUPrpEFhQ2g
 YwSvh5sEZdGpKAAs0XY7ZMCRDD39YkDiTo9oLFJgr0GvtZn+EbKtqkwlgbqLXxozFM2gCCbZtj
 rB719gorhuoikoGBBJgjUNxSBH1+8UwkASaScgPtMVglOxa0I8mBeSgsRdaSnDox+E1ZQUdzV0
 yVE=
From:   Niklas Cassel <Niklas.Cassel@wdc.com>
To:     =?iso-8859-1?Q?Javier_Gonz=E1lez?= <javier@javigon.com>
CC:     Keith Busch <kbusch@kernel.org>, Jens Axboe <axboe@fb.com>,
        Christoph Hellwig <hch@lst.de>,
        Sagi Grimberg <sagi@grimberg.me>,
        Igor Konopko <igor.j.konopko@intel.com>,
        =?iso-8859-1?Q?Matias_Bj=F8rling?= <mb@lightnvm.io>,
        Jens Axboe <axboe@kernel.dk>,
        "linux-nvme@lists.infradead.org" <linux-nvme@lists.infradead.org>,
        "linux-kernel@vger.kernel.org" <linux-kernel@vger.kernel.org>
Subject: Re: [PATCH] nvme: prevent double free in nvme_alloc_ns() error
 handling
Thread-Topic: [PATCH] nvme: prevent double free in nvme_alloc_ns() error
 handling
Thread-Index: AQHWHJBF5J2JgQiUdUSwnibdYIeruaiNQxmAgAAFd4CAANV9gIAAC/GA
Date:   Tue, 28 Apr 2020 07:49:36 +0000
Message-ID: <20200428074935.GA584449@localhost.localdomain>
References: <20200427123443.520469-1-niklas.cassel@wdc.com>
 <20200427180311.nssquibbak5ib4oo@mpHalley.localdomain>
 <20200427182245.GA547726@localhost.localdomain>
 <20200428070651.qbsyivvaakflipr4@mpHalley.localdomain>
In-Reply-To: <20200428070651.qbsyivvaakflipr4@mpHalley.localdomain>
Accept-Language: en-US
Content-Language: en-US
wdcipoutbound: EOP-TRUE
Content-Type: text/plain; charset="iso-8859-1"
Content-ID: <DDB25662D9CD22469FC093E0DFCE4BAA@namprd04.prod.outlook.com>
Content-Transfer-Encoding: quoted-printable
MIME-Version: 1.0
X-MS-Exchange-CrossTenant-Network-Message-Id: 64964c7f-58dd-496d-f1de-08d7eb48b2e7
X-MS-Exchange-CrossTenant-originalarrivaltime: 28 Apr 2020 07:49:36.5466
 (UTC)
X-MS-Exchange-CrossTenant-fromentityheader: Hosted
X-MS-Exchange-CrossTenant-id: b61c8803-16f3-4c35-9b17-6f65f441df86
X-MS-Exchange-CrossTenant-mailboxtype: HOSTED
X-MS-Exchange-CrossTenant-userprincipalname: yEm3cxScZjmRrDQ5Nua1//dYK5P9HjcsP3u4GqJzxYOjT2UAQPCEAnQSYlNQqAA4YOv6BS3XHlFXKOa5awm3mw==
X-MS-Exchange-Transport-CrossTenantHeadersStamped: BYAPR04MB3782
Sender: linux-kernel-owner@vger.kernel.org
Precedence: bulk
List-ID: <linux-kernel.vger.kernel.org>
X-Mailing-List: linux-kernel@vger.kernel.org

On Tue, Apr 28, 2020 at 09:06:51AM +0200, Javier Gonz=E1lez wrote:
> CAUTION: This email originated from outside of Western Digital. Do not cl=
ick on links or open attachments unless you recognize the sender and know t=
hat the content is safe.
>=20
>=20
> On 27.04.2020 18:22, Niklas Cassel wrote:
> > On Mon, Apr 27, 2020 at 08:03:11PM +0200, Javier Gonz=E1lez wrote:
> > > On 27.04.2020 14:34, Niklas Cassel wrote:
> > > > When jumping to the out_put_disk label, we will call put_disk(), wh=
ich will
> > > > trigger a call to disk_release(), which calls blk_put_queue().
> > > >
> > > > Later in the cleanup code, we do blk_cleanup_queue(), which will al=
so call
> > > > blk_put_queue().
> > > >
> > > > Putting the queue twice is incorrect, and will generate a KASAN spl=
at.
> > > >
> > > > Set the disk->queue pointer to NULL, before calling put_disk(), so =
that the
> > > > first call to blk_put_queue() will not free the queue.
> > > >
> > > > The second call to blk_put_queue() uses another pointer to the same=
 queue,
> > > > so this call will still free the queue.
> > > >
> > > > Fixes: 85136c010285 ("lightnvm: simplify geometry enumeration")
> > > > Signed-off-by: Niklas Cassel <niklas.cassel@wdc.com>
> > > > ---
> > > > drivers/nvme/host/core.c | 2 ++
> > > > 1 file changed, 2 insertions(+)
> > > >
> > > > diff --git a/drivers/nvme/host/core.c b/drivers/nvme/host/core.c
> > > > index 91c1bd659947..f2adea96b04c 100644
> > > > --- a/drivers/nvme/host/core.c
> > > > +++ b/drivers/nvme/host/core.c
> > > > @@ -3642,6 +3642,8 @@ static void nvme_alloc_ns(struct nvme_ctrl *c=
trl, unsigned nsid)
> > > >
> > > >       return;
> > > >  out_put_disk:
> > > > +      /* prevent double queue cleanup */
> > > > +      ns->disk->queue =3D NULL;
> > > >       put_disk(ns->disk);
> > > >  out_unlink_ns:
> > > >       mutex_lock(&ctrl->subsys->lock);
> > > > --
> > > > 2.25.3
> > > >
> > > What about delaying the assignment of ns->disk?
> > >=20
> > > diff --git i/drivers/nvme/host/core.c w/drivers/nvme/host/core.c
> > > index a4d8c90ee7cc..6da4a9ced945 100644
> > > --- i/drivers/nvme/host/core.c
> > > +++ w/drivers/nvme/host/core.c
> > > @@ -3541,7 +3541,6 @@ static int nvme_alloc_ns(struct nvme_ctrl *ctrl=
, unsigned nsid)
> > >         disk->queue =3D ns->queue;
> > >         disk->flags =3D flags;
> > >         memcpy(disk->disk_name, disk_name, DISK_NAME_LEN);
> > > -       ns->disk =3D disk;
> > >=20
> > >         __nvme_revalidate_disk(disk, id);
> > >=20
> > > @@ -3553,6 +3552,8 @@ static int nvme_alloc_ns(struct nvme_ctrl *ctrl=
, unsigned nsid)
> > >                 }
> > >         }
> > >=20
> > > +       ns->disk =3D disk;
> > > +
> >=20
> > Hello Javier!
> >=20
> >=20
> > The only case where we jump to the out_put_disk label, is if the
> > nvme_nvm_register() call failed.
> >=20
> > In that case, we want to undo the alloc_disk_node() operation, i.e.,
> > decrease the refcount.
> >=20
> > If we don't set "ns->disk =3D disk;" before the call to nvme_nvm_regist=
er(),
> > then, if register fails, and we jump to the put_disk(ns->disk) label,
> > ns->disk will be NULL, so the recount will not be decreased, so I assum=
e
> > that this memory would then be a memory leak.
> >=20
> >=20
> > I think that the problem is that the block functions are a bit messy.
> > Most drivers seem to do blk_cleanup_queue() first and then do put_disk(=
),
> > but some drivers do it in the opposite way, so I think that we might ha=
ve
> > some more use-after-free bugs in some of these drivers that do it in th=
e
> > opposite way.
> >=20
>=20
> Hi Niklas,
>=20
> Yes, the out_put_disk label was introduced at the same time as the
> LightNVM entry point. We can do a better job at separating the cleanup
> functions, but as far as I can see ns->disk is not used in the LightNVM
> initialization, so delaying the initialization should be ok. Part of
> this should be also changing the out_put_disk to put_disk(disk).

Hello Javier,


If I understand correctly, you are suggesting this:

--- a/drivers/nvme/host/core.c
+++ b/drivers/nvme/host/core.c
@@ -3616,7 +3616,6 @@ static void nvme_alloc_ns(struct nvme_ctrl *ctrl, uns=
igned nsid)
        disk->queue =3D ns->queue;
        disk->flags =3D flags;
        memcpy(disk->disk_name, disk_name, DISK_NAME_LEN);
-       ns->disk =3D disk;

        __nvme_revalidate_disk(disk, id);

@@ -3628,6 +3627,8 @@ static void nvme_alloc_ns(struct nvme_ctrl *ctrl, uns=
igned nsid)
                }
        }

+       ns->disk =3D disk;
+
        down_write(&ctrl->namespaces_rwsem);
        list_add_tail(&ns->list, &ctrl->namespaces);
        up_write(&ctrl->namespaces_rwsem);
@@ -3642,7 +3643,7 @@ static void nvme_alloc_ns(struct nvme_ctrl *ctrl, uns=
igned nsid)

        return;
  out_put_disk:
-       put_disk(ns->disk);
+       put_disk(disk);
  out_unlink_ns:
        mutex_lock(&ctrl->subsys->lock);
        list_del_rcu(&ns->siblings);


That would not solve the double free error when the registration fails,
since disk->queue is still set, so this call will still free the queue.

Later in the cleanup code blk_cleanup_queue(ns->queue);
is called, which will still try to free what is already freed.

To remove the double free by delaying assignments, we would need to
delay the assignment of disk->queue until after the LightNVM entry point,
but I don't think that is possible, since __nvme_revalidate_disk(), which
is called before the LightNVM entry point uses disk->queue.

Christoph said that he would clean up this mess with better block layer
APIs. So perhaps the fix that is already queued is good enough until then.


Kind regards,
Niklas=