Received-SPF: pass (google.com: best guess record for domain of linux-kernel-owner@vger.kernel.org designates 209.132.180.67 as permitted sender) client-ip=209.132.180.67;
Date:   Wed, 15 Jan 2020 10:27:25 +0800
From:   Ming Lei <ming.lei@redhat.com>
To:     Zhiqiang Liu <liuzhiqiang26@huawei.com>
Cc:     Jens Axboe <axboe@kernel.dk>, linux-block@vger.kernel.org,
        "linux-kernel@vger.kernel.org" <linux-kernel@vger.kernel.org>,
        Mingfangsen <mingfangsen@huawei.com>, Guiyao <guiyao@huawei.com>,
        zhangsaisai <zhangsaisai@huawei.com>,
        "wubo (T)" <wubo40@huawei.com>
Subject: Re: [PATCH V3] brd: check and limit max_part par
Message-ID: <20200115022725.GA14585@ming.t460p>
References: <c8236e55-f64f-ef40-b394-8b7e86ce50df@huawei.com>
MIME-Version: 1.0
Content-Type: text/plain; charset=iso-8859-1
Content-Disposition: inline
In-Reply-To: <c8236e55-f64f-ef40-b394-8b7e86ce50df@huawei.com>
User-Agent: Mutt/1.12.1 (2019-06-15)
Content-Transfer-Encoding: quoted-printable
Sender: linux-kernel-owner@vger.kernel.org
Precedence: bulk

On Tue, Jan 14, 2020 at 07:56:07PM +0800, Zhiqiang Liu wrote:
>=20
> In brd_init func, rd_nr num of brd_device are firstly allocated
> and add in brd_devices, then brd_devices are traversed to add each
> brd_device by calling add_disk func. When allocating brd_device,
> the disk->first_minor is set to i * max_part, if rd_nr * max_part
> is larger than MINORMASK, two different brd_device may have the same
> devt, then only one of them can be successfully added.
> when rmmod brd.ko, it will cause oops when calling brd_exit.
>=20
> Follow those steps:
>   # modprobe brd rd_nr=3D3 rd_size=3D102400 max_part=3D1048576
>   # rmmod brd
> then, the oops will appear.
>=20
> Oops log:
> [  726.613722] Call trace:
> [  726.614175]  kernfs_find_ns+0x24/0x130
> [  726.614852]  kernfs_find_and_get_ns+0x44/0x68
> [  726.615749]  sysfs_remove_group+0x38/0xb0
> [  726.616520]  blk_trace_remove_sysfs+0x1c/0x28
> [  726.617320]  blk_unregister_queue+0x98/0x100
> [  726.618105]  del_gendisk+0x144/0x2b8
> [  726.618759]  brd_exit+0x68/0x560 [brd]
> [  726.619501]  __arm64_sys_delete_module+0x19c/0x2a0
> [  726.620384]  el0_svc_common+0x78/0x130
> [  726.621057]  el0_svc_handler+0x38/0x78
> [  726.621738]  el0_svc+0x8/0xc
> [  726.622259] Code: aa0203f6 aa0103f7 aa1e03e0 d503201f (7940e260)
>=20
> Here, we add brd_check_and_reset_par func to check and limit max_part p=
ar.
>=20
> --
> V2->V3: (suggested by Ming Lei)
>  - clear .minors when running out of consecutive minor space in brd_all=
oc
>  - remove limit of rd_nr
>=20
> V1->V2: add more checks in brd_check_par_valid as suggested by Ming Lei=
.
>=20
> Signed-off-by: Zhiqiang Liu <liuzhiqiang26@huawei.com>
> ---
>  drivers/block/brd.c | 35 ++++++++++++++++++++++++++++-------
>  1 file changed, 28 insertions(+), 7 deletions(-)
>=20
> diff --git a/drivers/block/brd.c b/drivers/block/brd.c
> index df8103dd40ac..2295a0bafb5e 100644
> --- a/drivers/block/brd.c
> +++ b/drivers/block/brd.c
> @@ -330,16 +330,16 @@ static const struct block_device_operations brd_f=
ops =3D {
>  /*
>   * And now the modules code and kernel interface.
>   */
> -static int rd_nr =3D CONFIG_BLK_DEV_RAM_COUNT;
> -module_param(rd_nr, int, 0444);
> +static unsigned int rd_nr =3D CONFIG_BLK_DEV_RAM_COUNT;
> +module_param(rd_nr, uint, 0444);

The above change isn't needed.

>  MODULE_PARM_DESC(rd_nr, "Maximum number of brd devices");
>=20
>  unsigned long rd_size =3D CONFIG_BLK_DEV_RAM_SIZE;
>  module_param(rd_size, ulong, 0444);
>  MODULE_PARM_DESC(rd_size, "Size of each RAM disk in kbytes.");
>=20
> -static int max_part =3D 1;
> -module_param(max_part, int, 0444);
> +static unsigned int max_part =3D 1;
> +module_param(max_part, uint, 0444);

The above change isn't needed.

>  MODULE_PARM_DESC(max_part, "Num Minors to reserve between devices");
>=20
>  MODULE_LICENSE("GPL");
> @@ -393,7 +393,14 @@ static struct brd_device *brd_alloc(int i)
>  	if (!disk)
>  		goto out_free_queue;
>  	disk->major		=3D RAMDISK_MAJOR;
> -	disk->first_minor	=3D i * max_part;
> +	/*
> +	 * Clear .minors when running out of consecutive minor space since
> +	 * GENHD_FL_EXT_DEVT is set, and we can allocate from extended devt.
> +	 */
> +	if ((i * disk->minors) & ~MINORMASK)
> +		disk->minors =3D 0;
> +	else
> +		disk->first_minor =3D i * disk->minors;

The above looks a bit ugly, one nice way could be to change in
brd_alloc():

	disk =3D brd->brd_disk =3D alloc_disk(((i * max_part) & ~MINORMASK) ?
		0 : max_part);

>  	disk->fops		=3D &brd_fops;
>  	disk->private_data	=3D brd;
>  	disk->queue		=3D brd->brd_queue;
> @@ -468,6 +475,21 @@ static struct kobject *brd_probe(dev_t dev, int *p=
art, void *data)
>  	return kobj;
>  }
>=20
> +static inline void brd_check_and_reset_par(void)
> +{
> +	if (unlikely(!rd_nr))
> +		rd_nr =3D 1;

zero rd_nr should work as expected, given user can create dev file via
mknod, and brd_probe() will be called for populate brd disk/queue when
the disk file is opened.

> +static inline void brd_check_and_reset_par(void)
> +{
> + =A0 =A0 =A0 if (unlikely(!rd_nr))
> + =A0 =A0 =A0 =A0 =A0 =A0 =A0 rd_nr =3D 1;
> +
> + =A0 =A0 =A0 if (unlikely(!max_part))
> + =A0 =A0 =A0 =A0 =A0 =A0 =A0 max_part =3D 1;

Another limit is that 'max_part' needs to be divided exactly by (1U <<
MINORBITS), something like:

	max_part =3D 1UL << fls(max_part);


Thanks,=20
Ming