Currently a scsi device won't contribute to kernel randomness when it
uses blk-mq. Since we commonly use scsi on rotational device with
blk-mq, it make sense to keep contributing to kernel randomness in these
cases. This is especially important for virtual machines.
commit b5b6e8c8d3b4 ("scsi: virtio_scsi: fix IO hang caused by automatic
irq vector affinity") made all virtio-scsi device to use blk-mq, which
does not contribute to randomness today. So for a virtual machine only
having virtio-scsi disk (which is common), it will simple stop getting
randomness from its disks in today's implementation.
With this patch, if the above VM has rotational virtio-scsi device, then
it can still benefit from the entropy generated from the disk.
Reported-by: Xuewei Zhang <[email protected]>
Signed-off-by: Xuewei Zhang <[email protected]>
---
drivers/scsi/sd.c | 3 +++
1 file changed, 3 insertions(+)
diff --git a/drivers/scsi/sd.c b/drivers/scsi/sd.c
index b79b366a94f7..5e4f10d28065 100644
--- a/drivers/scsi/sd.c
+++ b/drivers/scsi/sd.c
@@ -2959,6 +2959,9 @@ static void sd_read_block_characteristics(struct scsi_disk *sdkp)
if (rot == 1) {
blk_queue_flag_set(QUEUE_FLAG_NONROT, q);
blk_queue_flag_clear(QUEUE_FLAG_ADD_RANDOM, q);
+ } else {
+ blk_queue_flag_clear(QUEUE_FLAG_NONROT, q);
+ blk_queue_flag_set(QUEUE_FLAG_ADD_RANDOM, q);
}
if (sdkp->device->type == TYPE_ZBC) {
--
2.19.0.rc2.392.g5ba43deb5a-goog
On Thu, 2018-09-06 at 13:37 -0700, Xuewei Zhang wrote:
+AD4 Currently a scsi device won't contribute to kernel randomness when it
+AD4 uses blk-mq. Since we commonly use scsi on rotational device with
+AD4 blk-mq, it make sense to keep contributing to kernel randomness in these
+AD4 cases. This is especially important for virtual machines.
+AD4
+AD4 commit b5b6e8c8d3b4 (+ACI-scsi: virtio+AF8-scsi: fix IO hang caused by automatic
+AD4 irq vector affinity+ACI) made all virtio-scsi device to use blk-mq, which
+AD4 does not contribute to randomness today. So for a virtual machine only
+AD4 having virtio-scsi disk (which is common), it will simple stop getting
+AD4 randomness from its disks in today's implementation.
+AD4
+AD4 With this patch, if the above VM has rotational virtio-scsi device, then
+AD4 it can still benefit from the entropy generated from the disk.
+AD4
+AD4 Reported-by: Xuewei Zhang +ADw-xueweiz+AEA-google.com+AD4
+AD4 Signed-off-by: Xuewei Zhang +ADw-xueweiz+AEA-google.com+AD4
+AD4 ---
+AD4 drivers/scsi/sd.c +AHw 3 +-+-+-
+AD4 1 file changed, 3 insertions(+-)
+AD4
+AD4 diff --git a/drivers/scsi/sd.c b/drivers/scsi/sd.c
+AD4 index b79b366a94f7..5e4f10d28065 100644
+AD4 --- a/drivers/scsi/sd.c
+AD4 +-+-+- b/drivers/scsi/sd.c
+AD4 +AEAAQA -2959,6 +-2959,9 +AEAAQA static void sd+AF8-read+AF8-block+AF8-characteristics(struct
+AD4 scsi+AF8-disk +ACo-sdkp)
+AD4 if (rot +AD0APQ 1) +AHs
+AD4 blk+AF8-queue+AF8-flag+AF8-set(QUEUE+AF8-FLAG+AF8-NONROT, q)+ADs
+AD4 blk+AF8-queue+AF8-flag+AF8-clear(QUEUE+AF8-FLAG+AF8-ADD+AF8-RANDOM, q)+ADs
+AD4 +- +AH0 else +AHs
+AD4 +- blk+AF8-queue+AF8-flag+AF8-clear(QUEUE+AF8-FLAG+AF8-NONROT, q)+ADs
+AD4 +- blk+AF8-queue+AF8-flag+AF8-set(QUEUE+AF8-FLAG+AF8-ADD+AF8-RANDOM, q)+ADs
+AD4 +AH0
+AD4
+AD4 if (sdkp-+AD4-device-+AD4-type +AD0APQ TYPE+AF8-ZBC) +AHs
Although this patch looks fine to me, seeing this patch makes me wonder
whether the default should be changed (QUEUE+AF8-FLAG+AF8-MQ+AF8-DEFAULT) instead of
modifying the sd driver. Can anyone remind me why QUEUE+AF8-FLAG+AF8-MQ+AF8-DEFAULT does
not include QUEUE+AF8-FLAG+AF8-ADD+AF8-RANDOM?
Thanks,
Bart.
On Thu, Sep 06, 2018 at 03:27:53PM -0700, Bart Van Assche wrote:
>
> Although this patch looks fine to me, seeing this patch makes me wonder
> whether the default should be changed (QUEUE_FLAG_MQ_DEFAULT) instead of
> modifying the sd driver. Can anyone remind me why QUEUE_FLAG_MQ_DEFAULT does
> not include QUEUE_FLAG_ADD_RANDOM?
There was a discussion about a number of *years* ago; blk-mq has been
baking for a very long time. In the early days of block_mq, the
overwhelming percentage of the users of blk-mq where those who were
using PCIe attached flash. So when, I raised this question, the
argument was that SSD users have no entropy. Which I agree with; but
now that blk-mq is the default, and hard drives are using blk-mq, it's
time for a patch like Xuewei's.
Cheers,
- Ted
On Thu, Sep 6, 2018 at 3:42 PM Theodore Y. Ts'o <[email protected]> wrote:
>
> On Thu, Sep 06, 2018 at 03:27:53PM -0700, Bart Van Assche wrote:
> >
> > Although this patch looks fine to me, seeing this patch makes me wonder
> > whether the default should be changed (QUEUE_FLAG_MQ_DEFAULT) instead of
> > modifying the sd driver. Can anyone remind me why QUEUE_FLAG_MQ_DEFAULT does
> > not include QUEUE_FLAG_ADD_RANDOM?
Besides Ted's point of "SSD users have no entropy", I think there are two more
reasons:
1. setting QUEUE_FLAG_ADD_RANDOM has a more visible performance hit
on SSD disks than rotational disks.
2. SSD disks provide less entropy than rotational disks.
I actually experimented on Container-Optimized OS, running on Google Compute
Engine with QUEUE_FLAG_ADD_RANDOM set.
Turns out the VM will have ~800 bit of entropy provided on boot on
rotational disk;
and will only have ~70 bit of entropy if running on SSD (and remember there are
~50 bit contributed from other sources).
(in the above experiment, both disks were virtualized disks)
>
> There was a discussion about a number of *years* ago; blk-mq has been
> baking for a very long time. In the early days of block_mq, the
> overwhelming percentage of the users of blk-mq where those who were
> using PCIe attached flash. So when, I raised this question, the
> argument was that SSD users have no entropy. Which I agree with; but
> now that blk-mq is the default, and hard drives are using blk-mq, it's
> time for a patch like Xuewei's.
>
> Cheers,
>
> - Ted
On 09/06/18 16:03, Xuewei Zhang wrote:
> On Thu, Sep 6, 2018 at 3:42 PM Theodore Y. Ts'o <[email protected]> wrote:
>> There was a discussion about a number of *years* ago; blk-mq has been
>> baking for a very long time. In the early days of block_mq, the
>> overwhelming percentage of the users of blk-mq where those who were
>> using PCIe attached flash. So when, I raised this question, the
>> argument was that SSD users have no entropy. Which I agree with; but
>> now that blk-mq is the default, and hard drives are using blk-mq, it's
>> time for a patch like Xuewei's.
>
> Besides Ted's point of "SSD users have no entropy", I think there are
> two more reasons:
> 1. setting QUEUE_FLAG_ADD_RANDOM has a more visible performance hit
> on SSD disks than rotational disks.
> 2. SSD disks provide less entropy than rotational disks.
> I actually experimented on Container-Optimized OS, running on Google
> Compute Engine with QUEUE_FLAG_ADD_RANDOM set.
> Turns out the VM will have ~800 bit of entropy provided on boot on
> rotational disk;
> and will only have ~70 bit of entropy if running on SSD (and remember
> there are ~50 bit contributed from other sources).
> (in the above experiment, both disks were virtualized disks)
All of the above makes sense to me and is much less risky than changing
QUEUE_FLAG_MQ_DEFAULT. Hence:
Reviewed-by: Bart Van Assche <[email protected]>
On Thu, Sep 06, 2018 at 01:37:19PM -0700, Xuewei Zhang wrote:
> Currently a scsi device won't contribute to kernel randomness when it
> uses blk-mq. Since we commonly use scsi on rotational device with
> blk-mq, it make sense to keep contributing to kernel randomness in these
> cases. This is especially important for virtual machines.
>
> commit b5b6e8c8d3b4 ("scsi: virtio_scsi: fix IO hang caused by automatic
> irq vector affinity") made all virtio-scsi device to use blk-mq, which
> does not contribute to randomness today. So for a virtual machine only
> having virtio-scsi disk (which is common), it will simple stop getting
> randomness from its disks in today's implementation.
>
> With this patch, if the above VM has rotational virtio-scsi device, then
> it can still benefit from the entropy generated from the disk.
>
> Reported-by: Xuewei Zhang <[email protected]>
> Signed-off-by: Xuewei Zhang <[email protected]>
> ---
> drivers/scsi/sd.c | 3 +++
> 1 file changed, 3 insertions(+)
>
> diff --git a/drivers/scsi/sd.c b/drivers/scsi/sd.c
> index b79b366a94f7..5e4f10d28065 100644
> --- a/drivers/scsi/sd.c
> +++ b/drivers/scsi/sd.c
> @@ -2959,6 +2959,9 @@ static void sd_read_block_characteristics(struct scsi_disk *sdkp)
> if (rot == 1) {
> blk_queue_flag_set(QUEUE_FLAG_NONROT, q);
> blk_queue_flag_clear(QUEUE_FLAG_ADD_RANDOM, q);
> + } else {
> + blk_queue_flag_clear(QUEUE_FLAG_NONROT, q);
> + blk_queue_flag_set(QUEUE_FLAG_ADD_RANDOM, q);
> }
>
> if (sdkp->device->type == TYPE_ZBC) {
> --
> 2.19.0.rc2.392.g5ba43deb5a-goog
>
Look reasonable, especially the disk randomness is added by SCSI itself.
Reviewed-by: Ming Lei <[email protected]>
Thanks,
Ming
On Sun, Sep 9, 2018 at 4:52 AM, Ming Lei <[email protected]> wrote:
> On Thu, Sep 06, 2018 at 01:37:19PM -0700, Xuewei Zhang wrote:
>> Currently a scsi device won't contribute to kernel randomness when it
>> uses blk-mq. Since we commonly use scsi on rotational device with
>> blk-mq, it make sense to keep contributing to kernel randomness in these
>> cases. This is especially important for virtual machines.
>>
>> commit b5b6e8c8d3b4 ("scsi: virtio_scsi: fix IO hang caused by automatic
>> irq vector affinity") made all virtio-scsi device to use blk-mq, which
>> does not contribute to randomness today. So for a virtual machine only
>> having virtio-scsi disk (which is common), it will simple stop getting
>> randomness from its disks in today's implementation.
>>
>> With this patch, if the above VM has rotational virtio-scsi device, then
>> it can still benefit from the entropy generated from the disk.
>>
>> Reported-by: Xuewei Zhang <[email protected]>
>> Signed-off-by: Xuewei Zhang <[email protected]>
>> ---
>> drivers/scsi/sd.c | 3 +++
>> 1 file changed, 3 insertions(+)
>>
>> diff --git a/drivers/scsi/sd.c b/drivers/scsi/sd.c
>> index b79b366a94f7..5e4f10d28065 100644
>> --- a/drivers/scsi/sd.c
>> +++ b/drivers/scsi/sd.c
>> @@ -2959,6 +2959,9 @@ static void sd_read_block_characteristics(struct scsi_disk *sdkp)
>> if (rot == 1) {
>> blk_queue_flag_set(QUEUE_FLAG_NONROT, q);
>> blk_queue_flag_clear(QUEUE_FLAG_ADD_RANDOM, q);
>> + } else {
>> + blk_queue_flag_clear(QUEUE_FLAG_NONROT, q);
>> + blk_queue_flag_set(QUEUE_FLAG_ADD_RANDOM, q);
>> }
>>
>> if (sdkp->device->type == TYPE_ZBC) {
>> --
>> 2.19.0.rc2.392.g5ba43deb5a-goog
>>
>
> Look reasonable, especially the disk randomness is added by SCSI itself.
>
> Reviewed-by: Ming Lei <[email protected]>
>
> Thanks,
> Ming
Also, see: https://bugzilla.redhat.com/show_bug.cgi?id=1572944
where we're having randomness starvation problems on FC28
running 4.18.5 due to lack of virtio-rng device in VM.
(VM boot takes 9+ hours or 2 haswell VMs)
I'd kindly request we get this not only into 4.19 but also stable trees.
(along with Ted's other randomization fixes)
Reviewed-by: Maciej Żenczykowski <[email protected]>
Xuewei,
> Currently a scsi device won't contribute to kernel randomness when it
> uses blk-mq. Since we commonly use scsi on rotational device with
> blk-mq, it make sense to keep contributing to kernel randomness in these
> cases. This is especially important for virtual machines.
Applied to 4.19/scsi-fixes, thank you!
--
Martin K. Petersen Oracle Linux Engineering