Attached 2 x brand new Western Digital 8TB USB 3.0 drives awhile back and
ran some file copy tests and was getting these warnings-- is there any way
to avoid these warnings?? I did confirm with parted that the partition was
aligned but this appears to be something related to the firmware on the
device according to [1] and [2]?
[1]?https://patchwork.kernel.org/patch/9573203/
[2] https://patchwork.kernel.org/patch/9597797/
Part of the patch in [2] - if the firmware is buggy is there a patch or
workaround for these drives (when used in the USB enclosures) to avoid this
issue?
+ default:
+ /*
+ * In case of bogus fw or device, we could end up having
+ * an unaligned partial completion. Check this here and
force
+ * alignment.
+ */
+ resid = scsi_get_resid(SCpnt);
+ if (resid & (sector_size - 1)) {
+ sd_printk(KERN_INFO, sdkp,
+ "Unaligned partial completion (resid=%u,
sector_sz=%u)\n",
+ resid, sector_size);
+ resid = min(scsi_bufflen(SCpnt),
+ round_up(resid, sector_size));
+ scsi_set_resid(SCpnt, resid);
+ }
Errors:
Jul 18 16:25:02 name kernel: [87305.605993] sd 8:0:0:0: [sdg] Unaligned
partial completion (resid=78, sector_sz=512)
Jul 18 16:25:02 name?kernel: [87305.605993] sd 8:0:0:0: [sdg] Unaligned
partial completion (resid=78, sector_sz=512)
Jul 18 16:25:15 name?kernel: [87318.292262] sd 7:0:0:0: [sdf] Unaligned
partial completion (resid=78, sector_sz=512)
Jul 18 16:25:15 name?kernel: [87318.292262] sd 7:0:0:0: [sdf] Unaligned
partial completion (resid=78, sector_sz=512)
I can reproduce this with kernel 5.2.2 and a WD drive (WD40EZRZ-22GXCB0)
in a USB 3.0 enclosure featuring a JMicron SATA Bridge (ID 152d:2329)
I'm also interested in getting rid of the warnings.
* Justin Piszcz <[email protected]> wrote:
> Attached 2 x brand new Western Digital 8TB USB 3.0 drives awhile
> back and ran some file copy tests and was getting these warnings--
> is there any way to avoid these warnings?? I did confirm with
> parted that the partition was aligned but this appears to be
> something related to the firmware on the device according to [1]
> and [2]?
>
> [1]?https://patchwork.kernel.org/patch/9573203/
> [2] https://patchwork.kernel.org/patch/9597797/
>
> Part of the patch in [2] - if the firmware is buggy is there a
> patch or workaround for these drives (when used in the USB
> enclosures) to avoid this issue?
>
> + default:
> + /*
> + * In case of bogus fw or device, we could end up having
> + * an unaligned partial completion. Check this here and
> force
> + * alignment.
> + */
> + resid = scsi_get_resid(SCpnt);
> + if (resid & (sector_size - 1)) {
> + sd_printk(KERN_INFO, sdkp,
> + "Unaligned partial completion (resid=%u,
> sector_sz=%u)\n",
> + resid, sector_size);
> + resid = min(scsi_bufflen(SCpnt),
> + round_up(resid, sector_size));
> + scsi_set_resid(SCpnt, resid);
> + }
>
> Errors:
>
> Jul 18 16:25:02 name kernel: [87305.605993] sd 8:0:0:0: [sdg] Unaligned
> partial completion (resid=78, sector_sz=512)
> Jul 18 16:25:02 name?kernel: [87305.605993] sd 8:0:0:0: [sdg] Unaligned
> partial completion (resid=78, sector_sz=512)
>
> Jul 18 16:25:15 name?kernel: [87318.292262] sd 7:0:0:0: [sdf] Unaligned
> partial completion (resid=78, sector_sz=512)
> Jul 18 16:25:15 name?kernel: [87318.292262] sd 7:0:0:0: [sdf] Unaligned
> partial completion (resid=78, sector_sz=512)
--
left blank, right bald
On Sat, 3 Aug 2019, Justin Piszcz wrote:
> Attached 2 x brand new Western Digital 8TB USB 3.0 drives awhile back and
> ran some file copy tests and was getting these warnings-- is there any way
> to avoid these warnings?? I did confirm with parted that the partition was
> aligned but this appears to be something related to the firmware on the
> device according to [1] and [2]?
>
> [1]?https://patchwork.kernel.org/patch/9573203/
> [2] https://patchwork.kernel.org/patch/9597797/
Just out of curiosity, why did you not address your email to the author
or committer of this patch? Surely they would be the people in the
best position to answer your questions.
Alan Stern
-----Original Message-----
From: Alan Stern [mailto:[email protected]]
Sent: Saturday, August 3, 2019 3:00 PM
To: Justin Piszcz
Cc: 'LKML'; [email protected]; [email protected]
Subject: Re: 5.2.x kernel: WD 8TB USB Drives: Unaligned partial completion (resid=78, sector_sz=512)
On Sat, 3 Aug 2019, Justin Piszcz wrote:
> Attached 2 x brand new Western Digital 8TB USB 3.0 drives awhile back and
> ran some file copy tests and was getting these warnings-- is there any way
> to avoid these warnings?� I did confirm with parted that the partition was
> aligned but this appears to be something related to the firmware on the
> device according to [1] and [2]?
>
> [1]�https://patchwork.kernel.org/patch/9573203/
> [2] https://patchwork.kernel.org/patch/9597797/
Just out of curiosity, why did you not address your email to the author
or committer of this patch? Surely they would be the people in the
best position to answer your questions.
[ .. ]
Fixed.
I had been researching this issue and did not find any useful notes on the mailing lists--hopefully this will help others if there's a copy of the response on the mailing list.
These drives [1] [2] are the ones noted as affected when used on Linux [1] plugged in directly to a Linux PC (w/USB 3.0/XHCI) and [2] when in a USB enclosure with a JMicron SATA Bridge.
[1] WDBBGB0080HBK-NESN - https://www.wd.com/products/external-storage/my-book-new.html#WDBBGB0080HBK-NESN
[2] WD40EZRZ-22GXCB0 - https://www.wd.com/products/internal-storage/wd-blue-pc-desktop-hard-drive.html (In a USB enclosure with a JMicron SATA Bridge - per Markus)
Regards,
Justin.
Justin,
> Attached 2 x brand new Western Digital 8TB USB 3.0 drives awhile back and
> ran some file copy tests and was getting these warnings-- is there any way
> to avoid these warnings? I did confirm with parted that the partition was
> aligned but this appears to be something related to the firmware on the
> device according to [1] and [2]?
Please send us the output of:
# sg_vpd -p bl /dev/sdN
# sg_vpd -p bdc /dev/sdN
# sg_readcap -l /dev/sdN
--
Martin K. Petersen Oracle Linux Engineering
-----Original Message-----
From: Martin K. Petersen [mailto:[email protected]]
Sent: Wednesday, August 7, 2019 10:04 PM
To: Justin Piszcz
Cc: 'LKML'; [email protected]; [email protected]
Subject: Re: 5.2.x kernel: WD 8TB USB Drives: Unaligned partial completion (resid=78, sector_sz=512)
Justin,
> Attached 2 x brand new Western Digital 8TB USB 3.0 drives awhile back and
> ran some file copy tests and was getting these warnings-- is there any way
> to avoid these warnings? I did confirm with parted that the partition was
> aligned but this appears to be something related to the firmware on the
> device according to [1] and [2]?
Please send us the output of:
# sg_vpd -p bl /dev/sdN
# sg_vpd -p bdc /dev/sdN
# sg_readcap -l /dev/sdN
[ .. ]
Disk type:
---
Disk /dev/sdf: 7.3 TiB, 8001562869760 bytes, 15628052480 sectors
Disk model: My Book 25EE
# sg_vpd -p bl /dev/sdf > /tmp/sg_vpd_bl.txt
# sg_vpd -p bdc /dev/sdf > /tmp/sg_vpd_bdc.txt
# sg_readcap -l /dev/sdf > /tmp/sg_readcap.txt
# ls -l /tmp/sg_vpd_bl.txt /tmp/sg_vpd_bdc.txt /tmp/sg_readcap.txt
-rw-r--r-- 1 root root 421 Aug 8 04:26 /tmp/sg_readcap.txt
-rw-r--r-- 1 root root 244 Aug 8 04:25 /tmp/sg_vpd_bdc.txt
-rw-r--r-- 1 root root 972 Aug 8 04:25 /tmp/sg_vpd_bl.txt
Output:
sg_readcap.txt
Read Capacity results:
Protection: prot_en=0, p_type=0, p_i_exponent=0
Logical block provisioning: lbpme=0, lbprz=0
Last LBA=15628052479 (0x3a38127ff), Number of logical blocks=15628052480
Logical block length=512 bytes
Logical blocks per physical block exponent=3 [so physical block length=4096 bytes]
Lowest aligned LBA=0
Hence:
Device size: 8001562869760 bytes, 7630885.0 MiB, 8001.56 GB, 8.00 TB
sg_vpd_bdc.txt
Block device characteristics VPD page (SBC):
Nominal rotation rate: 5400 rpm
Product type: Not specified
WABEREQ=0
WACEREQ=0
Nominal form factor: 3.5 inch
ZONED=0
RBWZ=0
BOCS=0
FUAB=0
VBULS=0
DEPOPULATION_TIME=0 (seconds)
sg_vpd_bl.txt
Block limits VPD page (SBC):
Write same non-zero (WSNZ): 0
Maximum compare and write length: 0 blocks [Command not implemented]
Optimal transfer length granularity: 8 blocks
Maximum transfer length: 65535 blocks
Optimal transfer length: 65535 blocks
Maximum prefetch transfer length: 65535 blocks
Maximum unmap LBA count: 0 [Unmap command not implemented]
Maximum unmap block descriptor count: 0 [Unmap command not implemented]
Optimal unmap granularity: 0 blocks [not reported]
Unmap granularity alignment valid: false
Unmap granularity alignment: 0 [invalid]
Maximum write same length: 0 blocks [not reported]
Maximum atomic transfer length: 0 blocks [not reported]
Atomic alignment: 0 [unaligned atomic writes permitted]
Atomic transfer length granularity: 0 [no granularity requirement
Maximum atomic transfer length with atomic boundary: 0 blocks [not reported]
Maximum atomic boundary size: 0 blocks [can only write atomic 1 block]
Justin,
>> Attached 2 x brand new Western Digital 8TB USB 3.0 drives awhile back
>> and ran some file copy tests and was getting these warnings-- is
>> there any way to avoid these warnings? I did confirm with parted
>> that the partition was aligned but this appears to be something
>> related to the firmware on the device according to [1] and [2]?
> sg_vpd_bdc.txt
> Block device characteristics VPD page (SBC):
> Nominal rotation rate: 5400 rpm
> Product type: Not specified
> WABEREQ=0
> WACEREQ=0
> Nominal form factor: 3.5 inch
> ZONED=0
Damien: What can we do to limit the messages in cases like this? Would
it make sense to make the residual warning conditional on sd_is_zoned()?
--
Martin K. Petersen Oracle Linux Engineering
On 2019/08/12 19:12, Martin K. Petersen wrote:
>
> Justin,
>
>>> Attached 2 x brand new Western Digital 8TB USB 3.0 drives awhile back
>>> and ran some file copy tests and was getting these warnings-- is
>>> there any way to avoid these warnings? I did confirm with parted
>>> that the partition was aligned but this appears to be something
>>> related to the firmware on the device according to [1] and [2]?
>
>> sg_vpd_bdc.txt
>> Block device characteristics VPD page (SBC):
>> Nominal rotation rate: 5400 rpm
>> Product type: Not specified
>> WABEREQ=0
>> WACEREQ=0
>> Nominal form factor: 3.5 inch
>> ZONED=0
>
> Damien: What can we do to limit the messages in cases like this? Would
> it make sense to make the residual warning conditional on sd_is_zoned()?
>
These WD drives are regular disks, not SMR. Making the warning conditional on
sd_is_zoned() will not reduce the amount of messages. REPORT ZONES is the only
command that could result in a resid not being aligned to the block size since
by definition the command reply is composed of 64B zone descriptors. But that
command is now processed through a device method and is not a REQ_OP_XXX
anymore, so as an internal req, it does not go through sd_done() for completion
right ? All other zone commands either have no buffer, or the exact same
requirement as regular disks, expecting a block aligned resid. So I think that
using sd_is_zoned() is not relevant to this problem.
Bottom line: this USB adapter is weird and likely triggers all the unaligned
resid. I do get regularly reports of similar problem with SAS HBAs, all always
fixed with HBA FW updates. Not sure if there is a FW available for these drives.
I will ask internally.
In the mean time, rate limiting or removing the sd_printk() call may be the only
option. Failing all commands with an invalid resid would be safer I guess, but
will at best likely cause a lot of retry on these buggy devices, and break the
drive operation/perfomance completely in the worst case.
Or the usb mass storage device driver could add some silent forced resid
alignment too.
Best regards.
--
Damien Le Moal
Western Digital Research
On 2019/08/12 19:12, Martin K. Petersen wrote:
>
> Justin,
>
>>> Attached 2 x brand new Western Digital 8TB USB 3.0 drives awhile back
>>> and ran some file copy tests and was getting these warnings-- is
>>> there any way to avoid these warnings? I did confirm with parted
>>> that the partition was aligned but this appears to be something
>>> related to the firmware on the device according to [1] and [2]?
>
>> sg_vpd_bdc.txt
>> Block device characteristics VPD page (SBC):
>> Nominal rotation rate: 5400 rpm
>> Product type: Not specified
>> WABEREQ=0
>> WACEREQ=0
>> Nominal form factor: 3.5 inch
>> ZONED=0
>
> Damien: What can we do to limit the messages in cases like this? Would
> it make sense to make the residual warning conditional on sd_is_zoned()?
>
Justin,
Can you send the output of "lsscsi" for these drives ? I need the exact disk
model ref name and FW version to see if there is an update for this problem, if
it is a known one. If it is not, I will signal it and get a fix started.
Best regards.
--
Damien Le Moal
Western Digital Research
On Tue, Aug 13, 2019 at 6:48 AM Damien Le Moal <[email protected]> wrote:
>
> On 2019/08/12 19:12, Martin K. Petersen wrote:
> >
> > Justin,
> >
> >>> Attached 2 x brand new Western Digital 8TB USB 3.0 drives awhile back
> >>> and ran some file copy tests and was getting these warnings-- is
> >>> there any way to avoid these warnings? I did confirm with parted
> >>> that the partition was aligned but this appears to be something
> >>> related to the firmware on the device according to [1] and [2]?
> >
> >> sg_vpd_bdc.txt
> >> Block device characteristics VPD page (SBC):
> >> Nominal rotation rate: 5400 rpm
> >> Product type: Not specified
> >> WABEREQ=0
> >> WACEREQ=0
> >> Nominal form factor: 3.5 inch
> >> ZONED=0
> >
> > Damien: What can we do to limit the messages in cases like this? Would
> > it make sense to make the residual warning conditional on sd_is_zoned()?
> >
>
> Justin,
>
> Can you send the output of "lsscsi" for these drives ? I need the exact disk
> model ref name and FW version to see if there is an update for this problem, if
> it is a known one. If it is not, I will signal it and get a fix started.
>
Requested lsscsi output:
[7:0:0:0] disk WD My Book 25EE 4004 /dev/sdf
[7:0:0:1] enclosu WD SES Device 4004 -
[8:0:0:0] disk WD My Book 25EE 4004 /dev/sdg
[8:0:0:1] enclosu WD SES Device 4004 -
Host: scsi7 Channel: 00 Target: 00 Lun: 00
Vendor: WD Model: My Book 25EE Rev: 4004
Type: Direct-Access ANSI SCSI revision: 06
Host: scsi7 Channel: 00 Target: 00 Lun: 01
Vendor: WD Model: SES Device Rev: 4004
Type: Enclosure ANSI SCSI revision: 06
Host: scsi8 Channel: 00 Target: 00 Lun: 00
Vendor: WD Model: My Book 25EE Rev: 4004
Type: Direct-Access ANSI SCSI revision: 06
Host: scsi8 Channel: 00 Target: 00 Lun: 01
Vendor: WD Model: SES Device Rev: 4004
Type: Enclosure ANSI SCSI revision: 06
[7:0:0:0] disk WD My Book 25EE 4004 /dev/sdf
state=running queue_depth=1 scsi_level=7 type=0 device_blocked=0 timeout=30
[7:0:0:1] enclosu WD SES Device 4004 -
state=running queue_depth=1 scsi_level=7 type=13 device_blocked=0 timeout=30
[8:0:0:0] disk WD My Book 25EE 4004 /dev/sdg
state=running queue_depth=1 scsi_level=7 type=0 device_blocked=0 timeout=30
[8:0:0:1] enclosu WD SES Device 4004 -
state=running queue_depth=1 scsi_level=7 type=13 device_blocked=0 timeout=30
[7:0:0:0] disk none /dev/sdf
[7:0:0:1] enclosu 5000cca252e63312 -
[8:0:0:0] disk none /dev/sdg
[8:0:0:1] enclosu 5000cca252e6a2f9 -
[7:0:0:0] disk WD My Book 25EE 4004 /dev/sdf 8.00TB
[7:0:0:1] enclosu WD SES Device 4004 - -
[8:0:0:0] disk WD My Book 25EE 4004 /dev/sdg 8.00TB
[8:0:0:1] enclosu WD SES Device 4004 - -
[7:0:0:0] disk usb:1-1.2:1.0 /dev/sdf
[7:0:0:1] enclosu usb:1-1.2:1.0 -
[8:0:0:0] disk usb:1-1.1:1.0 /dev/sdg
[8:0:0:1] enclosu usb:1-1.1:1.0 -
# smartctl -a /dev/sdg
/dev/sdg: Unknown USB bridge [0x1058:0x25ee (0x4004)]
# smartctl -a /dev/sdf
/dev/sdf: Unknown USB bridge [0x1058:0x25ee (0x4004)]
Regards,
Justin.