There is a problem in the latest upstream kernel with the device:
$ grep -i lsi lspci
03:00.0 Serial Attached SCSI controller [0107]: LSI Logic / Symbios
Logic SAS1068 PCI-X Fusion-MPT SAS [1000:0054] (rev 01)
The device is simulated by the VMware ESXi 5.5
When hotplugging a new disk to the Guest Ubuntu OS, the latest kernel
cannot automatically probe the disk. However, on the v3.19.0-80.88
kernel, the disk can be dynamically probed and show the following
info message:
mptsas: ioc0: attaching ssp device: fw_channel 0, fw_id 1, phy 1,
sas_addr 0x5000c29a6bdae0f5
scsi 2:0:1:0: Direct-Access VMware Virtual disk 1.0 PQ: 0
ANSI: 2
sd 2:0:1:0: Attached scsi generic sg2 type 0
sd 2:0:1:0: [sdb] 2097152 512-byte logical blocks: (1.07 GB/1.00 GiB)
sd 2:0:1:0: [sdb] Write Protect is off
sd 2:0:1:0: [sdb] Mode Sense: 61 00 00 00
sd 2:0:1:0: [sdb] Cache data unavailable
sd 2:0:1:0: [sdb] Assuming drive cache: write through
sdb: unknown partition table
sd 2:0:1:0: [sdb] Attached SCSI disk
After looking up the message:
mptsas: ioc0: attaching ssp device: fw_channel 0, fw_id 1, phy 1,
sas_addr 0x5000c29a6bdae0f5
I found it comes from the path:
mptsas_firmware_event_work -> mptsas_send_sas_event ->
mptsas_hotplug_work -> mptsas_add_end_device
I'll appreciate if anyone can give the idea: If it's possible that the
irq from the simulated LSI SAS controller didn't come in to trigger
the event? However, it can work on the v3.19 kernel so if there is
any driver implementation issue in the latest kernel.
On Fri, Nov 3, 2017 at 6:59 PM, Hannes Reinecke <[email protected]> wrote:
> On 11/03/2017 04:38 AM, Gavin Guo wrote:
>> On Sat, Oct 28, 2017 at 11:35 AM, Gavin Guo <[email protected]> wrote:
>>> On Fri, Oct 27, 2017 at 10:53 PM, Hannes Reinecke <[email protected]> wrote:
>>>> On 10/27/2017 04:02 PM, Gavin Guo wrote:
>>>>> Hi Hannes,
>>>>>
>>>>> Thank you for looking into the issue. If there is anything I can help
>>>>> to test the patch? I appreciate your help. Thank you.
>>>>>
>>>> If you had checked linux-scsi you would have found this patch:
>>>> '[PATCH] mptsas: Fixup device hotplug for VMWare ESXi', which I guess is
>>>> already scheduled for inclusion in 4.14.
>>>> Anything else I could help you with?
>>>>
>>>> Cheers,
>>>>
>>>> Hannes
>>>> --
>>>> Dr. Hannes Reinecke Teamlead Storage & Networking
>>>> [email protected] +49 911 74053 688
>>>> SUSE LINUX GmbH, Maxfeldstr. 5, 90409 Nürnberg
>>>> GF: F. Imendörffer, J. Smithard, J. Guild, D. Upmanyu, G. Norton
>>>> HRB 21284 (AG Nürnberg)
>>>
>>> Really appreciate your help. I will proceed the testing and keep you posted.
>>
>> Hello Hannes,
>>
>> I've tested the MPT SAS device controller, the patch works perfectly.
>> However, the MPT SPI still cannot work with the hotplugging. The lspci
>> of the LSI SPI device is listed:
>>
>> 00:10.0 SCSI storage controller [0100]: LSI Logic / Symbios Logic
>> 53c1030 PCI-X Fusion-MPT Dual Ultra320 SCSI [1000:0030] (rev 01)
>> Subsystem: VMware LSI Logic Parallel SCSI Controller [15ad:1976]
>> Control: I/O+ Mem+ BusMaster+ SpecCycle- MemWINV- VGASnoop-
>> ParErr- Stepping- SERR- FastB2B- DisINTx-
>> Status: Cap- 66MHz- UDF- FastB2B+ ParErr- DEVSEL=medium
>>> TAbort- <TAbort- <MAbort- >SERR- <PERR- INTx-
>> Latency: 64 (1500ns min, 63750ns max)
>> Interrupt: pin A routed to IRQ 17
>> Region 0: I/O ports at 1400 [size=256]
>> Region 1: Memory at feba0000 (64-bit, non-prefetchable) [size=128K]
>> Region 3: Memory at febc0000 (64-bit, non-prefetchable) [size=128K]
>> [virtual] Expansion ROM at 40008000 [disabled] [size=16K]
>> Kernel driver in use: mptspi
>>
>> The vendor id/device id[1000:0030] can also be found in the
>> mptspi_pci_table(drivers/message/fusion/mptspi.c).
>>
>> PCI_VENDOR_ID_LSI_LOGIC, MPI_MANUFACTPAGE_DEVID_53C1030
>>
>> If there is anything I missed in the testing? I appreciate your help.
>>
> Don't. Ever. Try.
>
> SCSI Parallel hotplugging is so loosely defined that essentially every
> vendor can (and did!) implement their own thing, and _still_ be spec
> compliant.
> Plus there is no guarantee that we even see events when a hot-add
> occurs. So any thingie we put in here has a really high chance of
> breaking other installations.
> I wouldn't even bother to attempt this; it's virtualized anyway so you
> can as well switch to the SAS emulation here.
> Or use a different SCSI parallel emulation like the Symbios/LSI 53c800 one.
>
> Cheers,
>
> Hannes
> --
> Dr. Hannes Reinecke Teamlead Storage & Networking
> [email protected] +49 911 74053 688
> SUSE LINUX GmbH, Maxfeldstr. 5, 90409 Nürnberg
> GF: F. Imendörffer, J. Smithard, J. Guild, D. Upmanyu, G. Norton
> HRB 21284 (AG Nürnberg)
Hello Hannes,
I totally understood your concern and appreciate your help again on
the hotplugging patch for MPT SAS controller. :)
From 1583042338608200416@xxx Fri Nov 03 11:00:20 +0000 2017
X-GM-THRID: 1579677274363332634
X-Gmail-Labels: Inbox,Category Forums,HistoricalUnread
On 11/03/2017 04:38 AM, Gavin Guo wrote:
> On Sat, Oct 28, 2017 at 11:35 AM, Gavin Guo <[email protected]> wrote:
>> On Fri, Oct 27, 2017 at 10:53 PM, Hannes Reinecke <[email protected]> wrote:
>>> On 10/27/2017 04:02 PM, Gavin Guo wrote:
>>>> Hi Hannes,
>>>>
>>>> Thank you for looking into the issue. If there is anything I can help
>>>> to test the patch? I appreciate your help. Thank you.
>>>>
>>> If you had checked linux-scsi you would have found this patch:
>>> '[PATCH] mptsas: Fixup device hotplug for VMWare ESXi', which I guess is
>>> already scheduled for inclusion in 4.14.
>>> Anything else I could help you with?
>>>
>>> Cheers,
>>>
>>> Hannes
>>> --
>>> Dr. Hannes Reinecke Teamlead Storage & Networking
>>> [email protected] +49 911 74053 688
>>> SUSE LINUX GmbH, Maxfeldstr. 5, 90409 Nürnberg
>>> GF: F. Imendörffer, J. Smithard, J. Guild, D. Upmanyu, G. Norton
>>> HRB 21284 (AG Nürnberg)
>>
>> Really appreciate your help. I will proceed the testing and keep you posted.
>
> Hello Hannes,
>
> I've tested the MPT SAS device controller, the patch works perfectly.
> However, the MPT SPI still cannot work with the hotplugging. The lspci
> of the LSI SPI device is listed:
>
> 00:10.0 SCSI storage controller [0100]: LSI Logic / Symbios Logic
> 53c1030 PCI-X Fusion-MPT Dual Ultra320 SCSI [1000:0030] (rev 01)
> Subsystem: VMware LSI Logic Parallel SCSI Controller [15ad:1976]
> Control: I/O+ Mem+ BusMaster+ SpecCycle- MemWINV- VGASnoop-
> ParErr- Stepping- SERR- FastB2B- DisINTx-
> Status: Cap- 66MHz- UDF- FastB2B+ ParErr- DEVSEL=medium
>> TAbort- <TAbort- <MAbort- >SERR- <PERR- INTx-
> Latency: 64 (1500ns min, 63750ns max)
> Interrupt: pin A routed to IRQ 17
> Region 0: I/O ports at 1400 [size=256]
> Region 1: Memory at feba0000 (64-bit, non-prefetchable) [size=128K]
> Region 3: Memory at febc0000 (64-bit, non-prefetchable) [size=128K]
> [virtual] Expansion ROM at 40008000 [disabled] [size=16K]
> Kernel driver in use: mptspi
>
> The vendor id/device id[1000:0030] can also be found in the
> mptspi_pci_table(drivers/message/fusion/mptspi.c).
>
> PCI_VENDOR_ID_LSI_LOGIC, MPI_MANUFACTPAGE_DEVID_53C1030
>
> If there is anything I missed in the testing? I appreciate your help.
>
Don't. Ever. Try.
SCSI Parallel hotplugging is so loosely defined that essentially every
vendor can (and did!) implement their own thing, and _still_ be spec
compliant.
Plus there is no guarantee that we even see events when a hot-add
occurs. So any thingie we put in here has a really high chance of
breaking other installations.
I wouldn't even bother to attempt this; it's virtualized anyway so you
can as well switch to the SAS emulation here.
Or use a different SCSI parallel emulation like the Symbios/LSI 53c800 one.
Cheers,
Hannes
--
Dr. Hannes Reinecke Teamlead Storage & Networking
[email protected] +49 911 74053 688
SUSE LINUX GmbH, Maxfeldstr. 5, 90409 Nürnberg
GF: F. Imendörffer, J. Smithard, J. Guild, D. Upmanyu, G. Norton
HRB 21284 (AG Nürnberg)
From 1583014624689987804@xxx Fri Nov 03 03:39:50 +0000 2017
X-GM-THRID: 1579677274363332634
X-Gmail-Labels: Inbox,Category Forums,HistoricalUnread
On Sat, Oct 28, 2017 at 11:35 AM, Gavin Guo <[email protected]> wrote:
> On Fri, Oct 27, 2017 at 10:53 PM, Hannes Reinecke <[email protected]> wrote:
>> On 10/27/2017 04:02 PM, Gavin Guo wrote:
>>> Hi Hannes,
>>>
>>> Thank you for looking into the issue. If there is anything I can help
>>> to test the patch? I appreciate your help. Thank you.
>>>
>> If you had checked linux-scsi you would have found this patch:
>> '[PATCH] mptsas: Fixup device hotplug for VMWare ESXi', which I guess is
>> already scheduled for inclusion in 4.14.
>> Anything else I could help you with?
>>
>> Cheers,
>>
>> Hannes
>> --
>> Dr. Hannes Reinecke Teamlead Storage & Networking
>> [email protected] +49 911 74053 688
>> SUSE LINUX GmbH, Maxfeldstr. 5, 90409 Nürnberg
>> GF: F. Imendörffer, J. Smithard, J. Guild, D. Upmanyu, G. Norton
>> HRB 21284 (AG Nürnberg)
>
> Really appreciate your help. I will proceed the testing and keep you posted.
Hello Hannes,
I've tested the MPT SAS device controller, the patch works perfectly.
However, the MPT SPI still cannot work with the hotplugging. The lspci
of the LSI SPI device is listed:
00:10.0 SCSI storage controller [0100]: LSI Logic / Symbios Logic
53c1030 PCI-X Fusion-MPT Dual Ultra320 SCSI [1000:0030] (rev 01)
Subsystem: VMware LSI Logic Parallel SCSI Controller [15ad:1976]
Control: I/O+ Mem+ BusMaster+ SpecCycle- MemWINV- VGASnoop-
ParErr- Stepping- SERR- FastB2B- DisINTx-
Status: Cap- 66MHz- UDF- FastB2B+ ParErr- DEVSEL=medium
>TAbort- <TAbort- <MAbort- >SERR- <PERR- INTx-
Latency: 64 (1500ns min, 63750ns max)
Interrupt: pin A routed to IRQ 17
Region 0: I/O ports at 1400 [size=256]
Region 1: Memory at feba0000 (64-bit, non-prefetchable) [size=128K]
Region 3: Memory at febc0000 (64-bit, non-prefetchable) [size=128K]
[virtual] Expansion ROM at 40008000 [disabled] [size=16K]
Kernel driver in use: mptspi
The vendor id/device id[1000:0030] can also be found in the
mptspi_pci_table(drivers/message/fusion/mptspi.c).
PCI_VENDOR_ID_LSI_LOGIC, MPI_MANUFACTPAGE_DEVID_53C1030
If there is anything I missed in the testing? I appreciate your help.
From 1582470841856653969@xxx Sat Oct 28 03:36:38 +0000 2017
X-GM-THRID: 1579677274363332634
X-Gmail-Labels: Inbox,Category Forums
On Fri, Oct 27, 2017 at 10:53 PM, Hannes Reinecke <[email protected]> wrote:
> On 10/27/2017 04:02 PM, Gavin Guo wrote:
>> Hi Hannes,
>>
>> Thank you for looking into the issue. If there is anything I can help
>> to test the patch? I appreciate your help. Thank you.
>>
> If you had checked linux-scsi you would have found this patch:
> '[PATCH] mptsas: Fixup device hotplug for VMWare ESXi', which I guess is
> already scheduled for inclusion in 4.14.
> Anything else I could help you with?
>
> Cheers,
>
> Hannes
> --
> Dr. Hannes Reinecke Teamlead Storage & Networking
> [email protected] +49 911 74053 688
> SUSE LINUX GmbH, Maxfeldstr. 5, 90409 Nürnberg
> GF: F. Imendörffer, J. Smithard, J. Guild, D. Upmanyu, G. Norton
> HRB 21284 (AG Nürnberg)
Really appreciate your help. I will proceed the testing and keep you posted.
From 1582422884758682385@xxx Fri Oct 27 14:54:22 +0000 2017
X-GM-THRID: 1579677274363332634
X-Gmail-Labels: Inbox,Category Forums
On 10/27/2017 04:02 PM, Gavin Guo wrote:
> Hi Hannes,
>
> Thank you for looking into the issue. If there is anything I can help
> to test the patch? I appreciate your help. Thank you.
>
If you had checked linux-scsi you would have found this patch:
'[PATCH] mptsas: Fixup device hotplug for VMWare ESXi', which I guess is
already scheduled for inclusion in 4.14.
Anything else I could help you with?
Cheers,
Hannes
--
Dr. Hannes Reinecke Teamlead Storage & Networking
[email protected] +49 911 74053 688
SUSE LINUX GmbH, Maxfeldstr. 5, 90409 Nürnberg
GF: F. Imendörffer, J. Smithard, J. Guild, D. Upmanyu, G. Norton
HRB 21284 (AG Nürnberg)
From 1582419714560320171@xxx Fri Oct 27 14:03:59 +0000 2017
X-GM-THRID: 1579677274363332634
X-Gmail-Labels: Inbox,Category Forums
Hi Hannes,
Thank you for looking into the issue. If there is anything I can help
to test the patch? I appreciate your help. Thank you.
On Fri, Oct 6, 2017 at 9:08 AM, Gavin Guo <[email protected]> wrote:
> On Mon, Oct 2, 2017 at 10:32 PM, Hannes Reinecke <[email protected]> wrote:
>> On 09/27/2017 09:33 AM, Gavin Guo wrote:
>>> There is a problem in the latest upstream kernel with the device:
>>>
>>> $ grep -i lsi lspci
>>> 03:00.0 Serial Attached SCSI controller [0107]: LSI Logic / Symbios
>>> Logic SAS1068 PCI-X Fusion-MPT SAS [1000:0054] (rev 01)
>>>
>>> The device is simulated by the VMware ESXi 5.5
>>>
>>> When hotplugging a new disk to the Guest Ubuntu OS, the latest kernel
>>> cannot automatically probe the disk. However, on the v3.19.0-80.88
>>> kernel, the disk can be dynamically probed and show the following
>>> info message:
>>>
>>> mptsas: ioc0: attaching ssp device: fw_channel 0, fw_id 1, phy 1,
>>> sas_addr 0x5000c29a6bdae0f5
>>> scsi 2:0:1:0: Direct-Access VMware Virtual disk 1.0 PQ: 0
>>> ANSI: 2
>>> sd 2:0:1:0: Attached scsi generic sg2 type 0
>>> sd 2:0:1:0: [sdb] 2097152 512-byte logical blocks: (1.07 GB/1.00 GiB)
>>> sd 2:0:1:0: [sdb] Write Protect is off
>>> sd 2:0:1:0: [sdb] Mode Sense: 61 00 00 00
>>> sd 2:0:1:0: [sdb] Cache data unavailable
>>> sd 2:0:1:0: [sdb] Assuming drive cache: write through
>>> sdb: unknown partition table
>>> sd 2:0:1:0: [sdb] Attached SCSI disk
>>>
>>> After looking up the message:
>>> mptsas: ioc0: attaching ssp device: fw_channel 0, fw_id 1, phy 1,
>>> sas_addr 0x5000c29a6bdae0f5
>>>
>>> I found it comes from the path:
>>> mptsas_firmware_event_work -> mptsas_send_sas_event ->
>>> mptsas_hotplug_work -> mptsas_add_end_device
>>>
>>> I'll appreciate if anyone can give the idea: If it's possible that the
>>> irq from the simulated LSI SAS controller didn't come in to trigger
>>> the event? However, it can work on the v3.19 kernel so if there is
>>> any driver implementation issue in the latest kernel.
>>>
>> This is an issue with the mptsas driver, who originally assumed that no
>> system will have direct-attached SAS devices.
>> VMWare chose to implement exactly that, so the hotplug detection logic
>> is flawed here.
>> I'll be sending a patch.
>>
>> Cheers,
>>
>> Hannes
>> --
>> Dr. Hannes Reinecke Teamlead Storage & Networking
>> [email protected] +49 911 74053 688
>> SUSE LINUX GmbH, Maxfeldstr. 5, 90409 Nürnberg
>> GF: F. Imendörffer, J. Smithard, J. Guild, D. Upmanyu, G. Norton
>> HRB 21284 (AG Nürnberg)
>
> Thank you in advance. Please add me to the CC list when you send out
> the patch and I can help to verify that.
From 1580468458747068756@xxx Fri Oct 06 01:09:36 +0000 2017
X-GM-THRID: 1579677274363332634
X-Gmail-Labels: Inbox,Category Forums
On Mon, Oct 2, 2017 at 10:32 PM, Hannes Reinecke <[email protected]> wrote:
> On 09/27/2017 09:33 AM, Gavin Guo wrote:
>> There is a problem in the latest upstream kernel with the device:
>>
>> $ grep -i lsi lspci
>> 03:00.0 Serial Attached SCSI controller [0107]: LSI Logic / Symbios
>> Logic SAS1068 PCI-X Fusion-MPT SAS [1000:0054] (rev 01)
>>
>> The device is simulated by the VMware ESXi 5.5
>>
>> When hotplugging a new disk to the Guest Ubuntu OS, the latest kernel
>> cannot automatically probe the disk. However, on the v3.19.0-80.88
>> kernel, the disk can be dynamically probed and show the following
>> info message:
>>
>> mptsas: ioc0: attaching ssp device: fw_channel 0, fw_id 1, phy 1,
>> sas_addr 0x5000c29a6bdae0f5
>> scsi 2:0:1:0: Direct-Access VMware Virtual disk 1.0 PQ: 0
>> ANSI: 2
>> sd 2:0:1:0: Attached scsi generic sg2 type 0
>> sd 2:0:1:0: [sdb] 2097152 512-byte logical blocks: (1.07 GB/1.00 GiB)
>> sd 2:0:1:0: [sdb] Write Protect is off
>> sd 2:0:1:0: [sdb] Mode Sense: 61 00 00 00
>> sd 2:0:1:0: [sdb] Cache data unavailable
>> sd 2:0:1:0: [sdb] Assuming drive cache: write through
>> sdb: unknown partition table
>> sd 2:0:1:0: [sdb] Attached SCSI disk
>>
>> After looking up the message:
>> mptsas: ioc0: attaching ssp device: fw_channel 0, fw_id 1, phy 1,
>> sas_addr 0x5000c29a6bdae0f5
>>
>> I found it comes from the path:
>> mptsas_firmware_event_work -> mptsas_send_sas_event ->
>> mptsas_hotplug_work -> mptsas_add_end_device
>>
>> I'll appreciate if anyone can give the idea: If it's possible that the
>> irq from the simulated LSI SAS controller didn't come in to trigger
>> the event? However, it can work on the v3.19 kernel so if there is
>> any driver implementation issue in the latest kernel.
>>
> This is an issue with the mptsas driver, who originally assumed that no
> system will have direct-attached SAS devices.
> VMWare chose to implement exactly that, so the hotplug detection logic
> is flawed here.
> I'll be sending a patch.
>
> Cheers,
>
> Hannes
> --
> Dr. Hannes Reinecke Teamlead Storage & Networking
> [email protected] +49 911 74053 688
> SUSE LINUX GmbH, Maxfeldstr. 5, 90409 Nürnberg
> GF: F. Imendörffer, J. Smithard, J. Guild, D. Upmanyu, G. Norton
> HRB 21284 (AG Nürnberg)
Thank you in advance. Please add me to the CC list when you send out
the patch and I can help to verify that.
From 1580156714076320913@xxx Mon Oct 02 14:34:34 +0000 2017
X-GM-THRID: 1579677274363332634
X-Gmail-Labels: Inbox,Category Forums
On 09/27/2017 09:33 AM, Gavin Guo wrote:
> There is a problem in the latest upstream kernel with the device:
>
> $ grep -i lsi lspci
> 03:00.0 Serial Attached SCSI controller [0107]: LSI Logic / Symbios
> Logic SAS1068 PCI-X Fusion-MPT SAS [1000:0054] (rev 01)
>
> The device is simulated by the VMware ESXi 5.5
>
> When hotplugging a new disk to the Guest Ubuntu OS, the latest kernel
> cannot automatically probe the disk. However, on the v3.19.0-80.88
> kernel, the disk can be dynamically probed and show the following
> info message:
>
> mptsas: ioc0: attaching ssp device: fw_channel 0, fw_id 1, phy 1,
> sas_addr 0x5000c29a6bdae0f5
> scsi 2:0:1:0: Direct-Access VMware Virtual disk 1.0 PQ: 0
> ANSI: 2
> sd 2:0:1:0: Attached scsi generic sg2 type 0
> sd 2:0:1:0: [sdb] 2097152 512-byte logical blocks: (1.07 GB/1.00 GiB)
> sd 2:0:1:0: [sdb] Write Protect is off
> sd 2:0:1:0: [sdb] Mode Sense: 61 00 00 00
> sd 2:0:1:0: [sdb] Cache data unavailable
> sd 2:0:1:0: [sdb] Assuming drive cache: write through
> sdb: unknown partition table
> sd 2:0:1:0: [sdb] Attached SCSI disk
>
> After looking up the message:
> mptsas: ioc0: attaching ssp device: fw_channel 0, fw_id 1, phy 1,
> sas_addr 0x5000c29a6bdae0f5
>
> I found it comes from the path:
> mptsas_firmware_event_work -> mptsas_send_sas_event ->
> mptsas_hotplug_work -> mptsas_add_end_device
>
> I'll appreciate if anyone can give the idea: If it's possible that the
> irq from the simulated LSI SAS controller didn't come in to trigger
> the event? However, it can work on the v3.19 kernel so if there is
> any driver implementation issue in the latest kernel.
>
This is an issue with the mptsas driver, who originally assumed that no
system will have direct-attached SAS devices.
VMWare chose to implement exactly that, so the hotplug detection logic
is flawed here.
I'll be sending a patch.
Cheers,
Hannes
--
Dr. Hannes Reinecke Teamlead Storage & Networking
[email protected] +49 911 74053 688
SUSE LINUX GmbH, Maxfeldstr. 5, 90409 Nürnberg
GF: F. Imendörffer, J. Smithard, J. Guild, D. Upmanyu, G. Norton
HRB 21284 (AG Nürnberg)
From 1579677274363332634@xxx Wed Sep 27 07:34:04 +0000 2017
X-GM-THRID: 1579677274363332634
X-Gmail-Labels: Inbox,Category Forums