On Wed, 3 Mar 2021 00:22:36 +0000
Sergei Trofimovich <[email protected]> wrote:
> On Tue, 2 Mar 2021 23:31:32 +0100
> John Paul Adrian Glaubitz <[email protected]> wrote:
>
> > Hi Sergei!
> >
> > On 3/2/21 11:26 PM, Sergei Trofimovich wrote:
> > > Gave v5.12-rc1 a try today and got a similar boot failure around
> > > hpsa queue initialization, but my failure is later:
> > > https://dev.gentoo.org/~slyfox/configs/guppy-dmesg-5.12-rc1
> > > Maybe I get different error because I flipped on most debugging
> > > kernel options :)
> > >
> > > Looks like 'ERROR: Invalid distance value range' while being
> > > very scary are harmless. It's just a new spammy way for kernel
> > > to report lack of NUMA config on the machine (no SRAT and SLIT
> > > ACPI tables).
> > >
> > > At least I get hpsa detected on PCI bus. But I guess it's discovered
> > > configuration is very wrong as I get unaligned accesses:
> > > [ 19.811570] kernel unaligned access to 0xe000000105dd8295, ip=0xa000000100b874d1
> > >
> > > Bisecting now.
> >
> > Sounds good. I guess we should get Jens' fix for the signal regression
> > merged as well as your two fixes for strace.
>
> "bisected" (cheated halfway through) and verified that reverting
> f749d8b7a9896bc6e5ffe104cc64345037e0b152 makes rx3600 boot again.
>
> CCing authors who might be able to help us here.
>
> commit f749d8b7a9896bc6e5ffe104cc64345037e0b152
> Author: Don Brace <[email protected]>
> Date: Mon Feb 15 16:26:57 2021 -0600
>
> scsi: hpsa: Correct dev cmds outstanding for retried cmds
>
> Prevent incrementing device->commands_outstanding for ioaccel command
> retries that are driver initiated. If the command goes through the retry
> path, the device->commands_outstanding counter has already accounted for
> the number of commands outstanding to the device. Only commands going
> through function hpsa_cmd_resolve_events decrement this counter.
>
> - ioaccel commands go to either HBA disks or to logical volumes comprised
> of SSDs.
>
> The extra increment is causing device resets to hang.
>
> - Resets wait for all device outstanding commands to complete before
> returning.
>
> Replace unused field abort_pending with retry_pending. This is a
> maintenance driver so these changes have the least impact/risk.
>
> Link: https://lore.kernel.org/r/161342801747.29388.13045495968308188518.stgit@brunhilda
> Tested-by: Joe Szczypek <[email protected]>
> Reviewed-by: Scott Benesh <[email protected]>
> Reviewed-by: Scott Teel <[email protected]>
> Reviewed-by: Tomas Henzl <[email protected]>
> Signed-off-by: Don Brace <[email protected]>
> Signed-off-by: Martin K. Petersen <[email protected]>
>
> Don, do you happen to know why this patch caused some controller init failure
> for device
> 14:01.0 RAID bus controller: Hewlett-Packard Company Smart Array P600
> ?
>
> Boot failure: https://dev.gentoo.org/~slyfox/configs/guppy-dmesg-5.12-rc1
> Boot success: https://dev.gentoo.org/~slyfox/configs/guppy-dmesg-5.12-rc1-good
>
> The difference between the two boots is
> f749d8b7a9896bc6e5ffe104cc64345037e0b152 reverted on top of 5.12-rc1
> in -good case.
>
> Looks like hpsa controller fails to initialize in bad case (could be a race?).
Also CCing hpsa maintainer mailing lists.
Looking more into the suspect commit
https://git.kernel.org/pub/scm/linux/kernel/git/torvalds/linux.git/commit/?id=f749d8b7a9896bc6e5ffe104cc64345037e0b152
it roughly does the:
@@ -448,7 +448,7 @@ struct CommandList {
*/
struct hpsa_scsi_dev_t *phys_disk;
- int abort_pending;
+ bool retry_pending;
struct hpsa_scsi_dev_t *device;
atomic_t refcount; /* Must be last to avoid memset in hpsa_cmd_init() */
} __aligned(COMMANDLIST_ALIGNMENT);
...
@@ -1151,7 +1151,10 @@ static void __enqueue_cmd_and_start_io(struct ctlr_info *h,
{
dial_down_lockup_detection_during_fw_flash(h, c);
atomic_inc(&h->commands_outstanding);
- if (c->device)
+ /*
+ * Check to see if the command is being retried.
+ */
+ if (c->device && !c->retry_pending)
atomic_inc(&c->device->commands_outstanding);
But I don't immediately see anything wrong with it.
--
Sergei