Received-SPF: pass (google.com: domain of linux-kernel+bounces-119051-linux.lists.archive=gmail.com@vger.kernel.org designates 2604:1380:40f1:3f00::1 as permitted sender) client-ip=2604:1380:40f1:3f00::1;
Feedback-ID: ie8e14432:Fastmail
Date: Tue, 26 Mar 2024 22:18:58 +0900
From: Takashi Sakamoto <o-takashi@sakamocchi.jp>
To: Bjorn Helgaas <helgaas@kernel.org>
Cc: linux1394-devel@lists.sourceforge.net, linux-kernel@vger.kernel.org,
	linux-pci@vger.kernel.org, edmund.raile@proton.me
Subject: Re: [PATCH v2] PCI: Mark LSI FW643 to avoid bus reset
Message-ID: <20240326131858.GA140624@workstation.local>
Mail-Followup-To: Bjorn Helgaas <helgaas@kernel.org>,
	linux1394-devel@lists.sourceforge.net, linux-kernel@vger.kernel.org,
	linux-pci@vger.kernel.org, edmund.raile@proton.me
References: <20240325012135.36861-1-o-takashi@sakamocchi.jp>
 <20240325144149.GA1432902@bhelgaas>
Precedence: bulk
MIME-Version: 1.0
Content-Type: text/plain; charset=us-ascii
Content-Disposition: inline
In-Reply-To: <20240325144149.GA1432902@bhelgaas>

Hi Bjorn,

Thanks for your reply.

On Mon, Mar 25, 2024 at 09:41:49AM -0500, Bjorn Helgaas wrote:
> So even without this patch, you are able to pass the FW643 to a VM
> with VFIO, and you don't see any issues caused by VFIO resetting the
> device?
 
Absolutely yes, at least in my VM, for recent years to maintain Linux
FireWire subsystem and ALSA firewire stack.

> Can you collect the output of:
> 
>   $ find /sys/devices -name reset_method | xargs grep .
 
```
$ sudo find /sys/devices -name reset_method | xargs grep .
/sys/devices/pci0000:00/0000:00:01.2/0000:03:00.0/0000:04:09.0/0000:09:00.0/reset_method:bus
/sys/devices/pci0000:00/0000:00:01.2/0000:03:00.0/0000:04:09.0/reset_method:pm
/sys/devices/pci0000:00/0000:00:01.2/0000:03:00.0/0000:04:02.0/0000:06:00.0/reset_method:pm bus
/sys/devices/pci0000:00/0000:00:01.2/0000:03:00.0/0000:04:02.0/reset_method:pm
/sys/devices/pci0000:00/0000:00:01.2/0000:03:00.0/0000:04:05.0/0000:07:00.0/reset_method:bus
/sys/devices/pci0000:00/0000:00:01.2/0000:03:00.0/0000:04:05.0/reset_method:pm
/sys/devices/pci0000:00/0000:00:01.2/0000:03:00.0/0000:04:0a.0/0000:0a:00.0/reset_method:bus
/sys/devices/pci0000:00/0000:00:01.2/0000:03:00.0/0000:04:0a.0/reset_method:pm
/sys/devices/pci0000:00/0000:00:01.2/0000:03:00.0/0000:04:08.0/0000:08:00.0/reset_method:bus
/sys/devices/pci0000:00/0000:00:01.2/0000:03:00.0/0000:04:08.0/0000:08:00.3/reset_method:pm
/sys/devices/pci0000:00/0000:00:01.2/0000:03:00.0/0000:04:08.0/reset_method:pm
/sys/devices/pci0000:00/0000:00:01.2/0000:03:00.0/0000:04:01.0/reset_method:pm bus
/sys/devices/pci0000:00/0000:00:01.2/0000:03:00.0/0000:04:01.0/0000:05:00.0/reset_method:device_specific flr bus
/sys/devices/pci0000:00/0000:00:01.2/0000:03:00.0/reset_method:pm bus
/sys/devices/pci0000:00/0000:00:01.2/reset_method:pm
/sys/devices/pci0000:00/0000:00:08.1/0000:0c:00.0/reset_method:flr bus
/sys/devices/pci0000:00/0000:00:08.1/0000:0c:00.3/reset_method:pm
/sys/devices/pci0000:00/0000:00:08.1/0000:0c:00.1/reset_method:flr pm
/sys/devices/pci0000:00/0000:00:08.1/reset_method:pm
/sys/devices/pci0000:00/0000:00:08.1/0000:0c:00.4/reset_method:pm
/sys/devices/pci0000:00/0000:00:01.1/0000:01:00.0/0000:02:00.0/reset_method:pm bus
/sys/devices/pci0000:00/0000:00:01.1/0000:01:00.0/reset_method:bus
/sys/devices/pci0000:00/0000:00:01.1/reset_method:pm
/sys/devices/pci0000:00/0000:00:01.6/0000:0b:00.0/reset_method:flr bus
/sys/devices/pci0000:00/0000:00:01.6/reset_method:pm
```

If you need each PCI bus bridge information, I can provide it to you.
but I can promise they are typical hardware in AMD CPU or chipset for
Zen generation and nothing special.

> You should be able to manually reset the device with something like
> this (I don't know your topology, so you might have to replace "1d.6"
> with the bridge leading to 06:00.0):
> 
>   # sudo echo 1 > # /sys/devices/pci0000:00/0000:00:1d.6/0000:06:00.0/reset
 
```
$ echo 1 > sudo tee -a /sys/devices/pci0000:00/0000:00:01.2/0000:03:00.0/0000:04:02.0/0000:06:00.0/reset
(nothing happens)
$ journalctl -k -n10
(nothing specific)
```

Would I ask you any point to check after the reset operation?

> I don't *know* of a reason why a Secondary Bus Reset would work
> correctly on your hardware but not on a Mac, but there could be
> something weird going on.

Note that the hardware provided by Apple for the past decade has no
IEEE 1394 interface, thus the patch author seems to use any kind of
bus extension to connect the issued 1394 OHCI hardware. I guess:

* Apple Thunderbolt Display
   * https://lore.kernel.org/linux-pci/1372860295-8306-1-git-send-email-mika.westerberg@linux.intel.com/
* Apple Thunderbolt-OHCI1394 adapter
   * I know FW643 is used for the product.
* Some kind of eGPU box

> Does the patch cause a problem for you?  (Other than the fact that the
> device leaks state between VMs?)

It takes a bit time for me to set up my system with self-compiled v6.9-rc1
kernel. However the leak between VMs is really inconvenient to me by itself.


Thanks

Takashi Sakamoto