Qubes OS users are reporting that MTP doesn't work with USB passthrough.
Fastboot (used for flashing a custom OS to an Android device) also
doesn't work. Kernel-mode drivers, such as Bluetooth and USB storage,
seem to usually work as expected. Since MTP and fastboot are both
implemented in userspace, it appears that there is some problem with the
interaction of usbip, our USB proxy (which is based on USBIP), and
userspace programs that interact with USB devices directly.
The bug report can be found at [1] and the source code for the USB proxy
can be found at [2]. The script used on the sending side (the one with
the physical USB controller) is at [3] and the script used by the
receiving side (the one the device is attached to) is at [4]. All of
these links are for the current version as of this email being sent, so
that anyone looking at this email in the future doesn't get confused.
Is this a bug in usbip, or is this due to usbip being used incorrectly?
I'm happy to provide additional information needed to debug the problem,
but I don't have access to the reporter's system.
[1]: https://github.com/QubesOS/qubes-issues/issues/6330
[2]: https://github.com/QubesOS/qubes-app-linux-usb-proxy/tree/57ab3940d450b18e570da57886d65cb5707aa60f
[3]: https://github.com/QubesOS/qubes-app-linux-usb-proxy/blob/57ab3940d450b18e570da57886d65cb5707aa60f/src/usb-export
[4]: https://github.com/QubesOS/qubes-app-linux-usb-proxy/blob/57ab3940d450b18e570da57886d65cb5707aa60f/src/usb-import
--
Sincerely,
Demi Marie Obenour (she/her/hers)
Invisible Things Lab
On Tue, Mar 05, 2024 at 12:52:22AM +0100, Marek Marczykowski-G?recki wrote:
> On Mon, Mar 04, 2024 at 11:46:04PM +0100, Marek Marczykowski-G?recki wrote:
> > Terminology:
> > 1. sys-usb - a VM where USB controller (a PCI device) lives; here
> > usbip-host is attached to the device
> > 2. testvm - target VM where usbip is connected; here vhci-hcd is used
> > 3. qvm-usb - tool that connects the above two (equivalent of
> > userspace part of standard usbip)
> >
> > Specific steps:
> > 1. Connect android phone - at this point it's only in sys-usb
> > 2. Switch its mode to file transfer - observe reconnect in sys-usb
> > 3. Use qvm-usb to attach it to the testvm
> > 4. Call jmtpfs -d /mnt in testvm
>
> Or maybe reset or something is involved here too?
>
> After using qvm-usb to attach _and detach_ the device, it stays bound to
> usbip-host driver (that's intentional). But then, even after re-binding
> back to the "usb" driver, jmtpfs called in sys-usb directly fails the
> same way, including failure to reset.
>
> In fact, even without usbip involved at all, jmtpfs directly in sys-usb
> works only once. The second attempt (without either physically reconnecting
> the phone, or changing its more to "no data transfer" and back to "file
> transfer") fails the same way. After terminating the first instance, I
> see just this logged:
>
> [921332.525210] usb 2-1: reset high-speed USB device number 22 using xhci_hcd
If something doesn't work when usbip isn't involved, you definitely
shouldn't expect it to work when usbip _is_ involved.
It sounds like you're facing more than one type of problem. The best
approach is to attack them separately.
I'd start with problems that exist only on sys-usb first -- keeping
usbip out of the picture will make everything much simpler. You can try
capturing a usbmon trace, starting from before the phone is attached,
and continuing on through the mode changes and failures. In fact, break
it up into several traces, each starting just before one of the major
events (initial plug-in, mode change, whatever).
Once that's under control, I suggest using usbmon on both sides (sys-usb
and testvm) of the usbip connection. But start without usbip.
Alan Stern
On Mon, Mar 04, 2024 at 07:06:08PM -0500, Demi Marie Obenour wrote:
> On Tue, Mar 05, 2024 at 12:52:22AM +0100, Marek Marczykowski-Górecki wrote:
> > On Mon, Mar 04, 2024 at 11:46:04PM +0100, Marek Marczykowski-Górecki wrote:
> > > On Mon, Mar 04, 2024 at 09:04:00PM +0000, Greg KH wrote:
> > > > On Mon, Mar 04, 2024 at 03:01:51PM -0500, Demi Marie Obenour wrote:
> > > > > Qubes OS users are reporting that MTP doesn't work with USB passthrough.
> > > > > Fastboot (used for flashing a custom OS to an Android device) also
> > > > > doesn't work. Kernel-mode drivers, such as Bluetooth and USB storage,
> > > > > seem to usually work as expected. Since MTP and fastboot are both
> > > > > implemented in userspace, it appears that there is some problem with the
> > > > > interaction of usbip, our USB proxy (which is based on USBIP), and
> > > > > userspace programs that interact with USB devices directly.
> > > > >
> > > > > The bug report can be found at [1] and the source code for the USB proxy
> > > > > can be found at [2]. The script used on the sending side (the one with
> > > > > the physical USB controller) is at [3] and the script used by the
> > > > > receiving side (the one the device is attached to) is at [4]. All of
> > > > > these links are for the current version as of this email being sent, so
> > > > > that anyone looking at this email in the future doesn't get confused.
> > > > >
> > > > > Is this a bug in usbip, or is this due to usbip being used incorrectly?
> > > >
> > > > I'm amazed that usbip works with usbfs at all, I didn't think that was a
> > > > thing.
> > > >
> > > > If you have a reproducer, or some error messages somewhere, that might
> > > > be the easiest way forward. In reading the bug report, it looks like
> > > > the "bridge" you all made can't handle the device disconnecting itself
> > > > properly? But that's just a guess, could be anything.
> > >
> > > Device disconnecting itself indeed is an issue (our proxy doesn't
> > > automatically reconnect it, at least not yet). But that's definitely not
> > > the only issue, things break also when disconnect is not involved.
> > >
> > > Terminology:
> > > 1. sys-usb - a VM where USB controller (a PCI device) lives; here
> > > usbip-host is attached to the device
> > > 2. testvm - target VM where usbip is connected; here vhci-hcd is used
> > > 3. qvm-usb - tool that connects the above two (equivalent of
> > > userspace part of standard usbip)
> > >
> > > Specific steps:
> > > 1. Connect android phone - at this point it's only in sys-usb
> > > 2. Switch its mode to file transfer - observe reconnect in sys-usb
> > > 3. Use qvm-usb to attach it to the testvm
> > > 4. Call jmtpfs -d /mnt in testvm
> >
> > Or maybe reset or something is involved here too?
> >
> > After using qvm-usb to attach _and detach_ the device, it stays bound to
> > usbip-host driver (that's intentional). But then, even after re-binding
> > back to the "usb" driver, jmtpfs called in sys-usb directly fails the
> > same way, including failure to reset.
> >
> > In fact, even without usbip involved at all, jmtpfs directly in sys-usb
> > works only once. The second attempt (without either physically reconnecting
> > the phone, or changing its more to "no data transfer" and back to "file
> > transfer") fails the same way. After terminating the first instance, I
> > see just this logged:
> >
> > [921332.525210] usb 2-1: reset high-speed USB device number 22 using xhci_hcd
>
> What happens if the device is not bound to _any_ driver in sys-usb? For
> instance, does the problem go away if the only USB-related drivers in
> sys-usb are xhci-pci, usbip-host, and their dependencies? If not, what
> about if sys-usb's kernel has no other USB-related drivers included at
> all?
I don't think I can get rid of the "usb" driver, and that's what binds to
the device. There is no driver bound to the first interface of the
device (2-1:1.0) at all.
> Also, if you have not done so already, could you try uninstalling gvfs
> from sys-usb's template? gvfs might be interacting with the device.
> Using a -minimal template might help.
I highly doubt it would change anything.
> > Since both problematic cases involve Android, maybe the actual issue is
> > somewhere in Android instead of usbip? But then, fastboot could be a
> > completely different issue, likely related to reconnection (this one I
> > haven't tried to reproduce, and can't find much details in our issues
> > tracker).
> > On the other hand, USB tethering works just fine, so at least some
> > Android functions do work with usbip (but then, it's a kernel driver,
> > not usbfs).
> > Could be also an issue with just MTP implementation on either side of
> > the connection (I did tried also gvfs instead of jmtpfs with similar
> > result, but they likely use the same implementation). Since that's
> > Android, the device side is Linux too, but I don't know how MTP is
> > implemented there (I don't see MTP gadget function in upstream Linux).
>
> I suspect MTP is implemented in userspace somewhere in AOSP.
--
Best Regards,
Marek Marczykowski-Górecki
Invisible Things Lab
On Mon, Mar 04, 2024 at 10:19:24PM -0500, Alan Stern wrote:
> On Tue, Mar 05, 2024 at 12:52:22AM +0100, Marek Marczykowski-Górecki wrote:
> > On Mon, Mar 04, 2024 at 11:46:04PM +0100, Marek Marczykowski-Górecki wrote:
> > > Terminology:
> > > 1. sys-usb - a VM where USB controller (a PCI device) lives; here
> > > usbip-host is attached to the device
> > > 2. testvm - target VM where usbip is connected; here vhci-hcd is used
> > > 3. qvm-usb - tool that connects the above two (equivalent of
> > > userspace part of standard usbip)
> > >
> > > Specific steps:
> > > 1. Connect android phone - at this point it's only in sys-usb
> > > 2. Switch its mode to file transfer - observe reconnect in sys-usb
> > > 3. Use qvm-usb to attach it to the testvm
> > > 4. Call jmtpfs -d /mnt in testvm
> >
> > Or maybe reset or something is involved here too?
> >
> > After using qvm-usb to attach _and detach_ the device, it stays bound to
> > usbip-host driver (that's intentional). But then, even after re-binding
> > back to the "usb" driver, jmtpfs called in sys-usb directly fails the
> > same way, including failure to reset.
> >
> > In fact, even without usbip involved at all, jmtpfs directly in sys-usb
> > works only once. The second attempt (without either physically reconnecting
> > the phone, or changing its more to "no data transfer" and back to "file
> > transfer") fails the same way. After terminating the first instance, I
> > see just this logged:
> >
> > [921332.525210] usb 2-1: reset high-speed USB device number 22 using xhci_hcd
>
> If something doesn't work when usbip isn't involved, you definitely
> shouldn't expect it to work when usbip _is_ involved.
With usbip, jmtpfs fails the _first_ time, instead of only the _second_
time. This might be related: maybe the attachment of usbip acts like
the first attach?
--
Sincerely,
Demi Marie Obenour (she/her/hers)
Invisible Things Lab