Received-SPF: pass (google.com: domain of linux-kernel+bounces-91456-linux.lists.archive=gmail.com@vger.kernel.org designates 2604:1380:4601:e00::3 as permitted sender) client-ip=2604:1380:4601:e00::3;
Feedback-ID: i1568416f:Fastmail
Date: Tue, 5 Mar 2024 01:33:00 +0100
From: Marek =?utf-8?Q?Marczykowski-G=C3=B3recki?= <marmarek@invisiblethingslab.com>
To: Demi Marie Obenour <demi@invisiblethingslab.com>
Cc: Greg KH <gregkh@linuxfoundation.org>, linux-usb@vger.kernel.org,
	Linux Kernel Mailing List <linux-kernel@vger.kernel.org>
Subject: Re: usbip doesn't work with userspace code that accesses USB devices
Message-ID: <ZeZoPAJw2Ip_Lgch@mail-itl>
References: <ZeYov0k8njwcZzGX@itl-email>
 <2024030406-kilogram-raving-33c5@gregkh>
 <ZeZPLX6z5pyn2Eh_@mail-itl>
 <ZeZetkR-i1bCDr9v@mail-itl>
 <ZeZh82gIm2xo_UFM@itl-email>
Precedence: bulk
MIME-Version: 1.0
Content-Type: multipart/signed; micalg=pgp-sha256;
	protocol="application/pgp-signature"; boundary="WN7gfDbbAqHw1d2/"
Content-Disposition: inline
In-Reply-To: <ZeZh82gIm2xo_UFM@itl-email>


--WN7gfDbbAqHw1d2/
Content-Type: text/plain; protected-headers=v1; charset=utf-8
Content-Disposition: inline
Content-Transfer-Encoding: quoted-printable
Date: Tue, 5 Mar 2024 01:33:00 +0100
From: Marek =?utf-8?Q?Marczykowski-G=C3=B3recki?= <marmarek@invisiblethingslab.com>
To: Demi Marie Obenour <demi@invisiblethingslab.com>
Cc: Greg KH <gregkh@linuxfoundation.org>, linux-usb@vger.kernel.org,
	Linux Kernel Mailing List <linux-kernel@vger.kernel.org>
Subject: Re: usbip doesn't work with userspace code that accesses USB devices

On Mon, Mar 04, 2024 at 07:06:08PM -0500, Demi Marie Obenour wrote:
> On Tue, Mar 05, 2024 at 12:52:22AM +0100, Marek Marczykowski-G=C3=B3recki=
 wrote:
> > On Mon, Mar 04, 2024 at 11:46:04PM +0100, Marek Marczykowski-G=C3=B3rec=
ki wrote:
> > > On Mon, Mar 04, 2024 at 09:04:00PM +0000, Greg KH wrote:
> > > > On Mon, Mar 04, 2024 at 03:01:51PM -0500, Demi Marie Obenour wrote:
> > > > > Qubes OS users are reporting that MTP doesn't work with USB passt=
hrough.
> > > > > Fastboot (used for flashing a custom OS to an Android device) also
> > > > > doesn't work.  Kernel-mode drivers, such as Bluetooth and USB sto=
rage,
> > > > > seem to usually work as expected.  Since MTP and fastboot are both
> > > > > implemented in userspace, it appears that there is some problem w=
ith the
> > > > > interaction of usbip, our USB proxy (which is based on USBIP), and
> > > > > userspace programs that interact with USB devices directly.
> > > > >=20
> > > > > The bug report can be found at [1] and the source code for the US=
B proxy
> > > > > can be found at [2].  The script used on the sending side (the on=
e with
> > > > > the physical USB controller) is at [3] and the script used by the
> > > > > receiving side (the one the device is attached to) is at [4].  Al=
l of
> > > > > these links are for the current version as of this email being se=
nt, so
> > > > > that anyone looking at this email in the future doesn't get confu=
sed.
> > > > >=20
> > > > > Is this a bug in usbip, or is this due to usbip being used incorr=
ectly?
> > > >=20
> > > > I'm amazed that usbip works with usbfs at all, I didn't think that =
was a
> > > > thing.
> > > >=20
> > > > If you have a reproducer, or some error messages somewhere, that mi=
ght
> > > > be the easiest way forward.  In reading the bug report, it looks li=
ke
> > > > the "bridge" you all made can't handle the device disconnecting its=
elf
> > > > properly?  But that's just a guess, could be anything.
> > >=20
> > > Device disconnecting itself indeed is an issue (our proxy doesn't
> > > automatically reconnect it, at least not yet). But that's definitely =
not
> > > the only issue, things break also when disconnect is not involved.
> > >=20
> > > Terminology:
> > > 1. sys-usb - a VM where USB controller (a PCI device) lives; here
> > > usbip-host is attached to the device
> > > 2. testvm - target VM where usbip is connected; here vhci-hcd is used
> > > 3. qvm-usb - tool that connects the above two (equivalent of
> > > userspace part of standard usbip)
> > >=20
> > > Specific steps:
> > > 1. Connect android phone - at this point it's only in sys-usb
> > > 2. Switch its mode to file transfer - observe reconnect in sys-usb
> > > 3. Use qvm-usb to attach it to the testvm
> > > 4. Call jmtpfs -d /mnt in testvm
> >=20
> > Or maybe reset or something is involved here too?
> >=20
> > After using qvm-usb to attach _and detach_ the device, it stays bound to
> > usbip-host driver (that's intentional). But then, even after re-binding
> > back to the "usb" driver, jmtpfs called in sys-usb directly fails the
> > same way, including failure to reset.
> >=20
> > In fact, even without usbip involved at all, jmtpfs directly in sys-usb
> > works only once. The second attempt (without either physically reconnec=
ting
> > the phone, or changing its more to "no data transfer" and back to "file
> > transfer") fails the same way. After terminating the first instance, I
> > see just this logged:
> >=20
> >     [921332.525210] usb 2-1: reset high-speed USB device number 22 usin=
g xhci_hcd
>=20
> What happens if the device is not bound to _any_ driver in sys-usb?  For
> instance, does the problem go away if the only USB-related drivers in
> sys-usb are xhci-pci, usbip-host, and their dependencies?  If not, what
> about if sys-usb's kernel has no other USB-related drivers included at
> all?

I don't think I can get rid of the "usb" driver, and that's what binds to
the device. There is no driver bound to the first interface of the
device (2-1:1.0) at all.

> Also, if you have not done so already, could you try uninstalling gvfs
> from sys-usb's template?  gvfs might be interacting with the device.
> Using a -minimal template might help.

I highly doubt it would change anything.

> > Since both problematic cases involve Android, maybe the actual issue is
> > somewhere in Android instead of usbip? But then, fastboot could be a
> > completely different issue, likely related to reconnection (this one I
> > haven't tried to reproduce, and can't find much details in our issues
> > tracker).
> > On the other hand, USB tethering works just fine, so at least some
> > Android functions do work with usbip (but then, it's a kernel driver,
> > not usbfs).
> > Could be also an issue with just MTP implementation on either side of
> > the connection (I did tried also gvfs instead of jmtpfs with similar
> > result, but they likely use the same implementation). Since that's
> > Android, the device side is Linux too, but I don't know how MTP is
> > implemented there (I don't see MTP gadget function in upstream Linux).
>=20
> I suspect MTP is implemented in userspace somewhere in AOSP.

--=20
Best Regards,
Marek Marczykowski-G=C3=B3recki
Invisible Things Lab

--WN7gfDbbAqHw1d2/
Content-Type: application/pgp-signature; name="signature.asc"

-----BEGIN PGP SIGNATURE-----

iQEzBAEBCAAdFiEEhrpukzGPukRmQqkK24/THMrX1ywFAmXmaDwACgkQ24/THMrX
1yxsRgf+NaWUFQIbjKfO9lABM6LoyHD8VWptDVKH/sjrdvn2asmmNDnfqOmj+ZGl
rZwxXHfgOvL3atnUHRCbMCPo1PwH4+6C3xyPBkW+ISTCLE/hk1g9WUDkxx/RJ9z9
jSUIw07LThGcPp2zkLynNZRGBoc5Jz/JFX75GGIS2Fx/GnwVdmXpjJGou8cul7+e
rvXkYfslXT8JLG8SZ3XmmfWF6eQ6v109kyEE9csYC5buOe77hT0ej/ur8naSjstl
4QeWZFOxZ8WZIkiKwo9E4uGQI7ywU0Mur2Rjm46jfEc9kXVP/VvHzv2msbStQ5xU
QUsZHWfwDgSE4ZxQRlLwfPUfxIxK0g==
=HwDX
-----END PGP SIGNATURE-----

--WN7gfDbbAqHw1d2/--