Hi,
I have a question about using vhost-net with an IFF_TUN device. I'm
uncertain about the behaviour I'm seeing which is:
i) on RX, the descriptor contains no Ethernet header, which is what I
was expecting
ii) on TX, the first 14 bytes of the transmitted _IP_ packet are lost;
if I prepend an extra 14 bytes (zeroes) before the IP packet and extend
the packet length accordingly, then things appear to work as expected.
In vhost_net_build_xdp() it appears that the userspace packet data is
copied verbatim to an XDP packet structure that assumes the presence of
an ethernet header; as such, the IP header is copied into the ethernet
header area. I think this accounts for losing the first 14 bytes of the
IP header...
If I set SO_SNDBUF to something less than INT_MAX, then the XDP path is
bypassed and transmission of IP packets works. This means that knowing
the value of SO_SNDBUF becomes important in the userspace application in
order to know whether an extra 14 bytes needs to be prepended to the IP
packet... which is awkward, at best.
For an IFF_TUN device, should vhost-net not be adding an implicit
ethernet header in _build_xdp()? Can this be done without backward
compatibility implications?
Thanks,
Jonas
On Mon, Jan 16, 2023 at 4:59 PM Jonas Bonn <[email protected]> wrote:
>
> Hi,
>
> I have a question about using vhost-net with an IFF_TUN device. I'm
> uncertain about the behaviour I'm seeing which is:
>
> i) on RX, the descriptor contains no Ethernet header, which is what I
> was expecting
> ii) on TX, the first 14 bytes of the transmitted _IP_ packet are lost;
> if I prepend an extra 14 bytes (zeroes) before the IP packet and extend
> the packet length accordingly, then things appear to work as expected.
>
> In vhost_net_build_xdp() it appears that the userspace packet data is
> copied verbatim to an XDP packet structure that assumes the presence of
> an ethernet header; as such, the IP header is copied into the ethernet
> header area. I think this accounts for losing the first 14 bytes of the
> IP header...
>
> If I set SO_SNDBUF to something less than INT_MAX, then the XDP path is
> bypassed and transmission of IP packets works. This means that knowing
> the value of SO_SNDBUF becomes important in the userspace application in
> order to know whether an extra 14 bytes needs to be prepended to the IP
> packet... which is awkward, at best.
It's a bug.
>
> For an IFF_TUN device, should vhost-net not be adding an implicit
> ethernet header in _build_xdp()?
Probably.
Actually, this makes me think that we should disable XDP for TUN?
> Can this be done without backward
> compatibility implications?
>
The path is used by vhost-net only, so I think we are fine.
Patch is more than welcomed.
Thanks
> Thanks,
> Jonas
>
Hi Jason,
Thanks for your feedback.
On 17/01/2023 05:26, Jason Wang wrote:
> On Mon, Jan 16, 2023 at 4:59 PM Jonas Bonn <[email protected]> wrote:
>>
>> For an IFF_TUN device, should vhost-net not be adding an implicit
>> ethernet header in _build_xdp()?
>
> Probably.
>
> Actually, this makes me think that we should disable XDP for TUN?
After playing around with this for a week, I agree. It appears that as
soon as the XDP paths come into play there are requirements on having
valid ethernet headers in place; the TUN interface doesn't even have a
MAC address to validate against so packets in the TCP paths get dropped.
UDP packet validation apparently only cares about the ethernet "proto"
field so these can be made to go through with less rigorous ethernet
addressing.
That said, when the XDP path is bypassed, the TUN device works fine with
vhost-net.
>
>> Can this be done without backward
>> compatibility implications?
>>
>
> The path is used by vhost-net only, so I think we are fine.
>
> Patch is more than welcomed.
I'll try to put something together.
/Jonas