Return-Path: Received: (majordomo@vger.kernel.org) by vger.kernel.org via listexpand id S1754066AbbKGRTy (ORCPT ); Sat, 7 Nov 2015 12:19:54 -0500 Received: from mail-lf0-f41.google.com ([209.85.215.41]:34552 "EHLO mail-lf0-f41.google.com" rhost-flags-OK-OK-OK-OK) by vger.kernel.org with ESMTP id S1753943AbbKGRTw (ORCPT ); Sat, 7 Nov 2015 12:19:52 -0500 MIME-Version: 1.0 In-Reply-To: References: Date: Sat, 7 Nov 2015 09:19:50 -0800 Message-ID: Subject: Re: GSO with udp_tunnel_xmit_skb From: =?UTF-8?Q?Maciej_=C5=BBenczykowski?= To: "Jason A. Donenfeld" Cc: Tom Herbert , Jiri Benc , Netdev , LKML Content-Type: text/plain; charset=UTF-8 Sender: linux-kernel-owner@vger.kernel.org List-ID: X-Mailing-List: linux-kernel@vger.kernel.org Content-Length: 2128 Lines: 45 > I suppose this is about UFO. > > Specifically -- let's say I have a list of 500 skbs, which have their > data in place but don't yet have an IP or UDP header etc. I want to > send out these out using udp_tunnel_xmit_skb. Right now, if I just > send them all out, one after another, they don't seem to be getting > assembled into a super packet suitable for UFO. Instead, they're just > sent one at a time, and I get the vast majority of `perf top` CPU > usage in my ethernet card's driver and along the path to it -- the > problem that UFO is supposed to solve. > > So my question is -- how can I make UFO happen with udp_tunnel_xmit_skb? UFO will never collapse multiple (UDP) packets. It would be incorrect to do so, since UDP has to maintain packet framing boundaries, and the only way to mark that on the wire is via individual appropriately sized packets. UFO prevents the need to do IP fragmentation on overly large *singular* UDP packets. The case where UFO (should) help is if you are taking a TCP TSO segment of 10k and adding UDP headers and sending it out as an 20+8+10k UDP packet. Without UFO this would now need to be software (potentially checksummed and) ip fragmented into (8+10k)/(1500-20) packets (assuming 1500 mtu), with UFO hw offload the nic deals with that (it does the checksumming and it does the ip fragmentation). Although note: in the case of UDP+TCP TSO this has reliability issues, since a loss of a single frame will now lose the entire fragmented IP UDP datagram and thus lose the entire TCP TSO segment, meaning that you probably do not want to use this unless your network is lossless (ie. loopback, veth and other virtual networks come to mind). I guess UDP encap of a larger than mtu UDP is probably a valid use case for UFO, since we'd have ip fragmented anyway, and it's cheaper to ip fragment on the outer IP header than on the inner. -- To unsubscribe from this list: send the line "unsubscribe linux-kernel" in the body of a message to majordomo@vger.kernel.org More majordomo info at http://vger.kernel.org/majordomo-info.html Please read the FAQ at http://www.tux.org/lkml/