Return-Path: Received: (majordomo@vger.kernel.org) by vger.kernel.org via listexpand id S1757204Ab1ERNTQ (ORCPT ); Wed, 18 May 2011 09:19:16 -0400 Received: from mx1.redhat.com ([209.132.183.28]:4331 "EHLO mx1.redhat.com" rhost-flags-OK-OK-OK-OK) by vger.kernel.org with ESMTP id S1756930Ab1ERNTP (ORCPT ); Wed, 18 May 2011 09:19:15 -0400 Date: Wed, 18 May 2011 16:19:01 +0300 From: "Michael S. Tsirkin" To: =?utf-8?B?TWljaGHFgiBNaXJvc8WCYXc=?= Cc: Shirley Ma , Ben Hutchings , David Miller , Eric Dumazet , Avi Kivity , Arnd Bergmann , netdev@vger.kernel.org, kvm@vger.kernel.org, linux-kernel@vger.kernel.org Subject: Re: [PATCH V5 2/6 net-next] netdevice.h: Add zero-copy flag in netdevice Message-ID: <20110518131901.GA19695@redhat.com> References: <20110516211459.GE18148@redhat.com> <1305588738.3456.65.camel@localhost.localdomain> <1305671318.10756.49.camel@localhost.localdomain> <20110518103819.GL7589@redhat.com> <20110518111734.GO7589@redhat.com> <20110518115602.GT7589@redhat.com> MIME-Version: 1.0 Content-Type: text/plain; charset=utf-8 Content-Disposition: inline Content-Transfer-Encoding: 8bit In-Reply-To: User-Agent: Mutt/1.5.21 (2010-09-15) Sender: linux-kernel-owner@vger.kernel.org List-ID: X-Mailing-List: linux-kernel@vger.kernel.org Content-Length: 3872 Lines: 85 On Wed, May 18, 2011 at 02:48:24PM +0200, Michał Mirosław wrote: > W dniu 18 maja 2011 13:56 użytkownik Michael S. Tsirkin > napisał: > > On Wed, May 18, 2011 at 01:47:33PM +0200, Michał Mirosław wrote: > >> W dniu 18 maja 2011 13:17 użytkownik Michael S. Tsirkin > >> napisał: > >> > On Wed, May 18, 2011 at 01:10:50PM +0200, Michał Mirosław wrote: > >> >> 2011/5/18 Michael S. Tsirkin : > >> >> > On Tue, May 17, 2011 at 03:28:38PM -0700, Shirley Ma wrote: > >> >> >> On Tue, 2011-05-17 at 23:48 +0200, Michał Mirosław wrote: > >> >> >> > 2011/5/17 Shirley Ma : > >> >> >> > > Hello Michael, > >> >> >> > > > >> >> >> > > Looks like to use a new flag requires more time/work. I am thinking > >> >> >> > > whether we can just use HIGHDMA flag to enable zero-copy in macvtap > >> >> >> > to > >> >> >> > > avoid the new flag for now since mavctap uses real NICs as lower > >> >> >> > device? > >> >> >> > > >> >> >> > Is there any other restriction besides requiring driver to not recycle > >> >> >> > the skb? Are there any drivers that recycle TX skbs? > >> >> > Not just recycling skbs, keeping reference to any of the pages in the > >> >> > skb. Another requirement is to invoke the callback > >> >> > in a timely fashion.  For example virtio-net doesn't limit the time until > >> >> > that happens (skbs are only freed when some other packet is > >> >> > transmitted), so we need to avoid zcopy for such (nested-virt) > >> >> > scenarious, right? > >> >> Hmm. But every hardware driver supporting SG will keep reference to > >> >> the pages until the packet is sent (or DMA'd to the device). This can > >> >> take a long time if hardware queue happens to stall for some reason. > >> > That's a fundamental property of zero copy transmit. > >> > You can't let the application/guest reuse the memory until > >> > no one looks at it anymore. > >> > >> One more question: is userspace (or whatever is sending those packets) > >> denied from modifying passed pages? I assume it is, but just want to > >> be sure. > >> > >> Best Regards, > >> Michał Mirosław > > > > Good point. > > > > It's not denied in the sense that it still can modify them if it's > > buggy (the pages might not be read-only). > > But well-behaved userspace won't modify them until the callback > > is invoked. > > > > That would be a problem if the underlying device is > > a bridge where we might try to e.g. filter these packets - > > data can get modified after the filter. We'd have to copy > > whatever the filter accesses and use the copy - it's rarely > > the data itself. > > > > That's not normally a problem for macvtap connected to a physical NIC, > > as that already bypasses any and all filtering. > > > > But that's another limitation we should note in the comment, > > and another reason to limit to specific devices. > > It looks like this feature can be used only in very strict circumstances. True. I think it's reasonable to try and start with something restricted and then add features though - past attempts to solve the problem generally right away did not bear fruit. > What about tcpdump listening on the device or lowerdev? This path > might clone the skb for any device. > > Best Regards, > Michał Mirosław Thanks for bringing this up: taps do need to be fixed as they can hang on to a page for unlimited time. Further, as a malicious guest can change the packet at any time, data that taps get wouldn't be correct. We can either linearize the problematic skbs or disable zero copy if there are any taps for the given device. -- MST -- To unsubscribe from this list: send the line "unsubscribe linux-kernel" in the body of a message to majordomo@vger.kernel.org More majordomo info at http://vger.kernel.org/majordomo-info.html Please read the FAQ at http://www.tux.org/lkml/