Received-SPF: pass (google.com: best guess record for domain of linux-kernel-owner@vger.kernel.org designates 209.132.180.67 as permitted sender) client-ip=209.132.180.67;
Date:   Wed, 15 Aug 2018 19:49:15 -0700
From:   Alexei Starovoitov <alexei.starovoitov@gmail.com>
To:     Jason Wang <jasowang@redhat.com>
Cc:     David Ahern <dsahern@gmail.com>,
        Jesper Dangaard Brouer <jbrouer@redhat.com>,
        netdev@vger.kernel.org, linux-kernel@vger.kernel.org,
        ast@kernel.org, daniel@iogearbox.net, mst@redhat.com,
        Toshiaki Makita <makita.toshiaki@lab.ntt.co.jp>
Subject: Re: [RFC PATCH net-next V2 0/6] XDP rx handler
Message-ID: <20180816024913.57ykirt7eqrjntfy@ast-mbp.dhcp.thefacebook.com>
References: <1534130250-5302-1-git-send-email-jasowang@redhat.com>
 <20180814003253.fkgl6lyklc7fclvq@ast-mbp>
 <5de3d14f-f21a-c806-51f4-b5efd7d809b7@redhat.com>
 <20180814121734.105769fa@redhat.com>
 <03ab3b18-9b13-8169-7e68-ada307694bc1@redhat.com>
 <08bf7aec-078a-612d-833f-5b3d09a289d0@gmail.com>
 <aa9cf883-7822-70a7-5ab5-c873b69c2098@redhat.com>
 <20180815053550.5g4f5qeb7r4wtgm5@ast-mbp>
 <0809cbab-9c91-52f1-2abe-124a255d9304@redhat.com>
MIME-Version: 1.0
Content-Type: text/plain; charset=us-ascii
Content-Disposition: inline
In-Reply-To: <0809cbab-9c91-52f1-2abe-124a255d9304@redhat.com>
User-Agent: NeoMutt/20180223
Sender: linux-kernel-owner@vger.kernel.org
Precedence: bulk

On Wed, Aug 15, 2018 at 03:04:35PM +0800, Jason Wang wrote:
> 
> > > 3 Deliver XDP buff to userspace through macvtap.
> > I think I'm getting what you're trying to achieve.
> > You actually don't want any bpf programs in there at all.
> > You want macvlan builtin logic to act on raw packet frames.
> 
> The built-in logic is just used to find the destination macvlan device. It
> could be done by through another bpf program. Instead of inventing lots of
> generic infrastructure on kernel with specific userspace API, built-in logic
> has its own advantages:
> 
> - support hundreds or even thousands of macvlans

are you saying xdp bpf program cannot handle thousands macvlans?

> - using exist tools to configure network
> - immunity to topology changes

what do you mean specifically?

> 
> Besides the usage for containers, we can implement macvtap RX handler which
> allows a fast packet forwarding to userspace.

and try to reinvent af_xdp? the motivation for the patchset still escapes me.

> Actually, the idea is not limited to macvlan but for all device that is
> based on rx handler. Consider the case of bonding, this allows to set a very
> simple XDP program on slaves and keep a single main logic XDP program on the
> bond instead of duplicating it in all slaves.

I think such mixed environment of hardcoded in-kernel things like bond
mixed together with xdp programs will be difficult to manage and debug.
How admin suppose to debug it? Say something in the chain of
nic -> native xdp -> bond with your xdp rx -> veth -> xdp prog -> consumer
is dropping a packet. If all forwarding decisions are done by bpf progs
the progs will have packet tracing facility (like cilium does) to
show packet flow end-to-end. It works briliantly like traceroute within a host.
But when you have things like macvlan, bond, bridge in the middle
that can also act on packet, the admin will have a hard time.

Essentially what you're proposing is to make all kernel builtin packet
steering/forwarding facilities to understand raw xdp frames. That's a lot of code
and at the end of the chain you'd need fast xdp frame consumer otherwise
perf benefits are lost. If that consumer is xdp bpf program
why bother with xdp-fied macvlan or bond? If that consumer is tcp stack
than forwarding via xdp-fied bond is no faster than via skb-based bond.