Received: by 2002:a05:6a10:f347:0:0:0:0 with SMTP id d7csp1102416pxu; Sat, 12 Dec 2020 02:05:59 -0800 (PST) X-Google-Smtp-Source: ABdhPJxGoBllNX2S8+G9dnCfIARH8DuogHxv9YPrdqPymF4DcrtY0WzwGKmq0wANGCOFKHBTMHO+ X-Received: by 2002:a05:6402:352:: with SMTP id r18mr15294496edw.373.1607767559295; Sat, 12 Dec 2020 02:05:59 -0800 (PST) ARC-Seal: i=1; a=rsa-sha256; t=1607767559; cv=none; d=google.com; s=arc-20160816; b=G00ZYwE6njmz2H1an7c6QYpF1thlT2lEMtM0FQv4cMl+FF0NukJFiUmnhSG+IwXCpD qT3ALuhmJ/PZut3ehzQ0HUNXeaH4iTxDCuDHUfo2kZDwYOt8TTLT330BPSoayjBCCu8P sQms9DFfXqXzq3kys2Ry1YmlRn6/BDEGH3CNi2wnE5R6vz40jOI6osKFsJQ79Flx+1hp hrcS9iszmPej3pT0Z1u0ijAThxsX1ofKlyzjNdnN6GjnJt9L3X0FeWTo3iUJSEISmAoG p8OSBCAh5TWr8D79YoQ1ig583SrwqTYyzll1Z/DTrUQqkL+slJYKGDzz/UTalpgAcXkw ZNNw== ARC-Message-Signature: i=1; a=rsa-sha256; c=relaxed/relaxed; d=google.com; s=arc-20160816; h=list-id:precedence:message-id:date:subject:cc:to:from; bh=A046J5cibv//cuRkh812cjYs//Pi2NQhI9mfi9Gq+NE=; b=XZYaley2odh2na+8Wh9i8uoHWIuLP9MhD7+jmB5o47DxPedC5TZJ9Hgz0v/+EmZFHq fDvmZ7JeakouN+u20dB5QAr9VkeOfQnU1gAIhkGeU5wjcErzQay6RH62YY50Eb/cf4Jo Sc+rczh7diMxDgNgvrPzKkUDj5/qcwrVvBuEi8NMlE2ga8fblTxhmN36+uSvYLey0OFO rZrj4XdxpzsPhBw6IUTfhC/G5us6woixTac5pSELVPHawYYf29EohZUVnqHnhdkPBqdz UL0laEHGUilxVX3fkan/mG1R1gd8PYYLIFHdL+EpS+ANyh6SbVX4MGUqXW2/HxzXzmsQ SbLg== ARC-Authentication-Results: i=1; mx.google.com; spf=pass (google.com: domain of linux-kernel-owner@vger.kernel.org designates 23.128.96.18 as permitted sender) smtp.mailfrom=linux-kernel-owner@vger.kernel.org; dmarc=fail (p=NONE sp=NONE dis=NONE) header.from=alibaba.com Return-Path: Received: from vger.kernel.org (vger.kernel.org. [23.128.96.18]) by mx.google.com with ESMTP id h8si7218439ejj.161.2020.12.12.02.05.33; Sat, 12 Dec 2020 02:05:59 -0800 (PST) Received-SPF: pass (google.com: domain of linux-kernel-owner@vger.kernel.org designates 23.128.96.18 as permitted sender) client-ip=23.128.96.18; Authentication-Results: mx.google.com; spf=pass (google.com: domain of linux-kernel-owner@vger.kernel.org designates 23.128.96.18 as permitted sender) smtp.mailfrom=linux-kernel-owner@vger.kernel.org; dmarc=fail (p=NONE sp=NONE dis=NONE) header.from=alibaba.com Received: (majordomo@vger.kernel.org) by vger.kernel.org via listexpand id S2405606AbgLKJ04 (ORCPT + 99 others); Fri, 11 Dec 2020 04:26:56 -0500 Received: from out4436.biz.mail.alibaba.com ([47.88.44.36]:49018 "EHLO out4436.biz.mail.alibaba.com" rhost-flags-OK-OK-OK-OK) by vger.kernel.org with ESMTP id S2404450AbgLKJ0i (ORCPT ); Fri, 11 Dec 2020 04:26:38 -0500 X-Alimail-AntiSpam: AC=PASS;BC=-1|-1;BR=01201311R631e4;CH=green;DM=||false|;DS=||;FP=0|-1|-1|-1|0|-1|-1|-1;HT=e01e04423;MF=xuanzhuo@linux.alibaba.com;NM=1;PH=DS;RN=17;SR=0;TI=SMTPD_---0UIE-Rji_1607678745; Received: from localhost(mailfrom:xuanzhuo@linux.alibaba.com fp:SMTPD_---0UIE-Rji_1607678745) by smtp.aliyun-inc.com(127.0.0.1); Fri, 11 Dec 2020 17:25:45 +0800 From: Xuan Zhuo To: magnus.karlsson@intel.com Cc: =?UTF-8?q?Bj=C3=B6rn=20T=C3=B6pel?= , Jonathan Lemon , "David S. Miller" , Jakub Kicinski , Alexei Starovoitov , Daniel Borkmann , Jesper Dangaard Brouer , John Fastabend , Andrii Nakryiko , Martin KaFai Lau , Song Liu , Yonghong Song , KP Singh , netdev@vger.kernel.org (open list:XDP SOCKETS (AF_XDP)), bpf@vger.kernel.org (open list:XDP SOCKETS (AF_XDP)), linux-kernel@vger.kernel.org (open list) Subject: [PATCH bpf-next] xsk: save the undone skb Date: Fri, 11 Dec 2020 17:25:44 +0800 Message-Id: <8c251b09e29f5c36a824f73211a22e64460d4e4e.1607678556.git.xuanzhuo@linux.alibaba.com> X-Mailer: git-send-email 1.8.3.1 Precedence: bulk List-ID: X-Mailing-List: linux-kernel@vger.kernel.org We can reserve the skb. When sending fails, NETDEV_TX_BUSY or xskq_prod_reserve fails. As long as skb is successfully generated and successfully configured, we can reserve skb if we encounter exceptions later. Especially when NETDEV_TX_BUSY fails, there is no need to deal with the problem that xskq_prod_reserve has been updated. Signed-off-by: Xuan Zhuo --- include/net/xdp_sock.h | 3 +++ net/xdp/xsk.c | 36 +++++++++++++++++++++++++++--------- 2 files changed, 30 insertions(+), 9 deletions(-) diff --git a/include/net/xdp_sock.h b/include/net/xdp_sock.h index 4f4e93b..fead0c9 100644 --- a/include/net/xdp_sock.h +++ b/include/net/xdp_sock.h @@ -76,6 +76,9 @@ struct xdp_sock { struct mutex mutex; struct xsk_queue *fq_tmp; /* Only as tmp storage before bind */ struct xsk_queue *cq_tmp; /* Only as tmp storage before bind */ + + struct sk_buff *skb_undone; + bool skb_undone_reserve; }; #ifdef CONFIG_XDP_SOCKETS diff --git a/net/xdp/xsk.c b/net/xdp/xsk.c index e28c682..1051024 100644 --- a/net/xdp/xsk.c +++ b/net/xdp/xsk.c @@ -435,6 +435,19 @@ static int xsk_generic_xmit(struct sock *sk) if (xs->queue_id >= xs->dev->real_num_tx_queues) goto out; + if (xs->skb_undone) { + if (xs->skb_undone_reserve) { + if (xskq_prod_reserve(xs->pool->cq)) + goto out; + + xs->skb_undone_reserve = false; + } + + skb = xs->skb_undone; + xs->skb_undone = NULL; + goto xmit; + } + while (xskq_cons_peek_desc(xs->tx, &desc, xs->pool)) { char *buffer; u64 addr; @@ -454,12 +467,7 @@ static int xsk_generic_xmit(struct sock *sk) addr = desc.addr; buffer = xsk_buff_raw_get_data(xs->pool, addr); err = skb_store_bits(skb, 0, buffer, len); - /* This is the backpressure mechanism for the Tx path. - * Reserve space in the completion queue and only proceed - * if there is space in it. This avoids having to implement - * any buffering in the Tx path. - */ - if (unlikely(err) || xskq_prod_reserve(xs->pool->cq)) { + if (unlikely(err)) { kfree_skb(skb); goto out; } @@ -470,12 +478,22 @@ static int xsk_generic_xmit(struct sock *sk) skb_shinfo(skb)->destructor_arg = (void *)(long)desc.addr; skb->destructor = xsk_destruct_skb; + /* This is the backpressure mechanism for the Tx path. + * Reserve space in the completion queue and only proceed + * if there is space in it. This avoids having to implement + * any buffering in the Tx path. + */ + if (xskq_prod_reserve(xs->pool->cq)) { + xs->skb_undone_reserve = true; + xs->skb_undone = skb; + goto out; + } + +xmit: err = __dev_direct_xmit(skb, xs->queue_id); if (err == NETDEV_TX_BUSY) { /* Tell user-space to retry the send */ - skb->destructor = sock_wfree; - /* Free skb without triggering the perf drop trace */ - consume_skb(skb); + xs->skb_undone = skb; err = -EAGAIN; goto out; } -- 1.8.3.1