This series is intended as a playground to start experimenting/developing
with XDP/eBPF over WiFi and collect ideas/concerns about it.
Introduce XDP support to mt76x2e/mt76x0e drivers. Currently supported
actions are:
- XDP_PASS
- XDP_ABORTED
- XDP_DROP
Introduce ndo_bpf mac80211 callback in order to to load a bpf
program into low level driver XDP rx hook.
This series has been tested through a simple bpf program (available here:
https://github.com/LorenzoBianconi/bpf-workspace/tree/master/mt76_xdp_stats)
used to count frame types received by the device.
Possible eBPF use cases could be:
- implement new statistics through bpf maps
- implement fast packet filtering (e.g in monitor mode)
- ...
Lorenzo Bianconi (5):
mac80211: introduce ieee80211_xdp handler
mac80211: introduce ieee80211_vif_to_netdev routine
mt76: split mt76_dma_rx_reset in init_rx_reset and complete_rx_reset
mt76: make mt76x02_vif_init return int
mt76: add XDP support
drivers/net/wireless/mediatek/mt76/dma.c | 58 +++++++++++-
drivers/net/wireless/mediatek/mt76/mac80211.c | 18 ++++
drivers/net/wireless/mediatek/mt76/mt76.h | 12 ++-
.../net/wireless/mediatek/mt76/mt76x0/pci.c | 3 +
drivers/net/wireless/mediatek/mt76/mt76x02.h | 7 +-
.../net/wireless/mediatek/mt76/mt76x02_mac.h | 1 +
.../net/wireless/mediatek/mt76/mt76x02_mmio.c | 90 +++++++++++++++++++
.../net/wireless/mediatek/mt76/mt76x02_util.c | 17 +++-
.../wireless/mediatek/mt76/mt76x2/pci_init.c | 2 +
.../wireless/mediatek/mt76/mt76x2/pci_main.c | 1 +
.../wireless/mediatek/mt76/mt76x2/usb_main.c | 3 +-
include/net/mac80211.h | 4 +
net/mac80211/driver-ops.h | 11 +++
net/mac80211/iface.c | 14 +++
net/mac80211/util.c | 12 +++
15 files changed, 241 insertions(+), 12 deletions(-)
--
2.19.1
Initialize net_device_ops ndo_bpf callback with ieee80211_xdp routine
in order to load a bpf program into low level driver XDP rx hook
Signed-off-by: Lorenzo Bianconi <[email protected]>
---
include/net/mac80211.h | 2 ++
net/mac80211/driver-ops.h | 11 +++++++++++
net/mac80211/iface.c | 14 ++++++++++++++
3 files changed, 27 insertions(+)
diff --git a/include/net/mac80211.h b/include/net/mac80211.h
index 71985e95d2d9..a91f1733ce43 100644
--- a/include/net/mac80211.h
+++ b/include/net/mac80211.h
@@ -3623,6 +3623,7 @@ enum ieee80211_reconfig_type {
* skb is always a real frame, head may or may not be an A-MSDU.
* @get_ftm_responder_stats: Retrieve FTM responder statistics, if available.
* Statistics should be cumulative, currently no way to reset is provided.
+ * @xdp: main XDP handler
*/
struct ieee80211_ops {
void (*tx)(struct ieee80211_hw *hw,
@@ -3911,6 +3912,7 @@ struct ieee80211_ops {
int (*get_ftm_responder_stats)(struct ieee80211_hw *hw,
struct ieee80211_vif *vif,
struct cfg80211_ftm_responder_stats *ftm_stats);
+ int (*xdp)(struct ieee80211_hw *hw, struct netdev_bpf *xdp);
};
/**
diff --git a/net/mac80211/driver-ops.h b/net/mac80211/driver-ops.h
index 0b1747a2313d..48cad0ae0593 100644
--- a/net/mac80211/driver-ops.h
+++ b/net/mac80211/driver-ops.h
@@ -1278,4 +1278,15 @@ static inline void drv_del_nan_func(struct ieee80211_local *local,
trace_drv_return_void(local);
}
+static inline int drv_xdp(struct ieee80211_local *local,
+ struct netdev_bpf *xdp)
+{
+ might_sleep();
+
+ if (!local->ops->xdp)
+ return -EOPNOTSUPP;
+
+ return local->ops->xdp(&local->hw, xdp);
+}
+
#endif /* __MAC80211_DRIVER_OPS */
diff --git a/net/mac80211/iface.c b/net/mac80211/iface.c
index 5836ddeac9e3..e21971f7c652 100644
--- a/net/mac80211/iface.c
+++ b/net/mac80211/iface.c
@@ -1163,6 +1163,19 @@ ieee80211_get_stats64(struct net_device *dev, struct rtnl_link_stats64 *stats)
}
}
+static int ieee80211_xdp(struct net_device *dev, struct netdev_bpf *xdp)
+{
+ struct ieee80211_sub_if_data *sdata = IEEE80211_DEV_TO_SUB_IF(dev);
+
+ switch (xdp->command) {
+ case XDP_QUERY_PROG:
+ case XDP_SETUP_PROG:
+ return drv_xdp(sdata->local, xdp);
+ default:
+ return -EINVAL;
+ }
+}
+
static const struct net_device_ops ieee80211_dataif_ops = {
.ndo_open = ieee80211_open,
.ndo_stop = ieee80211_stop,
@@ -1172,6 +1185,7 @@ static const struct net_device_ops ieee80211_dataif_ops = {
.ndo_set_mac_address = ieee80211_change_mac,
.ndo_select_queue = ieee80211_netdev_select_queue,
.ndo_get_stats64 = ieee80211_get_stats64,
+ .ndo_bpf = ieee80211_xdp,
};
static u16 ieee80211_monitor_select_queue(struct net_device *dev,
--
2.19.1
Add ieee80211_vif_to_netdev utility routine to get the netdevice
pointer related to ieee80211_vif one even if the sdata has not been
inserted in the driver or running yet. ieee80211_vif_to_netdev
will be used by mt76 to properly configure xdp_rxq_info data
structure adding vif to the driver
Signed-off-by: Lorenzo Bianconi <[email protected]>
---
include/net/mac80211.h | 2 ++
net/mac80211/util.c | 12 ++++++++++++
2 files changed, 14 insertions(+)
diff --git a/include/net/mac80211.h b/include/net/mac80211.h
index a91f1733ce43..f2784b36f2e2 100644
--- a/include/net/mac80211.h
+++ b/include/net/mac80211.h
@@ -1603,6 +1603,8 @@ struct ieee80211_vif *wdev_to_ieee80211_vif(struct wireless_dev *wdev);
*/
struct wireless_dev *ieee80211_vif_to_wdev(struct ieee80211_vif *vif);
+struct net_device *ieee80211_vif_to_netdev(struct ieee80211_vif *vif);
+
/**
* enum ieee80211_key_flags - key flags
*
diff --git a/net/mac80211/util.c b/net/mac80211/util.c
index bec424316ea4..a98293d71818 100644
--- a/net/mac80211/util.c
+++ b/net/mac80211/util.c
@@ -827,6 +827,18 @@ struct wireless_dev *ieee80211_vif_to_wdev(struct ieee80211_vif *vif)
}
EXPORT_SYMBOL_GPL(ieee80211_vif_to_wdev);
+struct net_device *ieee80211_vif_to_netdev(struct ieee80211_vif *vif)
+{
+ struct ieee80211_sub_if_data *sdata;
+
+ if (!vif)
+ return NULL;
+
+ sdata = vif_to_sdata(vif);
+ return sdata->dev;
+}
+EXPORT_SYMBOL_GPL(ieee80211_vif_to_netdev);
+
/*
* Nothing should have been stuffed into the workqueue during
* the suspend->resume cycle. Since we can't check each caller
--
2.19.1
This is a preliminary patch to add XDP support to mt76 driver.
In order to reset the rx dma buffers with a proper buffer_size
(XDP does not support fragmented frames and requires one packet
per memory page) split mt76_dma_rx_reset routine in
mt76_dma_init_rx_reset where we free allocated memory and
mt76_dma_complete_rx_reset used to allocate dma memory with a proper
size according to the working mode (standard or xdp)
Signed-off-by: Lorenzo Bianconi <[email protected]>
---
drivers/net/wireless/mediatek/mt76/dma.c | 12 ++++++++++--
drivers/net/wireless/mediatek/mt76/mt76.h | 6 ++++--
2 files changed, 14 insertions(+), 4 deletions(-)
diff --git a/drivers/net/wireless/mediatek/mt76/dma.c b/drivers/net/wireless/mediatek/mt76/dma.c
index e2ba26378575..efa0eab7cf01 100644
--- a/drivers/net/wireless/mediatek/mt76/dma.c
+++ b/drivers/net/wireless/mediatek/mt76/dma.c
@@ -382,7 +382,7 @@ mt76_dma_rx_cleanup(struct mt76_dev *dev, struct mt76_queue *q)
}
static void
-mt76_dma_rx_reset(struct mt76_dev *dev, enum mt76_rxq_id qid)
+mt76_dma_init_rx_reset(struct mt76_dev *dev, enum mt76_rxq_id qid)
{
struct mt76_queue *q = &dev->q_rx[qid];
int i;
@@ -392,6 +392,13 @@ mt76_dma_rx_reset(struct mt76_dev *dev, enum mt76_rxq_id qid)
mt76_dma_rx_cleanup(dev, q);
mt76_dma_sync_idx(dev, q);
+}
+
+static void
+mt76_dma_complete_rx_reset(struct mt76_dev *dev, enum mt76_rxq_id qid)
+{
+ struct mt76_queue *q = &dev->q_rx[qid];
+
mt76_dma_rx_fill(dev, q, false);
}
@@ -518,7 +525,8 @@ static const struct mt76_queue_ops mt76_dma_ops = {
.add_buf = mt76_dma_add_buf,
.tx_queue_skb = mt76_dma_tx_queue_skb,
.tx_cleanup = mt76_dma_tx_cleanup,
- .rx_reset = mt76_dma_rx_reset,
+ .init_rx_reset = mt76_dma_init_rx_reset,
+ .complete_rx_reset = mt76_dma_complete_rx_reset,
.kick = mt76_dma_kick_queue,
};
diff --git a/drivers/net/wireless/mediatek/mt76/mt76.h b/drivers/net/wireless/mediatek/mt76/mt76.h
index 5cd508a68609..70924792d870 100644
--- a/drivers/net/wireless/mediatek/mt76/mt76.h
+++ b/drivers/net/wireless/mediatek/mt76/mt76.h
@@ -159,7 +159,8 @@ struct mt76_queue_ops {
void *(*dequeue)(struct mt76_dev *dev, struct mt76_queue *q, bool flush,
int *len, u32 *info, bool *more);
- void (*rx_reset)(struct mt76_dev *dev, enum mt76_rxq_id qid);
+ void (*init_rx_reset)(struct mt76_dev *dev, enum mt76_rxq_id qid);
+ void (*complete_rx_reset)(struct mt76_dev *dev, enum mt76_rxq_id qid);
void (*tx_cleanup)(struct mt76_dev *dev, enum mt76_txq_id qid,
bool flush);
@@ -552,9 +553,10 @@ static inline u16 mt76_rev(struct mt76_dev *dev)
#define mt76_init_queues(dev) (dev)->mt76.queue_ops->init(&((dev)->mt76))
#define mt76_queue_alloc(dev, ...) (dev)->mt76.queue_ops->alloc(&((dev)->mt76), __VA_ARGS__)
#define mt76_queue_add_buf(dev, ...) (dev)->mt76.queue_ops->add_buf(&((dev)->mt76), __VA_ARGS__)
-#define mt76_queue_rx_reset(dev, ...) (dev)->mt76.queue_ops->rx_reset(&((dev)->mt76), __VA_ARGS__)
#define mt76_queue_tx_cleanup(dev, ...) (dev)->mt76.queue_ops->tx_cleanup(&((dev)->mt76), __VA_ARGS__)
#define mt76_queue_kick(dev, ...) (dev)->mt76.queue_ops->kick(&((dev)->mt76), __VA_ARGS__)
+#define mt76_queue_init_rx_reset(dev, ...) (dev)->mt76.queue_ops->init_rx_reset(&((dev)->mt76), __VA_ARGS__)
+#define mt76_queue_complete_rx_reset(dev, ...) (dev)->mt76.queue_ops->complete_rx_reset(&((dev)->mt76), __VA_ARGS__)
static inline struct mt76_channel_state *
mt76_channel_state(struct mt76_dev *dev, struct ieee80211_channel *c)
--
2.19.1
This is a preliminary patch to add xdp support to mt76 driver
since mt76x02_vif_init will be used to xdp_rxq_info data structure
and can return an error code
Signed-off-by: Lorenzo Bianconi <[email protected]>
---
drivers/net/wireless/mediatek/mt76/mt76x02.h | 4 ++--
drivers/net/wireless/mediatek/mt76/mt76x02_util.c | 9 +++++----
drivers/net/wireless/mediatek/mt76/mt76x2/usb_main.c | 3 +--
3 files changed, 8 insertions(+), 8 deletions(-)
diff --git a/drivers/net/wireless/mediatek/mt76/mt76x02.h b/drivers/net/wireless/mediatek/mt76/mt76x02.h
index b1d83dd34204..3922854ffa06 100644
--- a/drivers/net/wireless/mediatek/mt76/mt76x02.h
+++ b/drivers/net/wireless/mediatek/mt76/mt76x02.h
@@ -116,8 +116,8 @@ void mt76x02_sta_remove(struct mt76_dev *mdev, struct ieee80211_vif *vif,
struct ieee80211_sta *sta);
void mt76x02_config_mac_addr_list(struct mt76x02_dev *dev);
-void mt76x02_vif_init(struct mt76x02_dev *dev, struct ieee80211_vif *vif,
- unsigned int idx);
+int mt76x02_vif_init(struct mt76x02_dev *dev, struct ieee80211_vif *vif,
+ unsigned int idx);
int mt76x02_add_interface(struct ieee80211_hw *hw,
struct ieee80211_vif *vif);
void mt76x02_remove_interface(struct ieee80211_hw *hw,
diff --git a/drivers/net/wireless/mediatek/mt76/mt76x02_util.c b/drivers/net/wireless/mediatek/mt76/mt76x02_util.c
index 3a70e5bf7d42..87195076cf62 100644
--- a/drivers/net/wireless/mediatek/mt76/mt76x02_util.c
+++ b/drivers/net/wireless/mediatek/mt76/mt76x02_util.c
@@ -207,8 +207,8 @@ void mt76x02_sta_remove(struct mt76_dev *mdev, struct ieee80211_vif *vif,
}
EXPORT_SYMBOL_GPL(mt76x02_sta_remove);
-void mt76x02_vif_init(struct mt76x02_dev *dev, struct ieee80211_vif *vif,
- unsigned int idx)
+int mt76x02_vif_init(struct mt76x02_dev *dev, struct ieee80211_vif *vif,
+ unsigned int idx)
{
struct mt76x02_vif *mvif = (struct mt76x02_vif *)vif->drv_priv;
struct mt76_txq *mtxq;
@@ -220,6 +220,8 @@ void mt76x02_vif_init(struct mt76x02_dev *dev, struct ieee80211_vif *vif,
mtxq->wcid = &mvif->group_wcid;
mt76_txq_init(&dev->mt76, vif->txq);
+
+ return 0;
}
EXPORT_SYMBOL_GPL(mt76x02_vif_init);
@@ -248,8 +250,7 @@ mt76x02_add_interface(struct ieee80211_hw *hw, struct ieee80211_vif *vif)
if (vif->type == NL80211_IFTYPE_STATION)
idx += 8;
- mt76x02_vif_init(dev, vif, idx);
- return 0;
+ return mt76x02_vif_init(dev, vif, idx);
}
EXPORT_SYMBOL_GPL(mt76x02_add_interface);
diff --git a/drivers/net/wireless/mediatek/mt76/mt76x2/usb_main.c b/drivers/net/wireless/mediatek/mt76/mt76x2/usb_main.c
index 2b48cc51a30d..61768c76e5d0 100644
--- a/drivers/net/wireless/mediatek/mt76/mt76x2/usb_main.c
+++ b/drivers/net/wireless/mediatek/mt76/mt76x2/usb_main.c
@@ -55,8 +55,7 @@ static int mt76x2u_add_interface(struct ieee80211_hw *hw,
if (!ether_addr_equal(dev->mt76.macaddr, vif->addr))
mt76x02_mac_setaddr(dev, vif->addr);
- mt76x02_vif_init(dev, vif, idx);
- return 0;
+ return mt76x02_vif_init(dev, vif, idx);
}
static int
--
2.19.1
Introduce XDP in mt76 driver. Add mt76_xdp and mt76x02_xdp_setup
routines in order to initialize xdp rx driver hook with the bpf
program received from mac80211.
Introduce the mt76_dma_rx_xdp routine in order to run the attached
bpf program and parse the action result. Currently supported actions
are:
- XDP_PASS
- XDP_ABORTED
- XDP_DROP
Signed-off-by: Lorenzo Bianconi <[email protected]>
---
drivers/net/wireless/mediatek/mt76/dma.c | 46 ++++++++++
drivers/net/wireless/mediatek/mt76/mac80211.c | 18 ++++
drivers/net/wireless/mediatek/mt76/mt76.h | 6 ++
.../net/wireless/mediatek/mt76/mt76x0/pci.c | 3 +
drivers/net/wireless/mediatek/mt76/mt76x02.h | 3 +
.../net/wireless/mediatek/mt76/mt76x02_mac.h | 1 +
.../net/wireless/mediatek/mt76/mt76x02_mmio.c | 90 +++++++++++++++++++
.../net/wireless/mediatek/mt76/mt76x02_util.c | 10 ++-
.../wireless/mediatek/mt76/mt76x2/pci_init.c | 2 +
.../wireless/mediatek/mt76/mt76x2/pci_main.c | 1 +
10 files changed, 179 insertions(+), 1 deletion(-)
diff --git a/drivers/net/wireless/mediatek/mt76/dma.c b/drivers/net/wireless/mediatek/mt76/dma.c
index efa0eab7cf01..b735849799e3 100644
--- a/drivers/net/wireless/mediatek/mt76/dma.c
+++ b/drivers/net/wireless/mediatek/mt76/dma.c
@@ -15,6 +15,7 @@
*/
#include <linux/dma-mapping.h>
+#include <linux/bpf_trace.h>
#include "mt76.h"
#include "dma.h"
@@ -421,6 +422,44 @@ mt76_add_fragment(struct mt76_dev *dev, struct mt76_queue *q, void *data,
dev->drv->rx_skb(dev, q - dev->q_rx, skb);
}
+static bool
+mt76_dma_rx_xdp(struct mt76_dev *dev, unsigned char *data,
+ int *len)
+{
+ struct page *page = virt_to_head_page(data);
+ struct xdp_buff xdp;
+ u32 action;
+
+ if (!dev->drv->xdp_rxq_info_lookup ||
+ dev->drv->xdp_rxq_info_lookup(dev, data, &xdp) < 0)
+ return false;
+
+ xdp.data_hard_start = page_address(page);
+ xdp.data = (void *)data;
+ xdp_set_data_meta_invalid(&xdp);
+ xdp.data_end = xdp.data + *len;
+
+ action = bpf_prog_run_xdp(dev->xdp_prog, &xdp);
+ /* bpf program chan modify the original frame */
+ *len = xdp.data_end - xdp.data;
+ data = xdp.data;
+
+ switch (action) {
+ case XDP_PASS:
+ return false;
+ default:
+ bpf_warn_invalid_xdp_action(action);
+ /* fall through */
+ case XDP_ABORTED:
+ trace_xdp_exception(xdp.rxq->dev, dev->xdp_prog, action);
+ /* fall through */
+ case XDP_DROP:
+ /* XXX: we can reuse the buffer here */
+ skb_free_frag(data);
+ return true;
+ }
+}
+
static int
mt76_dma_rx_process(struct mt76_dev *dev, struct mt76_queue *q, int budget)
{
@@ -437,6 +476,13 @@ mt76_dma_rx_process(struct mt76_dev *dev, struct mt76_queue *q, int budget)
if (!data)
break;
+ /* xdp does not support fragmentes frames */
+ if (dev->xdp_prog && !more &&
+ mt76_dma_rx_xdp(dev, data, &len)) {
+ done++;
+ continue;
+ }
+
if (q->rx_head) {
mt76_add_fragment(dev, q, data, len, more);
continue;
diff --git a/drivers/net/wireless/mediatek/mt76/mac80211.c b/drivers/net/wireless/mediatek/mt76/mac80211.c
index b33074cf7e00..387c6bba80ca 100644
--- a/drivers/net/wireless/mediatek/mt76/mac80211.c
+++ b/drivers/net/wireless/mediatek/mt76/mac80211.c
@@ -707,3 +707,21 @@ int mt76_sta_state(struct ieee80211_hw *hw, struct ieee80211_vif *vif,
return 0;
}
EXPORT_SYMBOL_GPL(mt76_sta_state);
+
+int mt76_xdp(struct ieee80211_hw *hw, struct netdev_bpf *xdp)
+{
+ struct mt76_dev *dev = hw->priv;
+
+ switch (xdp->command) {
+ case XDP_SETUP_PROG:
+ if (!dev->drv->xdp_setup)
+ return -ENOTSUPP;
+ return dev->drv->xdp_setup(dev, xdp->prog);
+ case XDP_QUERY_PROG:
+ xdp->prog_id = dev->xdp_prog ? dev->xdp_prog->aux->id : 0;
+ return 0;
+ default:
+ return -EINVAL;
+ }
+}
+EXPORT_SYMBOL_GPL(mt76_xdp);
diff --git a/drivers/net/wireless/mediatek/mt76/mt76.h b/drivers/net/wireless/mediatek/mt76/mt76.h
index 70924792d870..b46e162dbb80 100644
--- a/drivers/net/wireless/mediatek/mt76/mt76.h
+++ b/drivers/net/wireless/mediatek/mt76/mt76.h
@@ -295,6 +295,9 @@ struct mt76_driver_ops {
void (*sta_remove)(struct mt76_dev *dev, struct ieee80211_vif *vif,
struct ieee80211_sta *sta);
+ int (*xdp_setup)(struct mt76_dev *dev, struct bpf_prog *prog);
+ int (*xdp_rxq_info_lookup)(struct mt76_dev *dev, unsigned char *data,
+ struct xdp_buff *xdp);
};
struct mt76_channel_state {
@@ -461,6 +464,8 @@ struct mt76_dev {
struct mt76_mmio mmio;
struct mt76_usb usb;
};
+
+ struct bpf_prog *xdp_prog;
};
enum mt76_phy_type {
@@ -737,5 +742,6 @@ void mt76u_queues_deinit(struct mt76_dev *dev);
void mt76u_mcu_complete_urb(struct urb *urb);
int mt76u_mcu_init_rx(struct mt76_dev *dev);
void mt76u_mcu_deinit(struct mt76_dev *dev);
+int mt76_xdp(struct ieee80211_hw *hw, struct netdev_bpf *xdp);
#endif
diff --git a/drivers/net/wireless/mediatek/mt76/mt76x0/pci.c b/drivers/net/wireless/mediatek/mt76/mt76x0/pci.c
index d895b6f3dc44..38272593f839 100644
--- a/drivers/net/wireless/mediatek/mt76/mt76x0/pci.c
+++ b/drivers/net/wireless/mediatek/mt76/mt76x0/pci.c
@@ -105,6 +105,7 @@ static const struct ieee80211_ops mt76x0e_ops = {
.release_buffered_frames = mt76_release_buffered_frames,
.set_coverage_class = mt76x02_set_coverage_class,
.set_rts_threshold = mt76x02_set_rts_threshold,
+ .xdp = mt76_xdp,
};
static int mt76x0e_register_device(struct mt76x02_dev *dev)
@@ -163,6 +164,8 @@ mt76x0e_probe(struct pci_dev *pdev, const struct pci_device_id *id)
.sta_ps = mt76x02_sta_ps,
.sta_add = mt76x02_sta_add,
.sta_remove = mt76x02_sta_remove,
+ .xdp_setup = mt76x02_xdp_setup,
+ .xdp_rxq_info_lookup = mt76x02_xdp_rxq_info_lookup,
};
struct mt76x02_dev *dev;
int ret;
diff --git a/drivers/net/wireless/mediatek/mt76/mt76x02.h b/drivers/net/wireless/mediatek/mt76/mt76x02.h
index 3922854ffa06..c32185f226c3 100644
--- a/drivers/net/wireless/mediatek/mt76/mt76x02.h
+++ b/drivers/net/wireless/mediatek/mt76/mt76x02.h
@@ -170,6 +170,9 @@ extern const u16 mt76x02_beacon_offsets[16];
void mt76x02_init_beacon_config(struct mt76x02_dev *dev);
void mt76x02_set_irq_mask(struct mt76x02_dev *dev, u32 clear, u32 set);
void mt76x02_mac_start(struct mt76x02_dev *dev);
+int mt76x02_xdp_setup(struct mt76_dev *mdev, struct bpf_prog *prog);
+int mt76x02_xdp_rxq_info_lookup(struct mt76_dev *mdev, unsigned char *data,
+ struct xdp_buff *xdp);
void mt76x02_init_debugfs(struct mt76x02_dev *dev);
diff --git a/drivers/net/wireless/mediatek/mt76/mt76x02_mac.h b/drivers/net/wireless/mediatek/mt76/mt76x02_mac.h
index 4e597004c445..176625d23fb6 100644
--- a/drivers/net/wireless/mediatek/mt76/mt76x02_mac.h
+++ b/drivers/net/wireless/mediatek/mt76/mt76x02_mac.h
@@ -38,6 +38,7 @@ struct mt76x02_tx_status {
struct mt76x02_vif {
struct mt76_wcid group_wcid; /* must be first */
+ struct xdp_rxq_info xdp_rxq;
u8 idx;
};
diff --git a/drivers/net/wireless/mediatek/mt76/mt76x02_mmio.c b/drivers/net/wireless/mediatek/mt76/mt76x02_mmio.c
index 66315410aebe..76fbf136d415 100644
--- a/drivers/net/wireless/mediatek/mt76/mt76x02_mmio.c
+++ b/drivers/net/wireless/mediatek/mt76/mt76x02_mmio.c
@@ -27,6 +27,12 @@ struct beacon_bc_data {
struct sk_buff *tail[8];
};
+struct xdp_iter_data {
+ struct ieee80211_hdr *hdr;
+ struct mt76x02_dev *dev;
+ struct xdp_rxq_info **rxq_info;
+};
+
static void
mt76x02_update_beacon_iter(void *priv, u8 *mac, struct ieee80211_vif *vif)
{
@@ -384,3 +390,87 @@ void mt76x02_mac_start(struct mt76x02_dev *dev)
MT_INT_TX_STAT);
}
EXPORT_SYMBOL_GPL(mt76x02_mac_start);
+
+int mt76x02_xdp_setup(struct mt76_dev *mdev, struct bpf_prog *prog)
+{
+ struct bpf_prog *old_prog;
+ struct mt76x02_dev *dev;
+ bool reset;
+
+ if (prog) {
+ prog = bpf_prog_add(prog, 1);
+ if (IS_ERR(prog))
+ return -EINVAL;
+ }
+
+ mutex_lock(&mdev->mutex);
+
+ reset = (!!mdev->xdp_prog ^ !!prog);
+ dev = container_of(mdev, struct mt76x02_dev, mt76);
+
+ if (reset) {
+ mt76_clear(dev, MT_MAC_SYS_CTRL, MT_MAC_SYS_CTRL_ENABLE_RX);
+ napi_disable(&mdev->napi[MT_RXQ_MAIN]);
+ mt76_queue_init_rx_reset(dev, MT_RXQ_MAIN);
+ }
+
+ /* attach BPF program */
+ old_prog = xchg(&mdev->xdp_prog, prog);
+ if (old_prog)
+ bpf_prog_put(old_prog);
+
+ if (reset) {
+ struct mt76_queue *q = &mdev->q_rx[MT_RXQ_MAIN];
+
+ if (mdev->xdp_prog)
+ q->buf_size = PAGE_SIZE + XDP_PACKET_HEADROOM +
+ SKB_DATA_ALIGN(sizeof(struct skb_shared_info));
+ else
+ q->buf_size = MT_RX_BUF_SIZE;
+
+ mt76_queue_complete_rx_reset(dev, MT_RXQ_MAIN);
+ napi_enable(&mdev->napi[MT_RXQ_MAIN]);
+ mt76_set(dev, MT_MAC_SYS_CTRL, MT_MAC_SYS_CTRL_ENABLE_RX);
+ }
+
+ mutex_unlock(&mdev->mutex);
+
+ return 0;
+}
+EXPORT_SYMBOL_GPL(mt76x02_xdp_setup);
+
+static void
+mt76x02_xdp_iter(void *priv, u8 *mac, struct ieee80211_vif *vif)
+{
+ struct xdp_iter_data *data = (struct xdp_iter_data *)priv;
+
+ if (ether_addr_equal(mac, data->hdr->addr1) ||
+ (!data->rxq_info && is_multicast_ether_addr(data->hdr->addr1))) {
+ struct mt76x02_vif *mvif;
+
+ mvif = (struct mt76x02_vif *)vif->drv_priv;
+ *data->rxq_info = &mvif->xdp_rxq;
+ }
+}
+
+int mt76x02_xdp_rxq_info_lookup(struct mt76_dev *mdev, unsigned char *data,
+ struct xdp_buff *xdp)
+{
+ struct xdp_iter_data xdp_data = {};
+ struct ieee80211_hdr *hdr;
+ struct mt76x02_dev *dev;
+
+ hdr = (struct ieee80211_hdr *)(data + sizeof(struct mt76x02_rxwi));
+ dev = container_of(mdev, struct mt76x02_dev, mt76);
+
+ xdp_data.rxq_info = &xdp->rxq;
+ xdp_data.dev = dev;
+ xdp_data.hdr = hdr;
+
+ ieee80211_iterate_active_interfaces_atomic(mt76_hw(dev),
+ IEEE80211_IFACE_ITER_RESUME_ALL,
+ mt76x02_xdp_iter, &xdp_data);
+
+ return 0;
+}
+EXPORT_SYMBOL_GPL(mt76x02_xdp_rxq_info_lookup);
diff --git a/drivers/net/wireless/mediatek/mt76/mt76x02_util.c b/drivers/net/wireless/mediatek/mt76/mt76x02_util.c
index 87195076cf62..0270f7ef3002 100644
--- a/drivers/net/wireless/mediatek/mt76/mt76x02_util.c
+++ b/drivers/net/wireless/mediatek/mt76/mt76x02_util.c
@@ -211,6 +211,7 @@ int mt76x02_vif_init(struct mt76x02_dev *dev, struct ieee80211_vif *vif,
unsigned int idx)
{
struct mt76x02_vif *mvif = (struct mt76x02_vif *)vif->drv_priv;
+ struct net_device *ndev = ieee80211_vif_to_netdev(vif);
struct mt76_txq *mtxq;
mvif->idx = idx;
@@ -221,7 +222,7 @@ int mt76x02_vif_init(struct mt76x02_dev *dev, struct ieee80211_vif *vif,
mt76_txq_init(&dev->mt76, vif->txq);
- return 0;
+ return ndev ? xdp_rxq_info_reg(&mvif->xdp_rxq, ndev, 0) : 0;
}
EXPORT_SYMBOL_GPL(mt76x02_vif_init);
@@ -257,9 +258,16 @@ EXPORT_SYMBOL_GPL(mt76x02_add_interface);
void mt76x02_remove_interface(struct ieee80211_hw *hw,
struct ieee80211_vif *vif)
{
+ struct net_device *ndev = ieee80211_vif_to_netdev(vif);
struct mt76x02_dev *dev = hw->priv;
mt76_txq_remove(&dev->mt76, vif->txq);
+ if (ndev) {
+ struct mt76x02_vif *mvif;
+
+ mvif = (struct mt76x02_vif *)vif->drv_priv;
+ xdp_rxq_info_unreg(&mvif->xdp_rxq);
+ }
}
EXPORT_SYMBOL_GPL(mt76x02_remove_interface);
diff --git a/drivers/net/wireless/mediatek/mt76/mt76x2/pci_init.c b/drivers/net/wireless/mediatek/mt76/mt76x2/pci_init.c
index 6eaab156387a..c6a60d6809f0 100644
--- a/drivers/net/wireless/mediatek/mt76/mt76x2/pci_init.c
+++ b/drivers/net/wireless/mediatek/mt76/mt76x2/pci_init.c
@@ -325,6 +325,8 @@ struct mt76x02_dev *mt76x2_alloc_device(struct device *pdev)
.sta_ps = mt76x02_sta_ps,
.sta_add = mt76x02_sta_add,
.sta_remove = mt76x02_sta_remove,
+ .xdp_setup = mt76x02_xdp_setup,
+ .xdp_rxq_info_lookup = mt76x02_xdp_rxq_info_lookup,
};
struct mt76x02_dev *dev;
struct mt76_dev *mdev;
diff --git a/drivers/net/wireless/mediatek/mt76/mt76x2/pci_main.c b/drivers/net/wireless/mediatek/mt76/mt76x2/pci_main.c
index b54a32397486..6898f134e3fe 100644
--- a/drivers/net/wireless/mediatek/mt76/mt76x2/pci_main.c
+++ b/drivers/net/wireless/mediatek/mt76/mt76x2/pci_main.c
@@ -199,5 +199,6 @@ const struct ieee80211_ops mt76x2_ops = {
.set_antenna = mt76x2_set_antenna,
.get_antenna = mt76x2_get_antenna,
.set_rts_threshold = mt76x02_set_rts_threshold,
+ .xdp = mt76_xdp,
};
--
2.19.1
Lorenzo Bianconi <[email protected]> writes:
> This series is intended as a playground to start experimenting/developing
> with XDP/eBPF over WiFi and collect ideas/concerns about it.
> Introduce XDP support to mt76x2e/mt76x0e drivers. Currently supported
> actions are:
> - XDP_PASS
> - XDP_ABORTED
> - XDP_DROP
> Introduce ndo_bpf mac80211 callback in order to to load a bpf
> program into low level driver XDP rx hook.
> This series has been tested through a simple bpf program (available here:
> https://github.com/LorenzoBianconi/bpf-workspace/tree/master/mt76_xdp_stats)
> used to count frame types received by the device.
> Possible eBPF use cases could be:
> - implement new statistics through bpf maps
> - implement fast packet filtering (e.g in monitor mode)
> - ...
This is most likely a stupid question, but why do this in the driver and
not in mac80211 so that all drivers could benefit from it? I guess there
are reasons for that, I just can't figure that out.
--
Kalle Valo
> Lorenzo Bianconi <[email protected]> writes:
>
> > This series is intended as a playground to start experimenting/developing
> > with XDP/eBPF over WiFi and collect ideas/concerns about it.
> > Introduce XDP support to mt76x2e/mt76x0e drivers. Currently supported
> > actions are:
> > - XDP_PASS
> > - XDP_ABORTED
> > - XDP_DROP
> > Introduce ndo_bpf mac80211 callback in order to to load a bpf
> > program into low level driver XDP rx hook.
> > This series has been tested through a simple bpf program (available here:
> > https://github.com/LorenzoBianconi/bpf-workspace/tree/master/mt76_xdp_stats)
> > used to count frame types received by the device.
> > Possible eBPF use cases could be:
> > - implement new statistics through bpf maps
> > - implement fast packet filtering (e.g in monitor mode)
> > - ...
Hi Kalle,
>
> This is most likely a stupid question, but why do this in the driver and
> not in mac80211 so that all drivers could benefit from it? I guess there
> are reasons for that, I just can't figure that out.
>
> --
> Kalle Valo
I thought about that possibility (and I think it is definitely valuable) but
I preferred to work as close as possible to the hw running the bpf program
before skb allocation. My primary goal when I started thinking about eBPF over
WiFi was to perform fast packet filtering.
I think these two possibility are not mutually exclusive, we can fall-back to
mac80211 if hw driver does not support XDP.
This is an early stage implementation, at this point I would collect other
people opinions/concerns about using bpf/xdp directly on 802.11 frames.
Regards,
Lorenzo
Lorenzo Bianconi <[email protected]> writes:
>> Lorenzo Bianconi <[email protected]> writes:
>>
>> > This series is intended as a playground to start experimenting/developing
>> > with XDP/eBPF over WiFi and collect ideas/concerns about it.
>> > Introduce XDP support to mt76x2e/mt76x0e drivers. Currently supported
>> > actions are:
>> > - XDP_PASS
>> > - XDP_ABORTED
>> > - XDP_DROP
>> > Introduce ndo_bpf mac80211 callback in order to to load a bpf
>> > program into low level driver XDP rx hook.
>> > This series has been tested through a simple bpf program (available here:
>> > https://github.com/LorenzoBianconi/bpf-workspace/tree/master/mt76_xdp_stats)
>> > used to count frame types received by the device.
>> > Possible eBPF use cases could be:
>> > - implement new statistics through bpf maps
>> > - implement fast packet filtering (e.g in monitor mode)
>> > - ...
>
> Hi Kalle,
>
>>
>> This is most likely a stupid question, but why do this in the driver and
>> not in mac80211 so that all drivers could benefit from it? I guess there
>> are reasons for that, I just can't figure that out.
XDP achieves its speedup by running the eBPF program inside the driver
NAPI loop, before the kernel even touches the data in any other capacity
(and in particular, before it allocates an SKB). Which kinda means the
hook needs to be in the driver... Could be a fallback in mac80211,
though; although we'd have to figure out how that interacts with Generic
XDP.
> This is an early stage implementation, at this point I would collect
> other people opinions/concerns about using bpf/xdp directly on 802.11
> frames.
Thanks for looking into this!
I have two concerns with running XDP on 802.11 frames:
1. It makes it more difficult to add other XDP actions (such as
REDIRECT), as the XDP program would then have to make sure that the
outer packet headers are removed before, say, redirecting the packet
out of an ethernet interface. Also, if we do add redirect, we would
be bypassing mac80211 entirely; to what extent would that mess up
internal state?
2. UI consistency; suddenly, the user needs to know which kind of
frames to expect, and XDP program reuse becomes more difficult. This
may be unavoidable given the nature of XDP, but some thought needs to
go into this. Especially since we wouldn't necessarily be consistent
between WiFi drivers (there are fullmac devices that remove 802.11
headers before sending up the frame, right?).
Adding in Jesper; maybe he has some thoughts on this?
-Toke
On Wed, 28 Nov 2018 at 13:39, Toke Høiland-Jørgensen <[email protected]> wrote:[...]
> > This is an early stage implementation, at this point I would collect
> > other people opinions/concerns about using bpf/xdp directly on 802.11
> > frames.
>
> Thanks for looking into this!
I'm really hyped up with this especially when I think this could be
maybe offloaded to wlan cpu's for e.g. deep powersaving (allow host
system to suspend) without relying on blackbox logic you get today, or
802.11 -> 802.3 conversion (to offload host cpu).
> I have two concerns with running XDP on 802.11 frames:
>
> 1. It makes it more difficult to add other XDP actions (such as
> REDIRECT), as the XDP program would then have to make sure that the
> outer packet headers are removed before, say, redirecting the packet
> out of an ethernet interface. Also, if we do add redirect, we would
> be bypassing mac80211 entirely; to what extent would that mess up
> internal state?
>
> 2. UI consistency; suddenly, the user needs to know which kind of
> frames to expect, and XDP program reuse becomes more difficult. This
> may be unavoidable given the nature of XDP, but some thought needs to
> go into this. Especially since we wouldn't necessarily be consistent
> between WiFi drivers (there are fullmac devices that remove 802.11
> headers before sending up the frame, right?).
Yep - you can expect 802.3 frames or amsdu subframes (DA+SA+length)
too. In some cases you could also expect internal events (broadcom if
I understand their fullmac arch).
Michał
> >> > This series is intended as a playground to start experimenting/developing
> >> > with XDP/eBPF over WiFi and collect ideas/concerns about it.
> >> > Introduce XDP support to mt76x2e/mt76x0e drivers. Currently supported
> >> > actions are:
> >> > - XDP_PASS
> >> > - XDP_ABORTED
> >> > - XDP_DROP
> >> > Introduce ndo_bpf mac80211 callback in order to to load a bpf
> >> > program into low level driver XDP rx hook.
> >> > This series has been tested through a simple bpf program (available here:
> >> > https://github.com/LorenzoBianconi/bpf-workspace/tree/master/mt76_xdp_stats)
> >> > used to count frame types received by the device.
> >> > Possible eBPF use cases could be:
> >> > - implement new statistics through bpf maps
> >> > - implement fast packet filtering (e.g in monitor mode)
> >> > - ...
> >
> > Hi Kalle,
> >
> >>
> >> This is most likely a stupid question, but why do this in the driver and
> >> not in mac80211 so that all drivers could benefit from it? I guess there
> >> are reasons for that, I just can't figure that out.
>
> XDP achieves its speedup by running the eBPF program inside the driver
> NAPI loop, before the kernel even touches the data in any other capacity
> (and in particular, before it allocates an SKB). Which kinda means the
> hook needs to be in the driver... Could be a fallback in mac80211,
> though; although we'd have to figure out how that interacts with Generic
> XDP.
>
> > This is an early stage implementation, at this point I would collect
> > other people opinions/concerns about using bpf/xdp directly on 802.11
> > frames.
>
> Thanks for looking into this!
Hi Toke,
>
> I have two concerns with running XDP on 802.11 frames:
>
> 1. It makes it more difficult to add other XDP actions (such as
> REDIRECT), as the XDP program would then have to make sure that the
> outer packet headers are removed before, say, redirecting the packet
> out of an ethernet interface. Also, if we do add redirect, we would
> be bypassing mac80211 entirely; to what extent would that mess up
> internal state?
>
You are right, my assumption here is the logic/complexity is moved to the bpf
program that needs to take care of all possible issues that can be introduced.
More or less it is the same if a bpf program mess up with TCP segments on a
wired connection, isn't it?
> 2. UI consistency; suddenly, the user needs to know which kind of
> frames to expect, and XDP program reuse becomes more difficult. This
> may be unavoidable given the nature of XDP, but some thought needs to
> go into this. Especially since we wouldn't necessarily be consistent
> between WiFi drivers (there are fullmac devices that remove 802.11
> headers before sending up the frame, right?).
>
Right, maybe can we have some kind of 'wifi' bpf helpers?
Regards,
Lorenzo
>
> Adding in Jesper; maybe he has some thoughts on this?
>
> -Toke
Michał Kazior <[email protected]> writes:
> On Wed, 28 Nov 2018 at 13:39, Toke Høiland-Jørgensen <[email protected]> wrote:[...]
>> > This is an early stage implementation, at this point I would collect
>> > other people opinions/concerns about using bpf/xdp directly on 802.11
>> > frames.
>>
>> Thanks for looking into this!
>
> I'm really hyped up with this especially when I think this could be
> maybe offloaded to wlan cpu's for e.g. deep powersaving (allow host
> system to suspend) without relying on blackbox logic you get today, or
> 802.11 -> 802.3 conversion (to offload host cpu).
Yeah, good point!
>> I have two concerns with running XDP on 802.11 frames:
>>
>> 1. It makes it more difficult to add other XDP actions (such as
>> REDIRECT), as the XDP program would then have to make sure that the
>> outer packet headers are removed before, say, redirecting the packet
>> out of an ethernet interface. Also, if we do add redirect, we would
>> be bypassing mac80211 entirely; to what extent would that mess up
>> internal state?
>>
>> 2. UI consistency; suddenly, the user needs to know which kind of
>> frames to expect, and XDP program reuse becomes more difficult. This
>> may be unavoidable given the nature of XDP, but some thought needs to
>> go into this. Especially since we wouldn't necessarily be consistent
>> between WiFi drivers (there are fullmac devices that remove 802.11
>> headers before sending up the frame, right?).
>
> Yep - you can expect 802.3 frames or amsdu subframes (DA+SA+length)
> too. In some cases you could also expect internal events (broadcom if
> I understand their fullmac arch).
Oh boy. I figured it was going to be a can of worms; didn't realise it
was quite that big :/
-Toke
Lorenzo Bianconi <[email protected]> writes:
>> >> > This series is intended as a playground to start experimenting/developing
>> >> > with XDP/eBPF over WiFi and collect ideas/concerns about it.
>> >> > Introduce XDP support to mt76x2e/mt76x0e drivers. Currently supported
>> >> > actions are:
>> >> > - XDP_PASS
>> >> > - XDP_ABORTED
>> >> > - XDP_DROP
>> >> > Introduce ndo_bpf mac80211 callback in order to to load a bpf
>> >> > program into low level driver XDP rx hook.
>> >> > This series has been tested through a simple bpf program (available here:
>> >> > https://github.com/LorenzoBianconi/bpf-workspace/tree/master/mt76_xdp_stats)
>> >> > used to count frame types received by the device.
>> >> > Possible eBPF use cases could be:
>> >> > - implement new statistics through bpf maps
>> >> > - implement fast packet filtering (e.g in monitor mode)
>> >> > - ...
>> >
>> > Hi Kalle,
>> >
>> >>
>> >> This is most likely a stupid question, but why do this in the driver and
>> >> not in mac80211 so that all drivers could benefit from it? I guess there
>> >> are reasons for that, I just can't figure that out.
>>
>> XDP achieves its speedup by running the eBPF program inside the driver
>> NAPI loop, before the kernel even touches the data in any other capacity
>> (and in particular, before it allocates an SKB). Which kinda means the
>> hook needs to be in the driver... Could be a fallback in mac80211,
>> though; although we'd have to figure out how that interacts with Generic
>> XDP.
>>
>> > This is an early stage implementation, at this point I would collect
>> > other people opinions/concerns about using bpf/xdp directly on 802.11
>> > frames.
>>
>> Thanks for looking into this!
>
> Hi Toke,
>
>>
>> I have two concerns with running XDP on 802.11 frames:
>>
>> 1. It makes it more difficult to add other XDP actions (such as
>> REDIRECT), as the XDP program would then have to make sure that the
>> outer packet headers are removed before, say, redirecting the packet
>> out of an ethernet interface. Also, if we do add redirect, we would
>> be bypassing mac80211 entirely; to what extent would that mess up
>> internal state?
>>
>
> You are right, my assumption here is the logic/complexity is moved to
> the bpf program that needs to take care of all possible issues that
> can be introduced. More or less it is the same if a bpf program mess
> up with TCP segments on a wired connection, isn't it?
No, I guess not; except here it potentially applies to all packets
(things like BAW tracking), and it is *in addition* to TCP.
>> 2. UI consistency; suddenly, the user needs to know which kind of
>> frames to expect, and XDP program reuse becomes more difficult. This
>> may be unavoidable given the nature of XDP, but some thought needs to
>> go into this. Especially since we wouldn't necessarily be consistent
>> between WiFi drivers (there are fullmac devices that remove 802.11
>> headers before sending up the frame, right?).
>>
>
> Right, maybe can we have some kind of 'wifi' bpf helpers?
Yeah, I guess we would at least need helpers to update any internal
state in mac80211 (such as BAW), or BPF programs wouldn't even be able
to drop packets without messing things up...
-Toke
> Lorenzo Bianconi <[email protected]> writes:
>
> >> >> > This series is intended as a playground to start experimenting/developing
> >> >> > with XDP/eBPF over WiFi and collect ideas/concerns about it.
> >> >> > Introduce XDP support to mt76x2e/mt76x0e drivers. Currently supported
> >> >> > actions are:
> >> >> > - XDP_PASS
> >> >> > - XDP_ABORTED
> >> >> > - XDP_DROP
> >> >> > Introduce ndo_bpf mac80211 callback in order to to load a bpf
> >> >> > program into low level driver XDP rx hook.
> >> >> > This series has been tested through a simple bpf program (available here:
> >> >> > https://github.com/LorenzoBianconi/bpf-workspace/tree/master/mt76_xdp_stats)
> >> >> > used to count frame types received by the device.
> >> >> > Possible eBPF use cases could be:
> >> >> > - implement new statistics through bpf maps
> >> >> > - implement fast packet filtering (e.g in monitor mode)
> >> >> > - ...
> >> >
> >> > Hi Kalle,
> >> >
> >> >>
> >> >> This is most likely a stupid question, but why do this in the driver and
> >> >> not in mac80211 so that all drivers could benefit from it? I guess there
> >> >> are reasons for that, I just can't figure that out.
> >>
> >> XDP achieves its speedup by running the eBPF program inside the driver
> >> NAPI loop, before the kernel even touches the data in any other capacity
> >> (and in particular, before it allocates an SKB). Which kinda means the
> >> hook needs to be in the driver... Could be a fallback in mac80211,
> >> though; although we'd have to figure out how that interacts with Generic
> >> XDP.
> >>
> >> > This is an early stage implementation, at this point I would collect
> >> > other people opinions/concerns about using bpf/xdp directly on 802.11
> >> > frames.
> >>
> >> Thanks for looking into this!
> >
> > Hi Toke,
> >
> >>
> >> I have two concerns with running XDP on 802.11 frames:
> >>
> >> 1. It makes it more difficult to add other XDP actions (such as
> >> REDIRECT), as the XDP program would then have to make sure that the
> >> outer packet headers are removed before, say, redirecting the packet
> >> out of an ethernet interface. Also, if we do add redirect, we would
> >> be bypassing mac80211 entirely; to what extent would that mess up
> >> internal state?
> >>
> >
> > You are right, my assumption here is the logic/complexity is moved to
> > the bpf program that needs to take care of all possible issues that
> > can be introduced. More or less it is the same if a bpf program mess
> > up with TCP segments on a wired connection, isn't it?
>
> No, I guess not; except here it potentially applies to all packets
> (things like BAW tracking), and it is *in addition* to TCP.
Yes, here it is a little bit harder, but I was meaning that the bpf program
has to be very careful when dropping a packet :)
>
> >> 2. UI consistency; suddenly, the user needs to know which kind of
> >> frames to expect, and XDP program reuse becomes more difficult. This
> >> may be unavoidable given the nature of XDP, but some thought needs to
> >> go into this. Especially since we wouldn't necessarily be consistent
> >> between WiFi drivers (there are fullmac devices that remove 802.11
> >> headers before sending up the frame, right?).
> >>
> >
> > Right, maybe can we have some kind of 'wifi' bpf helpers?
>
> Yeah, I guess we would at least need helpers to update any internal
> state in mac80211 (such as BAW), or BPF programs wouldn't even be able
> to drop packets without messing things up...
>
Correct.
Regards,
Lorenzo
> -Toke
Lorenzo Bianconi <[email protected]> writes:
>> Lorenzo Bianconi <[email protected]> writes:
>>
>> >> >> > This series is intended as a playground to start experimenting/developing
>> >> >> > with XDP/eBPF over WiFi and collect ideas/concerns about it.
>> >> >> > Introduce XDP support to mt76x2e/mt76x0e drivers. Currently supported
>> >> >> > actions are:
>> >> >> > - XDP_PASS
>> >> >> > - XDP_ABORTED
>> >> >> > - XDP_DROP
>> >> >> > Introduce ndo_bpf mac80211 callback in order to to load a bpf
>> >> >> > program into low level driver XDP rx hook.
>> >> >> > This series has been tested through a simple bpf program (available here:
>> >> >> > https://github.com/LorenzoBianconi/bpf-workspace/tree/master/mt76_xdp_stats)
>> >> >> > used to count frame types received by the device.
>> >> >> > Possible eBPF use cases could be:
>> >> >> > - implement new statistics through bpf maps
>> >> >> > - implement fast packet filtering (e.g in monitor mode)
>> >> >> > - ...
>> >> >
>> >> > Hi Kalle,
>> >> >
>> >> >>
>> >> >> This is most likely a stupid question, but why do this in the driver and
>> >> >> not in mac80211 so that all drivers could benefit from it? I guess there
>> >> >> are reasons for that, I just can't figure that out.
>> >>
>> >> XDP achieves its speedup by running the eBPF program inside the driver
>> >> NAPI loop, before the kernel even touches the data in any other capacity
>> >> (and in particular, before it allocates an SKB). Which kinda means the
>> >> hook needs to be in the driver... Could be a fallback in mac80211,
>> >> though; although we'd have to figure out how that interacts with Generic
>> >> XDP.
>> >>
>> >> > This is an early stage implementation, at this point I would collect
>> >> > other people opinions/concerns about using bpf/xdp directly on 802.11
>> >> > frames.
>> >>
>> >> Thanks for looking into this!
>> >
>> > Hi Toke,
>> >
>> >>
>> >> I have two concerns with running XDP on 802.11 frames:
>> >>
>> >> 1. It makes it more difficult to add other XDP actions (such as
>> >> REDIRECT), as the XDP program would then have to make sure that the
>> >> outer packet headers are removed before, say, redirecting the packet
>> >> out of an ethernet interface. Also, if we do add redirect, we would
>> >> be bypassing mac80211 entirely; to what extent would that mess up
>> >> internal state?
>> >>
>> >
>> > You are right, my assumption here is the logic/complexity is moved to
>> > the bpf program that needs to take care of all possible issues that
>> > can be introduced. More or less it is the same if a bpf program mess
>> > up with TCP segments on a wired connection, isn't it?
>>
>> No, I guess not; except here it potentially applies to all packets
>> (things like BAW tracking), and it is *in addition* to TCP.
>
> Yes, here it is a little bit harder, but I was meaning that the bpf program
> has to be very careful when dropping a packet :)
Yeah. What kind of filtering were you thinking you would use this for in
the short term?
-Toke
On Nov 28, Toke H?iland-J?rgensen wrote:
> Lorenzo Bianconi <[email protected]> writes:
>
> >> Lorenzo Bianconi <[email protected]> writes:
> >>
> >> >> >> > This series is intended as a playground to start experimenting/developing
> >> >> >> > with XDP/eBPF over WiFi and collect ideas/concerns about it.
> >> >> >> > Introduce XDP support to mt76x2e/mt76x0e drivers. Currently supported
> >> >> >> > actions are:
> >> >> >> > - XDP_PASS
> >> >> >> > - XDP_ABORTED
> >> >> >> > - XDP_DROP
> >> >> >> > Introduce ndo_bpf mac80211 callback in order to to load a bpf
> >> >> >> > program into low level driver XDP rx hook.
> >> >> >> > This series has been tested through a simple bpf program (available here:
> >> >> >> > https://github.com/LorenzoBianconi/bpf-workspace/tree/master/mt76_xdp_stats)
> >> >> >> > used to count frame types received by the device.
> >> >> >> > Possible eBPF use cases could be:
> >> >> >> > - implement new statistics through bpf maps
> >> >> >> > - implement fast packet filtering (e.g in monitor mode)
> >> >> >> > - ...
> >> >> >
> >> >> > Hi Kalle,
> >> >> >
> >> >> >>
> >> >> >> This is most likely a stupid question, but why do this in the driver and
> >> >> >> not in mac80211 so that all drivers could benefit from it? I guess there
> >> >> >> are reasons for that, I just can't figure that out.
> >> >>
> >> >> XDP achieves its speedup by running the eBPF program inside the driver
> >> >> NAPI loop, before the kernel even touches the data in any other capacity
> >> >> (and in particular, before it allocates an SKB). Which kinda means the
> >> >> hook needs to be in the driver... Could be a fallback in mac80211,
> >> >> though; although we'd have to figure out how that interacts with Generic
> >> >> XDP.
> >> >>
> >> >> > This is an early stage implementation, at this point I would collect
> >> >> > other people opinions/concerns about using bpf/xdp directly on 802.11
> >> >> > frames.
> >> >>
> >> >> Thanks for looking into this!
> >> >
> >> > Hi Toke,
> >> >
> >> >>
> >> >> I have two concerns with running XDP on 802.11 frames:
> >> >>
> >> >> 1. It makes it more difficult to add other XDP actions (such as
> >> >> REDIRECT), as the XDP program would then have to make sure that the
> >> >> outer packet headers are removed before, say, redirecting the packet
> >> >> out of an ethernet interface. Also, if we do add redirect, we would
> >> >> be bypassing mac80211 entirely; to what extent would that mess up
> >> >> internal state?
> >> >>
> >> >
> >> > You are right, my assumption here is the logic/complexity is moved to
> >> > the bpf program that needs to take care of all possible issues that
> >> > can be introduced. More or less it is the same if a bpf program mess
> >> > up with TCP segments on a wired connection, isn't it?
> >>
> >> No, I guess not; except here it potentially applies to all packets
> >> (things like BAW tracking), and it is *in addition* to TCP.
> >
> > Yes, here it is a little bit harder, but I was meaning that the bpf program
> > has to be very careful when dropping a packet :)
>
> Yeah. What kind of filtering were you thinking you would use this for in
> the short term?
>
When I started working on XDP for mt76 I was thinking about BSSID filtering but
I was looking for a more general solution respect to add that feature in the
driver. Moreover we could use bpf for fast packet filtering when you add an
interface in monitor mode. Nevertheless I guess there could be other use cases not
limited to frame filtering. My primary goal with this series is to collect
ideas/concerns on WiFi XDP/eBPF possible uses cases.
Regards,
Lorenzo
> -Toke
On Wed, 28 Nov 2018 13:36:26 +0100
Toke Høiland-Jørgensen <[email protected]> wrote:
> Lorenzo Bianconi <[email protected]> writes:
>
> >> Lorenzo Bianconi <[email protected]> writes:
> >>
> >> > This series is intended as a playground to start experimenting/developing
> >> > with XDP/eBPF over WiFi and collect ideas/concerns about it.
> >> > Introduce XDP support to mt76x2e/mt76x0e drivers. Currently supported
> >> > actions are:
> >> > - XDP_PASS
> >> > - XDP_ABORTED
> >> > - XDP_DROP
> >> > Introduce ndo_bpf mac80211 callback in order to to load a bpf
> >> > program into low level driver XDP rx hook.
> >> > This series has been tested through a simple bpf program (available here:
> >> > https://github.com/LorenzoBianconi/bpf-workspace/tree/master/mt76_xdp_stats)
> >> > used to count frame types received by the device.
> >> > Possible eBPF use cases could be:
> >> > - implement new statistics through bpf maps
> >> > - implement fast packet filtering (e.g in monitor mode)
> >> > - ...
> >
> > Hi Kalle,
> >
> >>
> >> This is most likely a stupid question, but why do this in the driver and
> >> not in mac80211 so that all drivers could benefit from it? I guess there
> >> are reasons for that, I just can't figure that out.
>
> XDP achieves its speedup by running the eBPF program inside the driver
> NAPI loop, before the kernel even touches the data in any other capacity
> (and in particular, before it allocates an SKB). Which kinda means the
> hook needs to be in the driver... Could be a fallback in mac80211,
> though; although we'd have to figure out how that interacts with Generic
> XDP.
>
> > This is an early stage implementation, at this point I would collect
> > other people opinions/concerns about using bpf/xdp directly on 802.11
> > frames.
>
> Thanks for looking into this!
>
> I have two concerns with running XDP on 802.11 frames:
>
> 1. It makes it more difficult to add other XDP actions (such as
> REDIRECT), as the XDP program would then have to make sure that the
> outer packet headers are removed before, say, redirecting the packet
> out of an ethernet interface. Also, if we do add redirect, we would
> be bypassing mac80211 entirely; to what extent would that mess up
> internal state?
>
> 2. UI consistency; suddenly, the user needs to know which kind of
> frames to expect, and XDP program reuse becomes more difficult. This
> may be unavoidable given the nature of XDP, but some thought needs to
> go into this. Especially since we wouldn't necessarily be consistent
> between WiFi drivers (there are fullmac devices that remove 802.11
> headers before sending up the frame, right?).
>
>
> Adding in Jesper; maybe he has some thoughts on this?
Today XDP assumes the frame is an Ethernet frame. With WiFi I guess
this assumption change, right?
I worry a bit about this, as XDP is all about performance, and I don't
want to add performance regressions, by requiring all XDP programs or
core-code to having to check-frame-type before proceeding. That said, I
do think it is doable, without adding performance regressions.
Option #1 is to move the check-frame-type to setup time. By either
having frame-type be part of eBPF prog, or supply frame-type as option
XDP attach call. And then reject attaching XDP prog to a device, where
the expected frame-type does not match.
Option#2, leave it up to eBPF-programmer if they want to add runtime
checks. By extending xdp_rxq_info with frame-type (default to
Ethernet), which allow the eBPF-programmer choose to write a generic
XDP program that both work on Ethernet and WiFi, or skip-check as they
know this will e.g. only run on Wifi. (Note xdp_rxq_info is static
read-only info per RX-queue, will all Wifi frames have same frame-type?.
Also consider what happens in case of XDP_REDIRECT, from a Wifi NIC to
an Ethernet NIC. It would of-cause be cool to get this working cross,
Wifi-Ethernet.
Option#3 is to say, Wifi XDP is so different that we should create a
new (enum) bpf_prog_type. And then still see if we can leverage some
of the same core-code (as long as it doesn't slowdown performance).
--
Best regards,
Jesper Dangaard Brouer
MSc.CS, Principal Kernel Engineer at Red Hat
LinkedIn: http://www.linkedin.com/in/brouer
Lorenzo Bianconi <[email protected]> writes:
> On Nov 28, Toke Høiland-Jørgensen wrote:
>> Lorenzo Bianconi <[email protected]> writes:
>>
>> >> Lorenzo Bianconi <[email protected]> writes:
>> >>
>> >> >> >> > This series is intended as a playground to start experimenting/developing
>> >> >> >> > with XDP/eBPF over WiFi and collect ideas/concerns about it.
>> >> >> >> > Introduce XDP support to mt76x2e/mt76x0e drivers. Currently supported
>> >> >> >> > actions are:
>> >> >> >> > - XDP_PASS
>> >> >> >> > - XDP_ABORTED
>> >> >> >> > - XDP_DROP
>> >> >> >> > Introduce ndo_bpf mac80211 callback in order to to load a bpf
>> >> >> >> > program into low level driver XDP rx hook.
>> >> >> >> > This series has been tested through a simple bpf program (available here:
>> >> >> >> > https://github.com/LorenzoBianconi/bpf-workspace/tree/master/mt76_xdp_stats)
>> >> >> >> > used to count frame types received by the device.
>> >> >> >> > Possible eBPF use cases could be:
>> >> >> >> > - implement new statistics through bpf maps
>> >> >> >> > - implement fast packet filtering (e.g in monitor mode)
>> >> >> >> > - ...
>> >> >> >
>> >> >> > Hi Kalle,
>> >> >> >
>> >> >> >>
>> >> >> >> This is most likely a stupid question, but why do this in the driver and
>> >> >> >> not in mac80211 so that all drivers could benefit from it? I guess there
>> >> >> >> are reasons for that, I just can't figure that out.
>> >> >>
>> >> >> XDP achieves its speedup by running the eBPF program inside the driver
>> >> >> NAPI loop, before the kernel even touches the data in any other capacity
>> >> >> (and in particular, before it allocates an SKB). Which kinda means the
>> >> >> hook needs to be in the driver... Could be a fallback in mac80211,
>> >> >> though; although we'd have to figure out how that interacts with Generic
>> >> >> XDP.
>> >> >>
>> >> >> > This is an early stage implementation, at this point I would collect
>> >> >> > other people opinions/concerns about using bpf/xdp directly on 802.11
>> >> >> > frames.
>> >> >>
>> >> >> Thanks for looking into this!
>> >> >
>> >> > Hi Toke,
>> >> >
>> >> >>
>> >> >> I have two concerns with running XDP on 802.11 frames:
>> >> >>
>> >> >> 1. It makes it more difficult to add other XDP actions (such as
>> >> >> REDIRECT), as the XDP program would then have to make sure that the
>> >> >> outer packet headers are removed before, say, redirecting the packet
>> >> >> out of an ethernet interface. Also, if we do add redirect, we would
>> >> >> be bypassing mac80211 entirely; to what extent would that mess up
>> >> >> internal state?
>> >> >>
>> >> >
>> >> > You are right, my assumption here is the logic/complexity is moved to
>> >> > the bpf program that needs to take care of all possible issues that
>> >> > can be introduced. More or less it is the same if a bpf program mess
>> >> > up with TCP segments on a wired connection, isn't it?
>> >>
>> >> No, I guess not; except here it potentially applies to all packets
>> >> (things like BAW tracking), and it is *in addition* to TCP.
>> >
>> > Yes, here it is a little bit harder, but I was meaning that the bpf program
>> > has to be very careful when dropping a packet :)
>>
>> Yeah. What kind of filtering were you thinking you would use this for in
>> the short term?
>>
>
> When I started working on XDP for mt76 I was thinking about BSSID
> filtering but I was looking for a more general solution respect to add
> that feature in the driver. Moreover we could use bpf for fast packet
> filtering when you add an interface in monitor mode.
Yup, both of these make sense.
> Nevertheless I guess there could be other use cases not limited to
> frame filtering. My primary goal with this series is to collect
> ideas/concerns on WiFi XDP/eBPF possible uses cases.
Well, Michał's idea about offloading is cool if it is possible to get
vendors to implement it.
Other than that, if we can solve the issues with differences between
802.11 and plain Ethernet frames, I see no reason why it wouldn't be
possible to implement an XDP fast-path for WiFi-to-Ethernet forwarding,
which might be useful in an access point, especially as WiFi speeds
increase.
The other direction will probably be more difficult, at least if 802.11
frames need to be built in software. It *might* be possible with the XDP
egress hook we are planning (with a suitable set of helpers, the eBPF
program could build the 802.11 frames), but I'm not really sure if that
is worth doing as I'm quite sure there are some hairy edge cases
there...
-Toke
> On Wed, 28 Nov 2018 13:36:26 +0100
> Toke H?iland-J?rgensen <[email protected]> wrote:
>
> > Lorenzo Bianconi <[email protected]> writes:
> >
> > >> Lorenzo Bianconi <[email protected]> writes:
> > >>
> > >> > This series is intended as a playground to start experimenting/developing
> > >> > with XDP/eBPF over WiFi and collect ideas/concerns about it.
> > >> > Introduce XDP support to mt76x2e/mt76x0e drivers. Currently supported
> > >> > actions are:
> > >> > - XDP_PASS
> > >> > - XDP_ABORTED
> > >> > - XDP_DROP
> > >> > Introduce ndo_bpf mac80211 callback in order to to load a bpf
> > >> > program into low level driver XDP rx hook.
> > >> > This series has been tested through a simple bpf program (available here:
> > >> > https://github.com/LorenzoBianconi/bpf-workspace/tree/master/mt76_xdp_stats)
> > >> > used to count frame types received by the device.
> > >> > Possible eBPF use cases could be:
> > >> > - implement new statistics through bpf maps
> > >> > - implement fast packet filtering (e.g in monitor mode)
> > >> > - ...
> > >
> > > Hi Kalle,
> > >
> > >>
> > >> This is most likely a stupid question, but why do this in the driver and
> > >> not in mac80211 so that all drivers could benefit from it? I guess there
> > >> are reasons for that, I just can't figure that out.
> >
> > XDP achieves its speedup by running the eBPF program inside the driver
> > NAPI loop, before the kernel even touches the data in any other capacity
> > (and in particular, before it allocates an SKB). Which kinda means the
> > hook needs to be in the driver... Could be a fallback in mac80211,
> > though; although we'd have to figure out how that interacts with Generic
> > XDP.
> >
> > > This is an early stage implementation, at this point I would collect
> > > other people opinions/concerns about using bpf/xdp directly on 802.11
> > > frames.
> >
> > Thanks for looking into this!
> >
> > I have two concerns with running XDP on 802.11 frames:
> >
> > 1. It makes it more difficult to add other XDP actions (such as
> > REDIRECT), as the XDP program would then have to make sure that the
> > outer packet headers are removed before, say, redirecting the packet
> > out of an ethernet interface. Also, if we do add redirect, we would
> > be bypassing mac80211 entirely; to what extent would that mess up
> > internal state?
> >
> > 2. UI consistency; suddenly, the user needs to know which kind of
> > frames to expect, and XDP program reuse becomes more difficult. This
> > may be unavoidable given the nature of XDP, but some thought needs to
> > go into this. Especially since we wouldn't necessarily be consistent
> > between WiFi drivers (there are fullmac devices that remove 802.11
> > headers before sending up the frame, right?).
> >
> >
> > Adding in Jesper; maybe he has some thoughts on this?
Hi Jesper,
>
> Today XDP assumes the frame is an Ethernet frame. With WiFi I guess
> this assumption change, right?
yes correct, SoftMAC devices report 802.11 frames to the stack
> I worry a bit about this, as XDP is all about performance, and I don't
> want to add performance regressions, by requiring all XDP programs or
> core-code to having to check-frame-type before proceeding. That said, I
> do think it is doable, without adding performance regressions.
>
> Option #1 is to move the check-frame-type to setup time. By either
> having frame-type be part of eBPF prog, or supply frame-type as option
> XDP attach call. And then reject attaching XDP prog to a device, where
> the expected frame-type does not match.
>
I guess it will be enough to avoid loading a 'non-WiFi' bpf program on a 802.11
netdevice (and vice versa). We could add a flag (or something similar) in
XDP_SETUP_PROG section of netdev_bpf data structure and use ieee80211_ptr
netdevice pointer in order to guarantee that the bpf program will work on the
expected 'frame-type'
> Option#2, leave it up to eBPF-programmer if they want to add runtime
> checks. By extending xdp_rxq_info with frame-type (default to
> Ethernet), which allow the eBPF-programmer choose to write a generic
> XDP program that both work on Ethernet and WiFi, or skip-check as they
> know this will e.g. only run on Wifi. (Note xdp_rxq_info is static
> read-only info per RX-queue, will all Wifi frames have same frame-type?.
>
802.11 standards define three frame subtype (data, management and control).
Subtypes could be detected parsing 802.11 header
>
> Also consider what happens in case of XDP_REDIRECT, from a Wifi NIC to
> an Ethernet NIC. It would of-cause be cool to get this working cross,
> Wifi-Ethernet.
>
Very cool :) On tx side the driver will accept standard ethernet frames in
ndo_xdp_xmit pointer
> Option#3 is to say, Wifi XDP is so different that we should create a
> new (enum) bpf_prog_type. And then still see if we can leverage some
> of the same core-code (as long as it doesn't slowdown performance).
>
Do you think that Option#3 will be more 'future-proof' respect to Option#1?
Regards,
Lorenzo
> --
> Best regards,
> Jesper Dangaard Brouer
> MSc.CS, Principal Kernel Engineer at Red Hat
> LinkedIn: http://www.linkedin.com/in/brouer
> Lorenzo Bianconi <[email protected]> writes:
>
> > On Nov 28, Toke Høiland-Jørgensen wrote:
> >> Lorenzo Bianconi <[email protected]> writes:
> >>
> >> >> Lorenzo Bianconi <[email protected]> writes:
> >> >>
> >> >> >> >> > This series is intended as a playground to start experimenting/developing
> >> >> >> >> > with XDP/eBPF over WiFi and collect ideas/concerns about it.
> >> >> >> >> > Introduce XDP support to mt76x2e/mt76x0e drivers. Currently supported
> >> >> >> >> > actions are:
> >> >> >> >> > - XDP_PASS
> >> >> >> >> > - XDP_ABORTED
> >> >> >> >> > - XDP_DROP
> >> >> >> >> > Introduce ndo_bpf mac80211 callback in order to to load a bpf
> >> >> >> >> > program into low level driver XDP rx hook.
> >> >> >> >> > This series has been tested through a simple bpf program (available here:
> >> >> >> >> > https://github.com/LorenzoBianconi/bpf-workspace/tree/master/mt76_xdp_stats)
> >> >> >> >> > used to count frame types received by the device.
> >> >> >> >> > Possible eBPF use cases could be:
> >> >> >> >> > - implement new statistics through bpf maps
> >> >> >> >> > - implement fast packet filtering (e.g in monitor mode)
> >> >> >> >> > - ...
> >> >> >> >
> >> >> >> > Hi Kalle,
> >> >> >> >
> >> >> >> >>
> >> >> >> >> This is most likely a stupid question, but why do this in the driver and
> >> >> >> >> not in mac80211 so that all drivers could benefit from it? I guess there
> >> >> >> >> are reasons for that, I just can't figure that out.
> >> >> >>
> >> >> >> XDP achieves its speedup by running the eBPF program inside the driver
> >> >> >> NAPI loop, before the kernel even touches the data in any other capacity
> >> >> >> (and in particular, before it allocates an SKB). Which kinda means the
> >> >> >> hook needs to be in the driver... Could be a fallback in mac80211,
> >> >> >> though; although we'd have to figure out how that interacts with Generic
> >> >> >> XDP.
> >> >> >>
> >> >> >> > This is an early stage implementation, at this point I would collect
> >> >> >> > other people opinions/concerns about using bpf/xdp directly on 802.11
> >> >> >> > frames.
> >> >> >>
> >> >> >> Thanks for looking into this!
> >> >> >
> >> >> > Hi Toke,
> >> >> >
> >> >> >>
> >> >> >> I have two concerns with running XDP on 802.11 frames:
> >> >> >>
> >> >> >> 1. It makes it more difficult to add other XDP actions (such as
> >> >> >> REDIRECT), as the XDP program would then have to make sure that the
> >> >> >> outer packet headers are removed before, say, redirecting the packet
> >> >> >> out of an ethernet interface. Also, if we do add redirect, we would
> >> >> >> be bypassing mac80211 entirely; to what extent would that mess up
> >> >> >> internal state?
> >> >> >>
> >> >> >
> >> >> > You are right, my assumption here is the logic/complexity is moved to
> >> >> > the bpf program that needs to take care of all possible issues that
> >> >> > can be introduced. More or less it is the same if a bpf program mess
> >> >> > up with TCP segments on a wired connection, isn't it?
> >> >>
> >> >> No, I guess not; except here it potentially applies to all packets
> >> >> (things like BAW tracking), and it is *in addition* to TCP.
> >> >
> >> > Yes, here it is a little bit harder, but I was meaning that the bpf program
> >> > has to be very careful when dropping a packet :)
> >>
> >> Yeah. What kind of filtering were you thinking you would use this for in
> >> the short term?
> >>
> >
> > When I started working on XDP for mt76 I was thinking about BSSID
> > filtering but I was looking for a more general solution respect to add
> > that feature in the driver. Moreover we could use bpf for fast packet
> > filtering when you add an interface in monitor mode.
>
> Yup, both of these make sense.
>
> > Nevertheless I guess there could be other use cases not limited to
> > frame filtering. My primary goal with this series is to collect
> > ideas/concerns on WiFi XDP/eBPF possible uses cases.
>
> Well, Michał's idea about offloading is cool if it is possible to get
> vendors to implement it.
>
Yep, would be very cool :)
> Other than that, if we can solve the issues with differences between
> 802.11 and plain Ethernet frames, I see no reason why it wouldn't be
> possible to implement an XDP fast-path for WiFi-to-Ethernet forwarding,
> which might be useful in an access point, especially as WiFi speeds
> increase.
>
Agree
> The other direction will probably be more difficult, at least if 802.11
> frames need to be built in software. It *might* be possible with the XDP
> egress hook we are planning (with a suitable set of helpers, the eBPF
> program could build the 802.11 frames), but I'm not really sure if that
> is worth doing as I'm quite sure there are some hairy edge cases
> there...
The possible issue with XDP_DROP action you are referring to here is A-MPDU
reordering on rx side, right? If so I guess the issue will be fixed by
tid_agg_rx->reorder_timer. Are you referring to other possible edge cases?
Regards,
Lorenzo
>
> -Toke
Lorenzo Bianconi <[email protected]> writes:
>> On Wed, 28 Nov 2018 13:36:26 +0100
>> Toke Høiland-Jørgensen <[email protected]> wrote:
>>
>> > Lorenzo Bianconi <[email protected]> writes:
>> >
>> > >> Lorenzo Bianconi <[email protected]> writes:
>> > >>
>> > >> > This series is intended as a playground to start experimenting/developing
>> > >> > with XDP/eBPF over WiFi and collect ideas/concerns about it.
>> > >> > Introduce XDP support to mt76x2e/mt76x0e drivers. Currently supported
>> > >> > actions are:
>> > >> > - XDP_PASS
>> > >> > - XDP_ABORTED
>> > >> > - XDP_DROP
>> > >> > Introduce ndo_bpf mac80211 callback in order to to load a bpf
>> > >> > program into low level driver XDP rx hook.
>> > >> > This series has been tested through a simple bpf program (available here:
>> > >> > https://github.com/LorenzoBianconi/bpf-workspace/tree/master/mt76_xdp_stats)
>> > >> > used to count frame types received by the device.
>> > >> > Possible eBPF use cases could be:
>> > >> > - implement new statistics through bpf maps
>> > >> > - implement fast packet filtering (e.g in monitor mode)
>> > >> > - ...
>> > >
>> > > Hi Kalle,
>> > >
>> > >>
>> > >> This is most likely a stupid question, but why do this in the driver and
>> > >> not in mac80211 so that all drivers could benefit from it? I guess there
>> > >> are reasons for that, I just can't figure that out.
>> >
>> > XDP achieves its speedup by running the eBPF program inside the driver
>> > NAPI loop, before the kernel even touches the data in any other capacity
>> > (and in particular, before it allocates an SKB). Which kinda means the
>> > hook needs to be in the driver... Could be a fallback in mac80211,
>> > though; although we'd have to figure out how that interacts with Generic
>> > XDP.
>> >
>> > > This is an early stage implementation, at this point I would collect
>> > > other people opinions/concerns about using bpf/xdp directly on 802.11
>> > > frames.
>> >
>> > Thanks for looking into this!
>> >
>> > I have two concerns with running XDP on 802.11 frames:
>> >
>> > 1. It makes it more difficult to add other XDP actions (such as
>> > REDIRECT), as the XDP program would then have to make sure that the
>> > outer packet headers are removed before, say, redirecting the packet
>> > out of an ethernet interface. Also, if we do add redirect, we would
>> > be bypassing mac80211 entirely; to what extent would that mess up
>> > internal state?
>> >
>> > 2. UI consistency; suddenly, the user needs to know which kind of
>> > frames to expect, and XDP program reuse becomes more difficult. This
>> > may be unavoidable given the nature of XDP, but some thought needs to
>> > go into this. Especially since we wouldn't necessarily be consistent
>> > between WiFi drivers (there are fullmac devices that remove 802.11
>> > headers before sending up the frame, right?).
>> >
>> >
>> > Adding in Jesper; maybe he has some thoughts on this?
>
> Hi Jesper,
>
>>
>> Today XDP assumes the frame is an Ethernet frame. With WiFi I guess
>> this assumption change, right?
>
> yes correct, SoftMAC devices report 802.11 frames to the stack
>
>> I worry a bit about this, as XDP is all about performance, and I don't
>> want to add performance regressions, by requiring all XDP programs or
>> core-code to having to check-frame-type before proceeding. That said, I
>> do think it is doable, without adding performance regressions.
>>
>> Option #1 is to move the check-frame-type to setup time. By either
>> having frame-type be part of eBPF prog, or supply frame-type as option
>> XDP attach call. And then reject attaching XDP prog to a device, where
>> the expected frame-type does not match.
>>
>
> I guess it will be enough to avoid loading a 'non-WiFi' bpf program on
> a 802.11 netdevice (and vice versa). We could add a flag (or something
> similar) in XDP_SETUP_PROG section of netdev_bpf data structure and
> use ieee80211_ptr netdevice pointer in order to guarantee that the bpf
> program will work on the expected 'frame-type'
Yeah, a flag would be good; we've been discussing that for other XDP use
cases; it's not a done deal yet, but I think it would be useful.
>
>> Option#2, leave it up to eBPF-programmer if they want to add runtime
>> checks. By extending xdp_rxq_info with frame-type (default to
>> Ethernet), which allow the eBPF-programmer choose to write a generic
>> XDP program that both work on Ethernet and WiFi, or skip-check as they
>> know this will e.g. only run on Wifi. (Note xdp_rxq_info is static
>> read-only info per RX-queue, will all Wifi frames have same frame-type?.
>>
>
> 802.11 standards define three frame subtype (data, management and control).
> Subtypes could be detected parsing 802.11 header
>
>>
>> Also consider what happens in case of XDP_REDIRECT, from a Wifi NIC to
>> an Ethernet NIC. It would of-cause be cool to get this working cross,
>> Wifi-Ethernet.
>>
>
> Very cool :) On tx side the driver will accept standard ethernet frames in
> ndo_xdp_xmit pointer
How do you envision that will work with drivers that build software
802.11 frames? The TX hook would have to be in mac80211 somewhere,
wouldn't it?
>> Option#3 is to say, Wifi XDP is so different that we should create a
>> new (enum) bpf_prog_type. And then still see if we can leverage some
>> of the same core-code (as long as it doesn't slowdown performance).
>>
>
> Do you think that Option#3 will be more 'future-proof' respect to
> Option#1?
My feeling is that WiFi devices are not sufficiently different to
warrant a whole new program type. We risk combinatorial explosion for
all the stuff that is the same, but now need to be tested for two (or N)
types...
-Toke
Lorenzo Bianconi <[email protected]> writes:
>> Lorenzo Bianconi <[email protected]> writes:
>>
>> > On Nov 28, Toke Høiland-Jørgensen wrote:
>> >> Lorenzo Bianconi <[email protected]> writes:
>> >>
>> >> >> Lorenzo Bianconi <[email protected]> writes:
>> >> >>
>> >> >> >> >> > This series is intended as a playground to start experimenting/developing
>> >> >> >> >> > with XDP/eBPF over WiFi and collect ideas/concerns about it.
>> >> >> >> >> > Introduce XDP support to mt76x2e/mt76x0e drivers. Currently supported
>> >> >> >> >> > actions are:
>> >> >> >> >> > - XDP_PASS
>> >> >> >> >> > - XDP_ABORTED
>> >> >> >> >> > - XDP_DROP
>> >> >> >> >> > Introduce ndo_bpf mac80211 callback in order to to load a bpf
>> >> >> >> >> > program into low level driver XDP rx hook.
>> >> >> >> >> > This series has been tested through a simple bpf program (available here:
>> >> >> >> >> > https://github.com/LorenzoBianconi/bpf-workspace/tree/master/mt76_xdp_stats)
>> >> >> >> >> > used to count frame types received by the device.
>> >> >> >> >> > Possible eBPF use cases could be:
>> >> >> >> >> > - implement new statistics through bpf maps
>> >> >> >> >> > - implement fast packet filtering (e.g in monitor mode)
>> >> >> >> >> > - ...
>> >> >> >> >
>> >> >> >> > Hi Kalle,
>> >> >> >> >
>> >> >> >> >>
>> >> >> >> >> This is most likely a stupid question, but why do this in the driver and
>> >> >> >> >> not in mac80211 so that all drivers could benefit from it? I guess there
>> >> >> >> >> are reasons for that, I just can't figure that out.
>> >> >> >>
>> >> >> >> XDP achieves its speedup by running the eBPF program inside the driver
>> >> >> >> NAPI loop, before the kernel even touches the data in any other capacity
>> >> >> >> (and in particular, before it allocates an SKB). Which kinda means the
>> >> >> >> hook needs to be in the driver... Could be a fallback in mac80211,
>> >> >> >> though; although we'd have to figure out how that interacts with Generic
>> >> >> >> XDP.
>> >> >> >>
>> >> >> >> > This is an early stage implementation, at this point I would collect
>> >> >> >> > other people opinions/concerns about using bpf/xdp directly on 802.11
>> >> >> >> > frames.
>> >> >> >>
>> >> >> >> Thanks for looking into this!
>> >> >> >
>> >> >> > Hi Toke,
>> >> >> >
>> >> >> >>
>> >> >> >> I have two concerns with running XDP on 802.11 frames:
>> >> >> >>
>> >> >> >> 1. It makes it more difficult to add other XDP actions (such as
>> >> >> >> REDIRECT), as the XDP program would then have to make sure that the
>> >> >> >> outer packet headers are removed before, say, redirecting the packet
>> >> >> >> out of an ethernet interface. Also, if we do add redirect, we would
>> >> >> >> be bypassing mac80211 entirely; to what extent would that mess up
>> >> >> >> internal state?
>> >> >> >>
>> >> >> >
>> >> >> > You are right, my assumption here is the logic/complexity is moved to
>> >> >> > the bpf program that needs to take care of all possible issues that
>> >> >> > can be introduced. More or less it is the same if a bpf program mess
>> >> >> > up with TCP segments on a wired connection, isn't it?
>> >> >>
>> >> >> No, I guess not; except here it potentially applies to all packets
>> >> >> (things like BAW tracking), and it is *in addition* to TCP.
>> >> >
>> >> > Yes, here it is a little bit harder, but I was meaning that the bpf program
>> >> > has to be very careful when dropping a packet :)
>> >>
>> >> Yeah. What kind of filtering were you thinking you would use this for in
>> >> the short term?
>> >>
>> >
>> > When I started working on XDP for mt76 I was thinking about BSSID
>> > filtering but I was looking for a more general solution respect to add
>> > that feature in the driver. Moreover we could use bpf for fast packet
>> > filtering when you add an interface in monitor mode.
>>
>> Yup, both of these make sense.
>>
>> > Nevertheless I guess there could be other use cases not limited to
>> > frame filtering. My primary goal with this series is to collect
>> > ideas/concerns on WiFi XDP/eBPF possible uses cases.
>>
>> Well, Michał's idea about offloading is cool if it is possible to get
>> vendors to implement it.
>>
>
> Yep, would be very cool :)
>
>> Other than that, if we can solve the issues with differences between
>> 802.11 and plain Ethernet frames, I see no reason why it wouldn't be
>> possible to implement an XDP fast-path for WiFi-to-Ethernet forwarding,
>> which might be useful in an access point, especially as WiFi speeds
>> increase.
>>
>
> Agree
>
>> The other direction will probably be more difficult, at least if 802.11
>> frames need to be built in software. It *might* be possible with the XDP
>> egress hook we are planning (with a suitable set of helpers, the eBPF
>> program could build the 802.11 frames), but I'm not really sure if that
>> is worth doing as I'm quite sure there are some hairy edge cases
>> there...
>
> The possible issue with XDP_DROP action you are referring to here is
> A-MPDU reordering on rx side, right? If so I guess the issue will be
> fixed by tid_agg_rx->reorder_timer. Are you referring to other
> possible edge cases?
I'm not sure, which is why I was being deliberately vague... ;)
Just something to be aware of when going beyond proof of concept, I
guess: here be dragons.
-Toke
On Thu, 29 Nov 2018 at 14:31, Toke Høiland-Jørgensen <[email protected]> wrote:
[...]
> >> Option#3 is to say, Wifi XDP is so different that we should create a
> >> new (enum) bpf_prog_type. And then still see if we can leverage some
> >> of the same core-code (as long as it doesn't slowdown performance).
> >>
> >
> > Do you think that Option#3 will be more 'future-proof' respect to
> > Option#1?
>
> My feeling is that WiFi devices are not sufficiently different to
> warrant a whole new program type. We risk combinatorial explosion for
> all the stuff that is the same, but now need to be tested for two (or N)
> types...
I'm not sure if my understanding is correct, but XDP seems like it can
(and intends to be able to?) act as a general purpose packet
accelerator (REDIRECT action was mentioned so I'm inferring..). In
such case you'll need to be able to perform transformations on packets
too, e.g. strip/prepend vlan tags, gre headers and what have you. The
802.3 <-> 802.11 conversion could be treated on equal terms.
Michał
On Thu, 29 Nov 2018 at 14:00, Lorenzo Bianconi
<[email protected]> wrote:
[...]
> > The other direction will probably be more difficult, at least if 802.11
> > frames need to be built in software. It *might* be possible with the XDP
> > egress hook we are planning (with a suitable set of helpers, the eBPF
> > program could build the 802.11 frames), but I'm not really sure if that
> > is worth doing as I'm quite sure there are some hairy edge cases
> > there...
>
> The possible issue with XDP_DROP action you are referring to here is A-MPDU
> reordering on rx side, right? If so I guess the issue will be fixed by
> tid_agg_rx->reorder_timer. Are you referring to other possible edge cases?
What I'm thinking is reordering could be one of possible things to
offload to an XDP program. It would require per-station data structure
to keep track of the frame sequence numbers (among other things). Same
could be said for crypto offloads (would require XDP programs to be
able to use crypto apis I guess?).
Michał
Michał Kazior <[email protected]> writes:
> On Thu, 29 Nov 2018 at 14:31, Toke Høiland-Jørgensen <[email protected]> wrote:
> [...]
>> >> Option#3 is to say, Wifi XDP is so different that we should create a
>> >> new (enum) bpf_prog_type. And then still see if we can leverage some
>> >> of the same core-code (as long as it doesn't slowdown performance).
>> >>
>> >
>> > Do you think that Option#3 will be more 'future-proof' respect to
>> > Option#1?
>>
>> My feeling is that WiFi devices are not sufficiently different to
>> warrant a whole new program type. We risk combinatorial explosion for
>> all the stuff that is the same, but now need to be tested for two (or N)
>> types...
>
> I'm not sure if my understanding is correct, but XDP seems like it can
> (and intends to be able to?) act as a general purpose packet
> accelerator (REDIRECT action was mentioned so I'm inferring..). In
> such case you'll need to be able to perform transformations on packets
> too, e.g. strip/prepend vlan tags, gre headers and what have you. The
> 802.3 <-> 802.11 conversion could be treated on equal terms.
Sure, that is perfectly possible. It just needs to be implemented by the
eBPF program being loaded; and there is a facility to shrink/expand the
packet size for encapsulation processing. It's just that since currently
all interfaces process Ethernet frames, this frame type assumption has
kinda been built in. So something needs to be done to handle that.
The minimum is just to ignore the issue and make it up to the program
writer to handle this or risk breaking things, but something a bit
friendlier (such as an ability for the loaded program to indicate which
frame type it is prepared to handle, which can be sanity-checked by the
loader) might be a good idea... Especially if different WiFi devices
will emit different frame types (so we can't just go "it's a WiFi
device, you should know this").
Then there's the additional issue that since mac80211 and/or the driver
may have internal state, we need to expose appropriate helpers for an
XDP program to be able to update this as it's processing packets (before
bypassing the stack, for instance).
-Toke
Michał Kazior <[email protected]> writes:
> On Thu, 29 Nov 2018 at 14:00, Lorenzo Bianconi
> <[email protected]> wrote:
> [...]
>> > The other direction will probably be more difficult, at least if 802.11
>> > frames need to be built in software. It *might* be possible with the XDP
>> > egress hook we are planning (with a suitable set of helpers, the eBPF
>> > program could build the 802.11 frames), but I'm not really sure if that
>> > is worth doing as I'm quite sure there are some hairy edge cases
>> > there...
>>
>> The possible issue with XDP_DROP action you are referring to here is A-MPDU
>> reordering on rx side, right? If so I guess the issue will be fixed by
>> tid_agg_rx->reorder_timer. Are you referring to other possible edge cases?
>
> What I'm thinking is reordering could be one of possible things to
> offload to an XDP program. It would require per-station data structure
> to keep track of the frame sequence numbers (among other things). Same
> could be said for crypto offloads (would require XDP programs to be
> able to use crypto apis I guess?).
In principle, all of this could be done. But we need to think carefully
about what things it really makes sense to offload to XDP. In the
general case, we will end up re-implementing all of mac80211 in eBPF,
which is obviously not ideal. However, fast-path handling in XDP, which
will punt to the full stack on edge cases, is probably doable (and is
the general model we envision for XDP programs).
For crypto in particular, I wonder if there is much of a speedup to be
had if the crypto is in software anyway. Wouldn't that dominate the
processing time? Whereas, if the crypto is offloaded to hardware,
fast-path packet processing in XDP might make sense. This would
translate to the fallback mode I mention above: If hardware crypto is
enabled, handle in XDP fast-path, otherwise punt to mac80211 for full
(crypto) processing.
-Toke
> Lorenzo Bianconi <[email protected]> writes:
>
> >> On Wed, 28 Nov 2018 13:36:26 +0100
> >> Toke H?iland-J?rgensen <[email protected]> wrote:
> >>
> >> > Lorenzo Bianconi <[email protected]> writes:
> >> >
> >> > >> Lorenzo Bianconi <[email protected]> writes:
> >> > >>
> >> > >> > This series is intended as a playground to start experimenting/developing
> >> > >> > with XDP/eBPF over WiFi and collect ideas/concerns about it.
> >> > >> > Introduce XDP support to mt76x2e/mt76x0e drivers. Currently supported
> >> > >> > actions are:
> >> > >> > - XDP_PASS
> >> > >> > - XDP_ABORTED
> >> > >> > - XDP_DROP
> >> > >> > Introduce ndo_bpf mac80211 callback in order to to load a bpf
> >> > >> > program into low level driver XDP rx hook.
> >> > >> > This series has been tested through a simple bpf program (available here:
> >> > >> > https://github.com/LorenzoBianconi/bpf-workspace/tree/master/mt76_xdp_stats)
> >> > >> > used to count frame types received by the device.
> >> > >> > Possible eBPF use cases could be:
> >> > >> > - implement new statistics through bpf maps
> >> > >> > - implement fast packet filtering (e.g in monitor mode)
> >> > >> > - ...
> >> > >
> >> > > Hi Kalle,
> >> > >
> >> > >>
> >> > >> This is most likely a stupid question, but why do this in the driver and
> >> > >> not in mac80211 so that all drivers could benefit from it? I guess there
> >> > >> are reasons for that, I just can't figure that out.
> >> >
> >> > XDP achieves its speedup by running the eBPF program inside the driver
> >> > NAPI loop, before the kernel even touches the data in any other capacity
> >> > (and in particular, before it allocates an SKB). Which kinda means the
> >> > hook needs to be in the driver... Could be a fallback in mac80211,
> >> > though; although we'd have to figure out how that interacts with Generic
> >> > XDP.
> >> >
> >> > > This is an early stage implementation, at this point I would collect
> >> > > other people opinions/concerns about using bpf/xdp directly on 802.11
> >> > > frames.
> >> >
> >> > Thanks for looking into this!
> >> >
> >> > I have two concerns with running XDP on 802.11 frames:
> >> >
> >> > 1. It makes it more difficult to add other XDP actions (such as
> >> > REDIRECT), as the XDP program would then have to make sure that the
> >> > outer packet headers are removed before, say, redirecting the packet
> >> > out of an ethernet interface. Also, if we do add redirect, we would
> >> > be bypassing mac80211 entirely; to what extent would that mess up
> >> > internal state?
> >> >
> >> > 2. UI consistency; suddenly, the user needs to know which kind of
> >> > frames to expect, and XDP program reuse becomes more difficult. This
> >> > may be unavoidable given the nature of XDP, but some thought needs to
> >> > go into this. Especially since we wouldn't necessarily be consistent
> >> > between WiFi drivers (there are fullmac devices that remove 802.11
> >> > headers before sending up the frame, right?).
> >> >
> >> >
> >> > Adding in Jesper; maybe he has some thoughts on this?
> >
> > Hi Jesper,
> >
> >>
> >> Today XDP assumes the frame is an Ethernet frame. With WiFi I guess
> >> this assumption change, right?
> >
> > yes correct, SoftMAC devices report 802.11 frames to the stack
> >
> >> I worry a bit about this, as XDP is all about performance, and I don't
> >> want to add performance regressions, by requiring all XDP programs or
> >> core-code to having to check-frame-type before proceeding. That said, I
> >> do think it is doable, without adding performance regressions.
> >>
> >> Option #1 is to move the check-frame-type to setup time. By either
> >> having frame-type be part of eBPF prog, or supply frame-type as option
> >> XDP attach call. And then reject attaching XDP prog to a device, where
> >> the expected frame-type does not match.
> >>
> >
> > I guess it will be enough to avoid loading a 'non-WiFi' bpf program on
> > a 802.11 netdevice (and vice versa). We could add a flag (or something
> > similar) in XDP_SETUP_PROG section of netdev_bpf data structure and
> > use ieee80211_ptr netdevice pointer in order to guarantee that the bpf
> > program will work on the expected 'frame-type'
>
> Yeah, a flag would be good; we've been discussing that for other XDP use
> cases; it's not a done deal yet, but I think it would be useful.
Do you think something wifi specific is ok (e.g bool wifi) or do you prefer
something more general (e.g u32 frame_type)?
> >
> >> Option#2, leave it up to eBPF-programmer if they want to add runtime
> >> checks. By extending xdp_rxq_info with frame-type (default to
> >> Ethernet), which allow the eBPF-programmer choose to write a generic
> >> XDP program that both work on Ethernet and WiFi, or skip-check as they
> >> know this will e.g. only run on Wifi. (Note xdp_rxq_info is static
> >> read-only info per RX-queue, will all Wifi frames have same frame-type?.
> >>
> >
> > 802.11 standards define three frame subtype (data, management and control).
> > Subtypes could be detected parsing 802.11 header
> >
> >>
> >> Also consider what happens in case of XDP_REDIRECT, from a Wifi NIC to
> >> an Ethernet NIC. It would of-cause be cool to get this working cross,
> >> Wifi-Ethernet.
> >>
> >
> > Very cool :) On tx side the driver will accept standard ethernet frames in
> > ndo_xdp_xmit pointer
>
> How do you envision that will work with drivers that build software
> 802.11 frames? The TX hook would have to be in mac80211 somewhere,
> wouldn't it?
In order to perform 802.3 --> 802.11 xdp forwarding my current idea is is to have
ndo_xdp_xmit pointer in mac80211 that will forward the frame to the low-level driver
(more or less what I did in the RFC series to upload the bpf program to mt76).
We will probably need to pass some info to the driver from mac80211 (e.g sequence
number or hw key idx to use)
>
> >> Option#3 is to say, Wifi XDP is so different that we should create a
> >> new (enum) bpf_prog_type. And then still see if we can leverage some
> >> of the same core-code (as long as it doesn't slowdown performance).
> >>
> >
> > Do you think that Option#3 will be more 'future-proof' respect to
> > Option#1?
>
> My feeling is that WiFi devices are not sufficiently different to
> warrant a whole new program type. We risk combinatorial explosion for
> all the stuff that is the same, but now need to be tested for two (or N)
> types...
Agree
Regards,
Lorenzo
>
> -Toke
Lorenzo Bianconi <[email protected]> writes:
>> Lorenzo Bianconi <[email protected]> writes:
>>
>> >> On Wed, 28 Nov 2018 13:36:26 +0100
>> >> Toke Høiland-Jørgensen <[email protected]> wrote:
>> >>
>> >> > Lorenzo Bianconi <[email protected]> writes:
>> >> >
>> >> > >> Lorenzo Bianconi <[email protected]> writes:
>> >> > >>
>> >> > >> > This series is intended as a playground to start experimenting/developing
>> >> > >> > with XDP/eBPF over WiFi and collect ideas/concerns about it.
>> >> > >> > Introduce XDP support to mt76x2e/mt76x0e drivers. Currently supported
>> >> > >> > actions are:
>> >> > >> > - XDP_PASS
>> >> > >> > - XDP_ABORTED
>> >> > >> > - XDP_DROP
>> >> > >> > Introduce ndo_bpf mac80211 callback in order to to load a bpf
>> >> > >> > program into low level driver XDP rx hook.
>> >> > >> > This series has been tested through a simple bpf program (available here:
>> >> > >> > https://github.com/LorenzoBianconi/bpf-workspace/tree/master/mt76_xdp_stats)
>> >> > >> > used to count frame types received by the device.
>> >> > >> > Possible eBPF use cases could be:
>> >> > >> > - implement new statistics through bpf maps
>> >> > >> > - implement fast packet filtering (e.g in monitor mode)
>> >> > >> > - ...
>> >> > >
>> >> > > Hi Kalle,
>> >> > >
>> >> > >>
>> >> > >> This is most likely a stupid question, but why do this in the driver and
>> >> > >> not in mac80211 so that all drivers could benefit from it? I guess there
>> >> > >> are reasons for that, I just can't figure that out.
>> >> >
>> >> > XDP achieves its speedup by running the eBPF program inside the driver
>> >> > NAPI loop, before the kernel even touches the data in any other capacity
>> >> > (and in particular, before it allocates an SKB). Which kinda means the
>> >> > hook needs to be in the driver... Could be a fallback in mac80211,
>> >> > though; although we'd have to figure out how that interacts with Generic
>> >> > XDP.
>> >> >
>> >> > > This is an early stage implementation, at this point I would collect
>> >> > > other people opinions/concerns about using bpf/xdp directly on 802.11
>> >> > > frames.
>> >> >
>> >> > Thanks for looking into this!
>> >> >
>> >> > I have two concerns with running XDP on 802.11 frames:
>> >> >
>> >> > 1. It makes it more difficult to add other XDP actions (such as
>> >> > REDIRECT), as the XDP program would then have to make sure that the
>> >> > outer packet headers are removed before, say, redirecting the packet
>> >> > out of an ethernet interface. Also, if we do add redirect, we would
>> >> > be bypassing mac80211 entirely; to what extent would that mess up
>> >> > internal state?
>> >> >
>> >> > 2. UI consistency; suddenly, the user needs to know which kind of
>> >> > frames to expect, and XDP program reuse becomes more difficult. This
>> >> > may be unavoidable given the nature of XDP, but some thought needs to
>> >> > go into this. Especially since we wouldn't necessarily be consistent
>> >> > between WiFi drivers (there are fullmac devices that remove 802.11
>> >> > headers before sending up the frame, right?).
>> >> >
>> >> >
>> >> > Adding in Jesper; maybe he has some thoughts on this?
>> >
>> > Hi Jesper,
>> >
>> >>
>> >> Today XDP assumes the frame is an Ethernet frame. With WiFi I guess
>> >> this assumption change, right?
>> >
>> > yes correct, SoftMAC devices report 802.11 frames to the stack
>> >
>> >> I worry a bit about this, as XDP is all about performance, and I don't
>> >> want to add performance regressions, by requiring all XDP programs or
>> >> core-code to having to check-frame-type before proceeding. That said, I
>> >> do think it is doable, without adding performance regressions.
>> >>
>> >> Option #1 is to move the check-frame-type to setup time. By either
>> >> having frame-type be part of eBPF prog, or supply frame-type as option
>> >> XDP attach call. And then reject attaching XDP prog to a device, where
>> >> the expected frame-type does not match.
>> >>
>> >
>> > I guess it will be enough to avoid loading a 'non-WiFi' bpf program on
>> > a 802.11 netdevice (and vice versa). We could add a flag (or something
>> > similar) in XDP_SETUP_PROG section of netdev_bpf data structure and
>> > use ieee80211_ptr netdevice pointer in order to guarantee that the bpf
>> > program will work on the expected 'frame-type'
>>
>> Yeah, a flag would be good; we've been discussing that for other XDP use
>> cases; it's not a done deal yet, but I think it would be useful.
>
> Do you think something wifi specific is ok (e.g bool wifi) or do you prefer
> something more general (e.g u32 frame_type)?
My thought was a feature flag where the program can set a flag which
means "I expect 802.11 frames", and the driver can set a flag saying "I
emit 802.11 frames", and if those two flags don't match, the verifier
can refuse to load the program. This would not be fool-proof (an XDP
program can still corrupt things if written incorrectly), but it would
at least protect against the most obvious mistakes.
>> >> Option#2, leave it up to eBPF-programmer if they want to add runtime
>> >> checks. By extending xdp_rxq_info with frame-type (default to
>> >> Ethernet), which allow the eBPF-programmer choose to write a generic
>> >> XDP program that both work on Ethernet and WiFi, or skip-check as they
>> >> know this will e.g. only run on Wifi. (Note xdp_rxq_info is static
>> >> read-only info per RX-queue, will all Wifi frames have same frame-type?.
>> >>
>> >
>> > 802.11 standards define three frame subtype (data, management and control).
>> > Subtypes could be detected parsing 802.11 header
>> >
>> >>
>> >> Also consider what happens in case of XDP_REDIRECT, from a Wifi NIC to
>> >> an Ethernet NIC. It would of-cause be cool to get this working cross,
>> >> Wifi-Ethernet.
>> >>
>> >
>> > Very cool :) On tx side the driver will accept standard ethernet frames in
>> > ndo_xdp_xmit pointer
>>
>> How do you envision that will work with drivers that build software
>> 802.11 frames? The TX hook would have to be in mac80211 somewhere,
>> wouldn't it?
>
> In order to perform 802.3 --> 802.11 xdp forwarding my current idea is
> is to have ndo_xdp_xmit pointer in mac80211 that will forward the
> frame to the low-level driver (more or less what I did in the RFC
> series to upload the bpf program to mt76). We will probably need to
> pass some info to the driver from mac80211 (e.g sequence number or hw
> key idx to use)
So this means that the driver will need to do the 802.11 encapsulation?
I guess we could have a fallback implementation in mac80211; but there
is possibly quite a bit of refactoring needed to make the existing code
work without an skb. Also, we need to think about queueing; I'm not sure
it's a good idea to have redirected frames bypass the TXQs...
-Toke
> Lorenzo Bianconi <[email protected]> writes:
>
> >> Lorenzo Bianconi <[email protected]> writes:
> >>
> >> >> On Wed, 28 Nov 2018 13:36:26 +0100
> >> >> Toke H?iland-J?rgensen <[email protected]> wrote:
> >> >>
[...]
> >> >
> >> > I guess it will be enough to avoid loading a 'non-WiFi' bpf program on
> >> > a 802.11 netdevice (and vice versa). We could add a flag (or something
> >> > similar) in XDP_SETUP_PROG section of netdev_bpf data structure and
> >> > use ieee80211_ptr netdevice pointer in order to guarantee that the bpf
> >> > program will work on the expected 'frame-type'
> >>
> >> Yeah, a flag would be good; we've been discussing that for other XDP use
> >> cases; it's not a done deal yet, but I think it would be useful.
> >
> > Do you think something wifi specific is ok (e.g bool wifi) or do you prefer
> > something more general (e.g u32 frame_type)?
>
> My thought was a feature flag where the program can set a flag which
> means "I expect 802.11 frames", and the driver can set a flag saying "I
> emit 802.11 frames", and if those two flags don't match, the verifier
> can refuse to load the program. This would not be fool-proof (an XDP
> program can still corrupt things if written incorrectly), but it would
> at least protect against the most obvious mistakes.
I guess we can use iee80211_ptr in dev_xdp_install to double check if it is
allowed to upload a 802.11 (or 802.3) bpf program
>
> >> >> Option#2, leave it up to eBPF-programmer if they want to add runtime
> >> >> checks. By extending xdp_rxq_info with frame-type (default to
> >> >> Ethernet), which allow the eBPF-programmer choose to write a generic
> >> >> XDP program that both work on Ethernet and WiFi, or skip-check as they
> >> >> know this will e.g. only run on Wifi. (Note xdp_rxq_info is static
> >> >> read-only info per RX-queue, will all Wifi frames have same frame-type?.
> >> >>
> >> >
> >> > 802.11 standards define three frame subtype (data, management and control).
> >> > Subtypes could be detected parsing 802.11 header
> >> >
> >> >>
> >> >> Also consider what happens in case of XDP_REDIRECT, from a Wifi NIC to
> >> >> an Ethernet NIC. It would of-cause be cool to get this working cross,
> >> >> Wifi-Ethernet.
> >> >>
> >> >
> >> > Very cool :) On tx side the driver will accept standard ethernet frames in
> >> > ndo_xdp_xmit pointer
> >>
> >> How do you envision that will work with drivers that build software
> >> 802.11 frames? The TX hook would have to be in mac80211 somewhere,
> >> wouldn't it?
> >
> > In order to perform 802.3 --> 802.11 xdp forwarding my current idea is
> > is to have ndo_xdp_xmit pointer in mac80211 that will forward the
> > frame to the low-level driver (more or less what I did in the RFC
> > series to upload the bpf program to mt76). We will probably need to
> > pass some info to the driver from mac80211 (e.g sequence number or hw
> > key idx to use)
>
> So this means that the driver will need to do the 802.11 encapsulation?
> I guess we could have a fallback implementation in mac80211; but there
> is possibly quite a bit of refactoring needed to make the existing code
> work without an skb. Also, we need to think about queueing; I'm not sure
> it's a good idea to have redirected frames bypass the TXQs...
>
good point :)
Regards,
Lorenzo
> -Toke
Lorenzo Bianconi <[email protected]> writes:
>> Lorenzo Bianconi <[email protected]> writes:
>>
>> >> Lorenzo Bianconi <[email protected]> writes:
>> >>
>> >> >> On Wed, 28 Nov 2018 13:36:26 +0100
>> >> >> Toke Høiland-Jørgensen <[email protected]> wrote:
>> >> >>
>
> [...]
>
>> >> >
>> >> > I guess it will be enough to avoid loading a 'non-WiFi' bpf program on
>> >> > a 802.11 netdevice (and vice versa). We could add a flag (or something
>> >> > similar) in XDP_SETUP_PROG section of netdev_bpf data structure and
>> >> > use ieee80211_ptr netdevice pointer in order to guarantee that the bpf
>> >> > program will work on the expected 'frame-type'
>> >>
>> >> Yeah, a flag would be good; we've been discussing that for other XDP use
>> >> cases; it's not a done deal yet, but I think it would be useful.
>> >
>> > Do you think something wifi specific is ok (e.g bool wifi) or do you prefer
>> > something more general (e.g u32 frame_type)?
>>
>> My thought was a feature flag where the program can set a flag which
>> means "I expect 802.11 frames", and the driver can set a flag saying "I
>> emit 802.11 frames", and if those two flags don't match, the verifier
>> can refuse to load the program. This would not be fool-proof (an XDP
>> program can still corrupt things if written incorrectly), but it would
>> at least protect against the most obvious mistakes.
>
> I guess we can use iee80211_ptr in dev_xdp_install to double check if it is
> allowed to upload a 802.11 (or 802.3) bpf program
Yeah, I think it's more an issue of convincing the wider XDP community
that support for feature flags is in fact needed ;)
-Toke
Hi Toke, all,
Sorry I wasn't around for the discussion, I've been travelling and sick.
I'm picking this as an arbitrary point in the discussion to reply to,
hope you don't feel bad about that.
>> [A-MPDU reordering]
> In principle, all of this could be done. But we need to think carefully
> about what things it really makes sense to offload to XDP. In the
> general case, we will end up re-implementing all of mac80211 in eBPF,
> which is obviously not ideal.
Agree with this, completely. Mind you, it's not even simple to do that,
because each kind of hardware behaves differently. See below.
> For crypto in particular, I wonder if there is much of a speedup to be
> had if the crypto is in software anyway. Wouldn't that dominate the
> processing time?
Yeah, I'd also think the cost of skb allocation is dwarfed by the cost
of doing the crypto, but not really sure, maybe you have a fast special
AES instruction? :)
Anyway, here are my thoughts after reading most of the thread and having
discussed it with Lorenzo a while back and today:
1) I'm going to restrict most of the discussion to mac80211, but there
are interesting cases outside of it as well. Much of the same
arguments apply, though it means *more* feature combinatorial
explosion to be able to handle everything.
2) We don't yet have it (but it's been suggested for years and we might
also get it on Intel hardware eventually), but if we have TX and/or
RX 802.3/802.11 frame conversion offload then I think we have no
argument - XDP applies pure and simple as is in each direction you
have offloads for, unless you do something really stupid in HW like
still having to do reorder buffer or PN checking based on RX frame
metadata. I hope nobody does that, but then I suppose you could do it
before the XDP hooks w/o allocating SKBs.
3) TX from XDP (i.e. X -> wifi) could be implemented in mac80211, but is
subject to some rather iffy details, for example:
a) mac80211 may need more headroom than the input has, do we have to
copy the whole data? depending on xdp_mem_type I think we may need
that, rather than being able to make an skb with frag pages and
extra headroom? We don't know it was just a single page and I
guess we can't ref that page anyway, but maybe we could meddle
with the SKB frag data somehow to avoid that.
b) mac80211 may need to software encrypt - copy the whole data
c) lifetime issues - we need to call xdp_return_frame(), which I
guess we can tie to the skb we'd create anyway [2]
(and IMHO we need to allocate an skb anyway, due to TXQs like Toke
mentioned and much other possible software handling)
4) XDP during RX - well, that's _mostly_ what this thread has been
about, but really I think it's a can of worms. Consider:
a) drivers may or may not do SW decryption
b) drivers may or may not do PN checking - so your XDP program might
end up having to do that or risk security/replay bugs!
(which, btw, you can only do after reordering so see below)
c) drivers may or may not do A-MSDU deaggregation, do you[1] really
want to handle that in XDP? You could argue outer code should walk
over the subframes, but then dropping becomes hard - and this
really should only happen after a) and b)
Oh, and IIRC there are cases that use A-MSDUs for single frames
just to encapsulate different MAC addresses.
Speaking of which, you have to validate the inner MAC addresses
(but it's not a trivial comparison) or risk security bugs again.
d) drivers may or may not do A-MPDU reordering (this was discussed
earlier in the thread), which means even if you drop a frame you
haven't just affected that connection but potentially stalled
everything until the reorder buffer times out (~100ms), or we need
a way for mac80211 to insert a fake skb into the reorder buffer at
that spot (but then you allocate an skb again and high-speed
traffic where you'd care most is always in an A-MPDU session)
e) speaking of which, Intel's device has hardware assist things for
reordering, but not fully offloaded reordering, so you might end
up with hardware-specific ways of doing this in XDP?
f) Some data frames are used internally in the stack, e.g. for TDLS.
g) data frames may also affect the powersave state of a station, so
if you have e.g. ath9k and the station sends a frame to wake up,
but you decide you XDP_DROP it, the powersave state will get
messed up, unless ... special handling (either mac80211 or the
bpf program) again. Also, U-APSD.
h) There are also simple things like QoS/non-QoS that add to the
combinatorial explosion of things you have to handle
i) data frames could be sent by anyone, not just STAs connected to
your network, so I guess you'd have to filter that and mac80211
would have to expose a station table lookup helper or something?
Alternatively shuffle the code around so mac80211 does that first,
but then we already have an SKB anyway since we got it from the
driver, and you want this stuff to run in the driver.
j) in AP mode, if data frames are sent by somebody not belonging to
the network, we should send an event to hostapd so it can tell
them they're not on the network
k) some drivers implement client-side powersave in mac80211 and need
the more-data bit even if you were to XDP_DROP a frame. I hate
this code with a passion, but it's there for now.
m) did somebody mention fragmentation yet? and the special security
handling for CCMP/GCMP there?
n) I guess checking for port authorized/not would be one of the more
trivial things, just adds another helper/bit/state/something.
o) not supported yet by anyone (well, mac80211 at least), but the
standard is adding provisions to leave out the SNAP/LLC header,
which requires more out-of-band data (i.e. data not directly in
the frame)
p) there's also iwlwifi with it's multiqueue RX, but let's not go
there now
[Toke, do you still think it's not sufficiently different? ;-P]
Of course with XDP anyone can shoot themselves in the foot easily, but I
also think we shouldn't make it too easy to do things like XDP_REDIRECT
on a frame that hasn't even had its security properties validated. It'd
work, but an attacker's frame could be accepted as well. We have enough
security bugs as is, I'm not sure I'd want that extra surface...
Now, of course you can pick each one of these points and say "but that's
not relevant for my ${favourite_hw}", which is true. Each hardware
(family) will likely make *most* of fixed or not relevant. Restrict
sufficiently though and you'll end up with *exactly* ${favourite_hw}
doing it the way you want to assume it is like.
E.g. let's say you require PN checking in hardware - maybe 2 or 3
drivers (I'm thinking ath10k, maybe mt76, maybe some ralink family)?
Or let's say you want A-MPDU reordering offload to not have to worry
about that and replays and all - I'm guessing only ath10k now and that
perhaps even only in 802.3 mode?
Or let's say you have enough filtering to not have to worry about 4j),
what if you enable monitor mode at the same time? Can you tell the
difference in the driver before calling the XDP program?
As fun as it sounds (on paper) to have XDP available for wifi, I'm not
sure we really should do that. We'd have to handle many of the issues
outlined above (which is at minimum a LOT of helper code that each at
least somewhat secure and portable XDP-wifi program would have to call),
and it looks like in the future we'll be moving more towards offloaded
802.11/.3 frame conversion anyway, at which point it all becomes more or
less useless.
So I see a few ways this might go:
A) We basically allow everything, and let the XDP program authors
figure it out. They'll eventually learn to handle the whole mac80211
datapath (items in 4)), and will add a TON of helper code for it.
B) We try to foresee all the needed helper code, and essentially build
a parallel mac80211 datapath equivalent to the one today, but
without SKBs.
C) We rebuild the mac80211 datapath w/o SKBs so we can put XDP behind
it. Ugh.
D) We restrict it to ~1 driver that does most of the important things
anyway in the device, thus hardcoding most of the variables, but
then we expand out to A) or B)
E) Like D, but then that device learns to do 802.3/.11 conversion.
Oops.
F) Like D, but then that *driver* learns to do 802.3/.11 conversion.
Also oops.
Maybe F) kind of has a good direction in it? If the device does enough
that we don't have to worry about most things, then - provided we
actually implement the code to allow 802.3/.11 conversion offload in
mac80211 - the driver can probably easily do that itself rather than
punting to mac80211's fast-RX/fast-TX, which would likely be used?
Or perhaps that means we should teach mac80211 fast-RX to be skb-less,
or something like that? But I think that doesn't solve the reorder
buffer issue either.
(Let's also not forget that the original use case was something around
defective HW filters for BSSIDs - IMHO that should just be *hardcoded*
into the driver anyway because all other frames are useless to you,
except perhaps if also monitor mode is active but then you need an SKB!)
Ok, this got longer than I wanted...
johannes
[1] generally speaking, no particular "you" intended
[2] btw, i40e_clean_xdp_tx_buffer() looks kinda weird? it returns first
and then DMA-unmaps, you'd think it should be done the other way around?
Hi Johannes
I think your email can be basically summed up to:
> [ ... ] but really I think it's a can of worms.
...right? :)
I sort of had a feeling it would be, but thank you for spelling out in
excruciating detail why that is so.
Given this, I think I agree that it's not worth it for now, and we
should hold off on adding XDP support until we have 802.3/.11 conversion
offload working... Which I think is also where you ended up? :)
-Toke
On Mon, 2018-12-03 at 21:36 +0200, Toke Høiland-Jørgensen wrote:
> Hi Johannes
>
> I think your email can be basically summed up to:
>
> > [ ... ] but really I think it's a can of worms.
>
> ...right? :)
Heh, yeah :)
> I sort of had a feeling it would be, but thank you for spelling out in
> excruciating detail why that is so.
:-)
> Given this, I think I agree that it's not worth it for now, and we
> should hold off on adding XDP support until we have 802.3/.11 conversion
> offload working... Which I think is also where you ended up? :)
That case is at least easy, yeah. And it seems kinda likely that we'll
end up with that in all well-maintained drivers in the relatively near
future anyway?
BTW, in a sense I still kind of want to add eBPF to the mac80211 ingress
path, just not in the XDP sense. For example, I had a proposal a while
ago to add a filter to the monitor mode RX path(s) in eBPF; I still
think that's useful.
I also think it may be useful to put eBPF programs into per-netdev
ingress path, in order to e.g. collect statistics, rather than hard-
coding all kinds of statistics into mac80211.
All of these things I consider absolutely useful and helpful. I like
eBPF and the flexibility it affords. I just really don't think we should
call it XDP or let it do similar things to XDP like dropping or
redirecting frames.
johannes
Johannes Berg <[email protected]> writes:
> On Mon, 2018-12-03 at 21:36 +0200, Toke Høiland-Jørgensen wrote:
>> Hi Johannes
>>
>> I think your email can be basically summed up to:
>>
>> > [ ... ] but really I think it's a can of worms.
>>
>> ...right? :)
>
> Heh, yeah :)
>
>> I sort of had a feeling it would be, but thank you for spelling out in
>> excruciating detail why that is so.
>
> :-)
>
>> Given this, I think I agree that it's not worth it for now, and we
>> should hold off on adding XDP support until we have 802.3/.11
>> conversion offload working... Which I think is also where you ended
>> up? :)
>
> That case is at least easy, yeah. And it seems kinda likely that we'll
> end up with that in all well-maintained drivers in the relatively near
> future anyway?
Right; you probably know that better than me :)
> BTW, in a sense I still kind of want to add eBPF to the mac80211 ingress
> path, just not in the XDP sense. For example, I had a proposal a while
> ago to add a filter to the monitor mode RX path(s) in eBPF; I still
> think that's useful.
>
> I also think it may be useful to put eBPF programs into per-netdev
> ingress path, in order to e.g. collect statistics, rather than hard-
> coding all kinds of statistics into mac80211.
>
> All of these things I consider absolutely useful and helpful. I like
> eBPF and the flexibility it affords. I just really don't think we should
> call it XDP or let it do similar things to XDP like dropping or
> redirecting frames.
Absolutely; totally on board with that!
-Toke
> On Mon, 2018-12-03 at 21:36 +0200, Toke Høiland-Jørgensen wrote:
> > Hi Johannes
> >
> > I think your email can be basically summed up to:
> >
> > > [ ... ] but really I think it's a can of worms.
> >
> > ...right? :)
>
> Heh, yeah :)
>
> > I sort of had a feeling it would be, but thank you for spelling out in
> > excruciating detail why that is so.
>
> :-)
>
> > Given this, I think I agree that it's not worth it for now, and we
> > should hold off on adding XDP support until we have 802.3/.11 conversion
> > offload working... Which I think is also where you ended up? :)
>
> That case is at least easy, yeah. And it seems kinda likely that we'll
> end up with that in all well-maintained drivers in the relatively near
> future anyway?
Hi all,
thanks for sharing your thoughts and concerns, as already stated the
main goal of this series
is get feedbacks and/or blocking points to start experimenting with
eBPF over 802.11 devices.
Reviewing the points indicated by Johannes, 802.11 protocol is too
complicated to be managed without skb
allocation. I agree that for the moment XDP/eBPF can be useful for
devices that supports hw offloading
(or FullMac devices?) since all the tricky aspects are already managed
in the firmware and we can take care of XDP stuff.
Probably in the near future we will respin this argument :)
>
> BTW, in a sense I still kind of want to add eBPF to the mac80211 ingress
> path, just not in the XDP sense. For example, I had a proposal a while
> ago to add a filter to the monitor mode RX path(s) in eBPF; I still
> think that's useful.
>
> I also think it may be useful to put eBPF programs into per-netdev
> ingress path, in order to e.g. collect statistics, rather than hard-
> coding all kinds of statistics into mac80211.
>
> All of these things I consider absolutely useful and helpful. I like
> eBPF and the flexibility it affords. I just really don't think we should
> call it XDP or let it do similar things to XDP like dropping or
> redirecting frames.
>
+1 :)
Regards,
Lorenzo
> johannes
>