Received: by 2002:a05:6359:6284:b0:131:369:b2a3 with SMTP id se4csp442225rwb; Fri, 4 Aug 2023 15:44:19 -0700 (PDT) X-Google-Smtp-Source: AGHT+IEKx29wm2eiSqH3NUmd9r6jqwJ1meFTQPIelRV9bnPACy6XJleQJdPhHKYDyZVrJaalFVHu X-Received: by 2002:ac2:4ece:0:b0:4f8:710c:e2c7 with SMTP id p14-20020ac24ece000000b004f8710ce2c7mr1806410lfr.33.1691189059293; Fri, 04 Aug 2023 15:44:19 -0700 (PDT) ARC-Seal: i=1; a=rsa-sha256; t=1691189059; cv=none; d=google.com; s=arc-20160816; b=PHFPMsM1DWGKQl9EadYOKvOaybTSOcxct43+eQwJyGlEmlNTBQrZmugnUY//Q5qReq t+3T/0PlvzLXaLBnInNfnXWnmxcHYSymP35/jDa1g/Wl4lb1TSXr7vzq0NGzbDvR5kzd MNc/gXRTKjj0BSFb5ym8HR1faJN10cQgwCNj25DV/6RcXnKB2I1ZPnDzU647R4kdSS8+ 7tgQMU+nvt+ATx/gbFpiRQFwArZpX+vu6DoK8VMaDQj2ODimvuoUHw/kC1XB5mFSHmG+ 6xkXlu6eBknJ3tIv/zQJGc+o9dflQQ4N8u2JJ+x0VTFeLWTeb8dTfz18LzIn2gCk0fUI 3zgg== ARC-Message-Signature: i=1; a=rsa-sha256; c=relaxed/relaxed; d=google.com; s=arc-20160816; h=list-id:precedence:content-transfer-encoding:mime-version :message-id:date:subject:to:from:dkim-signature; bh=GHueQwLIWCEz3XlJcUxmIZmOh2PU2xR1L0WHNOS+ItE=; fh=H8TA95lOIGdTfjXXZfXC1EhiE8tIMz6S6lTo+yfm7hw=; b=y5wMnEro13VI9kq44VHOyHTIibN5WF/DYKnVF02g2rYeWPcgiStnqAzNH51yffIVTe AnPjeqf5Db8jPnOzn6AL1QMvLG6uBI4xJDzQ8yX+eTHGoHQokIh5U6O/14+MtC3rnTjq rTZY8Ul+h4LaolubxFh9j2i5AADf+vY4GFrhnVizwyQz79xRxl2wmiEXLB/JZaEvZ6p0 1O6LJJBZT++TqsOqSZxBMtmBwxK3PqHukRq51KszGaKTX4W8zgIZvKveb4GtsR/YVUXd j/tx8LoeAyCcTOaFVq+ge1K7bcT2fC+YQPKRGayZa+Wy/rJygCAwE4TcXEHKP8ubtakO 8Pug== ARC-Authentication-Results: i=1; mx.google.com; dkim=pass header.i=@gmail.com header.s=20221208 header.b=NubayiF9; spf=pass (google.com: domain of linux-kernel-owner@vger.kernel.org designates 2620:137:e000::1:20 as permitted sender) smtp.mailfrom=linux-kernel-owner@vger.kernel.org; dmarc=pass (p=NONE sp=QUARANTINE dis=NONE) header.from=gmail.com Return-Path: Received: from out1.vger.email (out1.vger.email. [2620:137:e000::1:20]) by mx.google.com with ESMTP id q7-20020aa7d447000000b00522567926c8si2104585edr.11.2023.08.04.15.43.55; Fri, 04 Aug 2023 15:44:19 -0700 (PDT) Received-SPF: pass (google.com: domain of linux-kernel-owner@vger.kernel.org designates 2620:137:e000::1:20 as permitted sender) client-ip=2620:137:e000::1:20; Authentication-Results: mx.google.com; dkim=pass header.i=@gmail.com header.s=20221208 header.b=NubayiF9; spf=pass (google.com: domain of linux-kernel-owner@vger.kernel.org designates 2620:137:e000::1:20 as permitted sender) smtp.mailfrom=linux-kernel-owner@vger.kernel.org; dmarc=pass (p=NONE sp=QUARANTINE dis=NONE) header.from=gmail.com Received: (majordomo@vger.kernel.org) by vger.kernel.org via listexpand id S229729AbjHDVa5 (ORCPT + 99 others); Fri, 4 Aug 2023 17:30:57 -0400 Received: from lindbergh.monkeyblade.net ([23.128.96.19]:45394 "EHLO lindbergh.monkeyblade.net" rhost-flags-OK-OK-OK-OK) by vger.kernel.org with ESMTP id S230211AbjHDVae (ORCPT ); Fri, 4 Aug 2023 17:30:34 -0400 Received: from mail-ot1-x336.google.com (mail-ot1-x336.google.com [IPv6:2607:f8b0:4864:20::336]) by lindbergh.monkeyblade.net (Postfix) with ESMTPS id 65BF04C04; Fri, 4 Aug 2023 14:30:02 -0700 (PDT) Received: by mail-ot1-x336.google.com with SMTP id 46e09a7af769-6bcb15aa074so1742342a34.0; Fri, 04 Aug 2023 14:30:02 -0700 (PDT) DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=gmail.com; s=20221208; t=1691184601; x=1691789401; h=content-transfer-encoding:mime-version:message-id:date:subject:to :from:from:to:cc:subject:date:message-id:reply-to; bh=GHueQwLIWCEz3XlJcUxmIZmOh2PU2xR1L0WHNOS+ItE=; b=NubayiF9StvSreesK/se9xgtK3Pj1akbCsTAN1F1SqvLsIuHg8gPSTuRlv9zfTbwln Ei4zEyKppfLO3i1hCk9UD/vA0X9pjgK+sRuD4UPJJMjRr/mf4MfvtSiYyovbRSgP4wj+ l7icgc8fH3KrL5PJ2kq5VleKHu9M+3agQdrXmPAz05wQjfOMJSvT7e4LoBX1FXj68w1s 97CeRGH7CWLuESRUNAGYgf/h/GPSseyUK+3WrvblYKXbT/AMBp5DR/Ahk1YazZ1kiMXx o43uB2S7e+zSpFjfkr4/02Avb6eCyVlJujjNxvAxwZvErB4XknSqv4M63PH56c6vQSaT I+MQ== X-Google-DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=1e100.net; s=20221208; t=1691184601; x=1691789401; h=content-transfer-encoding:mime-version:message-id:date:subject:to :from:x-gm-message-state:from:to:cc:subject:date:message-id:reply-to; bh=GHueQwLIWCEz3XlJcUxmIZmOh2PU2xR1L0WHNOS+ItE=; b=OXfqErhVQzPiNAwxNZkdo1qkll6JdUF28Mdt9Dcw+rNXjZofD1ThzdIsMlnvdJDIl0 LGzoKqYg5+ocCJD0RRmk5bLSEQtuzGYA8apoxeI8e0x9BGmrWbONImaqQrlLnoH5qPmZ k/PbhVwzRa0vI3nBXqXKrlMhEUp08WjUS+cGeRN4gIhjgcixTtIQhFllYGqGSkOQWcS4 J+v0l7a4HWOmZTx/fR5f4L8zomSd8I5lUBYPsDiBec5xNWGKfe0Ut60X/ZMib0Ilv0JB PYfs90KdET2ly6iwhx9VPG8xMZoby/GyFCpBrvxuwFQfbn1ivyXkoQZJ8ydWuqklaCXW /wUg== X-Gm-Message-State: AOJu0Yw2FxXm+lrQ0dgPePoGE/kTH3UI9gPvNHG8luakEsNAjjM5kBIj JmAolShJmnuBHt6y9LVdBmo= X-Received: by 2002:a05:6830:4101:b0:6bb:132f:a785 with SMTP id w1-20020a056830410100b006bb132fa785mr769625ott.10.1691184601561; Fri, 04 Aug 2023 14:30:01 -0700 (PDT) Received: from localhost.localdomain (ip98-167-164-238.ph.ph.cox.net. [98.167.164.238]) by smtp.gmail.com with ESMTPSA id e20-20020a05683013d400b006b9d21100d0sm1645420otq.64.2023.08.04.14.29.59 (version=TLS1_3 cipher=TLS_AES_256_GCM_SHA384 bits=256/256); Fri, 04 Aug 2023 14:30:01 -0700 (PDT) From: Matthew Cover X-Google-Original-From: Matthew Cover To: Michael Chan , "David S. Miller" , Eric Dumazet , Jakub Kicinski , Paolo Abeni , Richard Cochran , Alexei Starovoitov , Daniel Borkmann , Jesper Dangaard Brouer , John Fastabend , Matthew Cover , linux-kernel@vger.kernel.org, netdev@vger.kernel.org, bpf@vger.kernel.org Subject: [PATCH net-next] Add bnxt_netlink to facilitate representor pair configurations. Date: Fri, 4 Aug 2023 14:29:54 -0700 Message-Id: <20230804212954.98868-1-matthew.cover@stackpath.com> X-Mailer: git-send-email 2.24.3 (Apple Git-128) MIME-Version: 1.0 Content-Transfer-Encoding: 8bit X-Spam-Status: No, score=-2.1 required=5.0 tests=BAYES_00,DKIM_SIGNED, DKIM_VALID,DKIM_VALID_AU,DKIM_VALID_EF,FREEMAIL_FROM, RCVD_IN_DNSWL_NONE,SPF_HELO_NONE,SPF_PASS autolearn=ham autolearn_force=no version=3.4.6 X-Spam-Checker-Version: SpamAssassin 3.4.6 (2021-04-09) on lindbergh.monkeyblade.net Precedence: bulk List-ID: X-Mailing-List: linux-kernel@vger.kernel.org To leverage the SmartNIC capabilities available in Broadcom NetXtreme-C/E ethernet devices, representor pairs must be configured via bnxt-ctl. Without bnxt_netlink as a registered family, bnxt-ctl errors as seen below, even for the most basic of subcommands. $ sudo bnxt-ctl show-pair Error: Failed to resolve family for name: bnxt_netlink, Cannot allocate memory With this patch, bnxt-ctl functions as expected providing display of and changes to representor pair configurations. $ sudo bnxt-ctl show-pair Representor Pair[4]: interface: enp65s0f0 name: 0000:06:00.0 state: down member(a): Linux PF index 2 Firmware function id: 0x3 ... This patch is a minimally modified port of the bnxt_netlink code found in out-of-tree bnxt_en-1.10.2-218.1.182.15.tar.gz. The out-of-tree driver contains both the GPL-2.0 in the file COPYING and a GPL comment in each source file. Signed-off-by: Matthew Cover --- drivers/net/ethernet/broadcom/bnxt/Makefile | 2 +- drivers/net/ethernet/broadcom/bnxt/bnxt.c | 6 + drivers/net/ethernet/broadcom/bnxt/bnxt_netlink.c | 231 ++++++++++++++++++++++ drivers/net/ethernet/broadcom/bnxt/bnxt_netlink.h | 41 ++++ 4 files changed, 279 insertions(+), 1 deletion(-) create mode 100644 drivers/net/ethernet/broadcom/bnxt/bnxt_netlink.c create mode 100644 drivers/net/ethernet/broadcom/bnxt/bnxt_netlink.h diff --git a/drivers/net/ethernet/broadcom/bnxt/Makefile b/drivers/net/ethernet/broadcom/bnxt/Makefile index 2bc2b70..31e154f 100644 --- a/drivers/net/ethernet/broadcom/bnxt/Makefile +++ b/drivers/net/ethernet/broadcom/bnxt/Makefile @@ -1,6 +1,6 @@ # SPDX-License-Identifier: GPL-2.0-only obj-$(CONFIG_BNXT) += bnxt_en.o -bnxt_en-y := bnxt.o bnxt_hwrm.o bnxt_sriov.o bnxt_ethtool.o bnxt_dcb.o bnxt_ulp.o bnxt_xdp.o bnxt_ptp.o bnxt_vfr.o bnxt_devlink.o bnxt_dim.o bnxt_coredump.o +bnxt_en-y := bnxt.o bnxt_hwrm.o bnxt_sriov.o bnxt_ethtool.o bnxt_dcb.o bnxt_ulp.o bnxt_xdp.o bnxt_ptp.o bnxt_vfr.o bnxt_devlink.o bnxt_netlink.o bnxt_dim.o bnxt_coredump.o bnxt_en-$(CONFIG_BNXT_FLOWER_OFFLOAD) += bnxt_tc.o bnxt_en-$(CONFIG_DEBUG_FS) += bnxt_debugfs.o diff --git a/drivers/net/ethernet/broadcom/bnxt/bnxt.c b/drivers/net/ethernet/broadcom/bnxt/bnxt.c index 6a643aa..a33c7b6 100644 --- a/drivers/net/ethernet/broadcom/bnxt/bnxt.c +++ b/drivers/net/ethernet/broadcom/bnxt/bnxt.c @@ -67,6 +67,7 @@ #include "bnxt_dcb.h" #include "bnxt_xdp.h" #include "bnxt_ptp.h" +#include "bnxt_netlink.h" #include "bnxt_vfr.h" #include "bnxt_tc.h" #include "bnxt_devlink.h" @@ -14118,6 +14119,10 @@ static int __init bnxt_init(void) { int err; + err = bnxt_nl_register(); + if (err) + pr_info("%s : failed to load\n", BNXT_NL_NAME); + bnxt_debug_init(); err = pci_register_driver(&bnxt_pci_driver); if (err) { @@ -14130,6 +14135,7 @@ static int __init bnxt_init(void) static void __exit bnxt_exit(void) { + bnxt_nl_unregister(); pci_unregister_driver(&bnxt_pci_driver); if (bnxt_pf_wq) destroy_workqueue(bnxt_pf_wq); diff --git a/drivers/net/ethernet/broadcom/bnxt/bnxt_netlink.c b/drivers/net/ethernet/broadcom/bnxt/bnxt_netlink.c new file mode 100644 index 0000000..48c1357 --- /dev/null +++ b/drivers/net/ethernet/broadcom/bnxt/bnxt_netlink.c @@ -0,0 +1,231 @@ +/* Broadcom NetXtreme-C/E network driver. + * + * Copyright (c) 2014-2016 Broadcom Corporation + * Copyright (c) 2016-2017 Broadcom Limited + * + * This program is free software; you can redistribute it and/or modify + * it under the terms of the GNU General Public License as published by + * the Free Software Foundation. + */ +#include +#include +#include "bnxt_hsi.h" +#include "bnxt_netlink.h" +#include "bnxt.h" +#include "bnxt_hwrm.h" + +/* attribute policy */ +static struct nla_policy bnxt_netlink_policy[BNXT_NUM_ATTRS] = { + [BNXT_ATTR_PID] = { .type = NLA_U32 }, + [BNXT_ATTR_IF_INDEX] = { .type = NLA_U32 }, + [BNXT_ATTR_REQUEST] = { .type = NLA_BINARY }, + [BNXT_ATTR_RESPONSE] = { .type = NLA_BINARY }, +}; + +static struct genl_family bnxt_netlink_family; + +static int bnxt_parse_attrs(struct nlattr **a, struct bnxt **bp, + struct net_device **dev) +{ + pid_t pid; + struct net *ns = NULL; + const char *drivername; + + if (!a[BNXT_ATTR_PID]) { + netdev_err(*dev, "No process ID specified\n"); + goto err_inval; + } + pid = nla_get_u32(a[BNXT_ATTR_PID]); + ns = get_net_ns_by_pid(pid); + if (IS_ERR(ns)) { + netdev_err(*dev, "Invalid net namespace for pid %d (err: %ld)\n", + pid, PTR_ERR(ns)); + goto err_inval; + } + + if (!a[BNXT_ATTR_IF_INDEX]) { + netdev_err(*dev, "No interface index specified\n"); + goto err_inval; + } + *dev = dev_get_by_index(ns, nla_get_u32(a[BNXT_ATTR_IF_INDEX])); + + put_net(ns); + ns = NULL; + if (!*dev) { + netdev_err(*dev, "Invalid network interface index %d (err: %ld)\n", + nla_get_u32(a[BNXT_ATTR_IF_INDEX]), PTR_ERR(ns)); + goto err_inval; + } + if (!(*dev)->dev.parent || !(*dev)->dev.parent->driver || + !(*dev)->dev.parent->driver->name) { + netdev_err(*dev, "Unable to get driver name for device %s\n", + (*dev)->name); + goto err_inval; + } + drivername = (*dev)->dev.parent->driver->name; + if (strcmp(drivername, DRV_MODULE_NAME)) { + netdev_err(*dev, "Device %s (%s) is not a %s device!\n", + (*dev)->name, drivername, DRV_MODULE_NAME); + goto err_inval; + } + *bp = netdev_priv(*dev); + if (!*bp) { + netdev_warn((*bp)->dev, "No private data\n"); + goto err_inval; + } + + return 0; + +err_inval: + if (ns && !IS_ERR(ns)) + put_net(ns); + return -EINVAL; +} + +/* handler */ +static int bnxt_netlink_hwrm(struct sk_buff *skb, struct genl_info *info) +{ + struct nlattr **a = info->attrs; + struct net_device *dev = NULL; + struct sk_buff *reply = NULL; + struct input *req, *nl_req; + struct bnxt *bp = NULL; + struct output *resp; + int len, rc; + void *hdr; + + rc = bnxt_parse_attrs(a, &bp, &dev); + if (rc) + goto err_rc; + + if (!bp) { + rc = -EINVAL; + goto err_rc; + } + + if (!bp->hwrm_dma_pool) { + netdev_warn(bp->dev, "HWRM interface not currently available on %s\n", + dev->name); + rc = -EINVAL; + goto err_rc; + } + + if (!a[BNXT_ATTR_REQUEST]) { + netdev_warn(bp->dev, "No request specified\n"); + rc = -EINVAL; + goto err_rc; + } + len = nla_len(a[BNXT_ATTR_REQUEST]); + nl_req = nla_data(a[BNXT_ATTR_REQUEST]); + + reply = genlmsg_new(PAGE_SIZE, GFP_KERNEL); + if (!reply) { + netdev_warn(bp->dev, "Error: genlmsg_new failed\n"); + rc = -ENOMEM; + goto err_rc; + } + + rc = hwrm_req_init(bp, req, nl_req->req_type); + if (rc) + goto err_rc; + + rc = hwrm_req_replace(bp, req, nl_req, len); + if (rc) + goto err_rc; + + resp = hwrm_req_hold(bp, req); + rc = hwrm_req_send_silent(bp, req); + if (rc) { + /* + * Indicate success for the hwrm transport, while letting + * the hwrm error be passed back to the netlink caller in + * the response message. Caller is responsible for handling + * any errors. + * + * no kernel warnings are logged in this case. + */ + rc = 0; + } + hdr = genlmsg_put_reply(reply, info, &bnxt_netlink_family, 0, + BNXT_CMD_HWRM); + if (nla_put(reply, BNXT_ATTR_RESPONSE, resp->resp_len, resp)) { + netdev_warn(bp->dev, "No space for response attribte\n"); + hwrm_req_drop(bp, req); + rc = -ENOMEM; + goto err_rc; + } + genlmsg_end(reply, hdr); + hwrm_req_drop(bp, req); + + dev_put(dev); + dev = NULL; + + return genlmsg_reply(reply, info); + +err_rc: + if (reply && !IS_ERR(reply)) + kfree_skb(reply); + if (dev && !IS_ERR(dev)) + dev_put(dev); + + if (bp) + netdev_warn(bp->dev, "returning with error code %d\n", rc); + + return rc; +} + +/* handlers */ +#if defined(HAVE_GENL_REG_OPS_GRPS) || !defined(HAVE_GENL_REG_FAMILY_WITH_OPS) +static const struct genl_ops bnxt_netlink_ops[] = { +#else +static struct genl_ops bnxt_netlink_ops[] = { +#endif + { + .cmd = BNXT_CMD_HWRM, + .flags = GENL_ADMIN_PERM, /* Req's CAP_NET_ADMIN privilege */ +#ifndef HAVE_GENL_POLICY + .policy = bnxt_netlink_policy, +#endif + .doit = bnxt_netlink_hwrm, + .dumpit = NULL, + }, +}; + +/* family definition */ +static struct genl_family bnxt_netlink_family = { +#ifdef HAVE_GENL_ID_GENERATE + .id = GENL_ID_GENERATE, +#endif + .hdrsize = 0, + .name = BNXT_NL_NAME, + .version = BNXT_NL_VER, + .maxattr = BNXT_NUM_ATTRS, +#ifdef HAVE_GENL_POLICY + .policy = bnxt_netlink_policy, +#endif +#ifndef HAVE_GENL_REG_FAMILY_WITH_OPS + .ops = bnxt_netlink_ops, + .n_ops = ARRAY_SIZE(bnxt_netlink_ops) +#endif +}; + +int bnxt_nl_register(void) +{ +#ifndef HAVE_GENL_REG_FAMILY_WITH_OPS + return genl_register_family(&bnxt_netlink_family); +#elif defined(HAVE_GENL_REG_OPS_GRPS) + return genl_register_family_with_ops(&bnxt_netlink_family, + bnxt_netlink_ops); +#else + return genl_register_family_with_ops(&bnxt_netlink_family, + bnxt_netlink_ops, + ARRAY_SIZE(bnxt_netlink_ops)); +#endif + + return 0; +} + +int bnxt_nl_unregister(void) +{ + return genl_unregister_family(&bnxt_netlink_family); +} diff --git a/drivers/net/ethernet/broadcom/bnxt/bnxt_netlink.h b/drivers/net/ethernet/broadcom/bnxt/bnxt_netlink.h new file mode 100644 index 0000000..6939cd4 --- /dev/null +++ b/drivers/net/ethernet/broadcom/bnxt/bnxt_netlink.h @@ -0,0 +1,41 @@ +/* Broadcom NetXtreme-C/E network driver. + * + * Copyright (c) 2014-2016 Broadcom Corporation + * Copyright (c) 2016-2017 Broadcom Limited + * + * This program is free software; you can redistribute it and/or modify + * it under the terms of the GNU General Public License as published by + * the Free Software Foundation. + */ +#ifndef __BNXT_NETLINK_H__ +#define __BNXT_NETLINK_H__ + +#include +#include + +/* commands */ +enum { + BNXT_CMD_UNSPEC, + BNXT_CMD_HWRM, + BNXT_NUM_CMDS +}; +#define BNXT_CMD_MAX (BNXT_NUM_CMDS - 1) + +/* attributes */ +enum { + BNXT_ATTR_UNSPEC, + BNXT_ATTR_PID, + BNXT_ATTR_IF_INDEX, + BNXT_ATTR_REQUEST, + BNXT_ATTR_RESPONSE, + BNXT_NUM_ATTRS +}; +#define BNXT_ATTR_MAX (BNXT_NUM_ATTRS - 1) + +#define BNXT_NL_NAME "bnxt_netlink" +#define BNXT_NL_VER 1 + +int bnxt_nl_register(void); +int bnxt_nl_unregister(void); + +#endif -- 1.8.3.1