Received: by 2002:a6b:fb09:0:0:0:0:0 with SMTP id h9csp1038759iog; Wed, 15 Jun 2022 19:09:31 -0700 (PDT) X-Google-Smtp-Source: AGRyM1syzKwKc8DMiMh2rhRocMW5GwHSv7rjbMCIbTAgXwu0FQtXVf+Hi63Wk6CoqIdVEsRJ1zcS X-Received: by 2002:a17:902:aa0c:b0:168:faa0:50a1 with SMTP id be12-20020a170902aa0c00b00168faa050a1mr2614962plb.57.1655345371311; Wed, 15 Jun 2022 19:09:31 -0700 (PDT) ARC-Seal: i=1; a=rsa-sha256; t=1655345371; cv=none; d=google.com; s=arc-20160816; b=kKDprG1z+bkkhGwolrKWdie1E2onUm+4r2trBxnTVm/HaZCZdxu/dp4ZoT5H1ecvuR JRh37fRr7l5Hy8jBS3K8yHzh8DbAlThHNGZEnvXpAczJvJhHDxgaGFksMRHv0cjzvQtZ dV0AowQpQe0lO2kVbjGGhahrPzFwaoeosH+ahKrq4XA43iqlmld+juQJMkmGA1h4AaJs EyMf74sstQJviMCmVE2MO1CFfyVP+BP7blUPFJvrghGKinsvHyiIj4UBghGnWNE3gwPB vL65lvyZtlvo6jmTvejC9RFrsV3scsxVY6/R8mBsOE7MwbXjFZv40uWfW99Laakgvm51 zjqA== ARC-Message-Signature: i=1; a=rsa-sha256; c=relaxed/relaxed; d=google.com; s=arc-20160816; h=list-id:precedence:reply-to:references:in-reply-to:message-id:date :subject:cc:to:from:dkim-signature:dkim-filter; bh=6JaEm1a6tNhVzT6xGRcitQHJYk7ChgQN72DXm0YtbbQ=; b=XRQZSMeOsHdamz6DRY4633oASlSoK8It6SiRN/9COcsa4Tz2Dm2Qwed8/6MqxlT68h if6wvYSubXmI2j/W6X6LSI03NkU/HC/Vs6srxf7ohv21NWO2Xc3pqpv49cP6crNM59K0 bHZaoXuKKsKTIORc3xX8AG/BvbwDelUx08SOrKR6jwgJcEc7xpJ1AAuC6usrzuTDdD0T Y7rblNfMnSgO+DOwDBn7/s0u0I+O/j5w8SkomlrOdx1WY6CIBXVxxtt/0h2zPGWUmzWE SdTwLttvBqv6zr4oaG0w16YecIJ1Y+PnI4SdiIvUg3omRuqEDLyUu4xsGN/IeMKOk0Yq tZ+A== ARC-Authentication-Results: i=1; mx.google.com; dkim=pass header.i=@linuxonhyperv.com header.s=default header.b=Jx5UdVfQ; spf=pass (google.com: domain of linux-kernel-owner@vger.kernel.org designates 2620:137:e000::1:20 as permitted sender) smtp.mailfrom=linux-kernel-owner@vger.kernel.org; dmarc=pass (p=NONE sp=NONE dis=NONE) header.from=linuxonhyperv.com Return-Path: Received: from out1.vger.email (out1.vger.email. [2620:137:e000::1:20]) by mx.google.com with ESMTP id d23-20020aa78697000000b0050dfdded307si988804pfo.31.2022.06.15.19.09.18; Wed, 15 Jun 2022 19:09:31 -0700 (PDT) Received-SPF: pass (google.com: domain of linux-kernel-owner@vger.kernel.org designates 2620:137:e000::1:20 as permitted sender) client-ip=2620:137:e000::1:20; Authentication-Results: mx.google.com; dkim=pass header.i=@linuxonhyperv.com header.s=default header.b=Jx5UdVfQ; spf=pass (google.com: domain of linux-kernel-owner@vger.kernel.org designates 2620:137:e000::1:20 as permitted sender) smtp.mailfrom=linux-kernel-owner@vger.kernel.org; dmarc=pass (p=NONE sp=NONE dis=NONE) header.from=linuxonhyperv.com Received: (majordomo@vger.kernel.org) by vger.kernel.org via listexpand id S1358201AbiFPCH5 (ORCPT + 99 others); Wed, 15 Jun 2022 22:07:57 -0400 Received: from lindbergh.monkeyblade.net ([23.128.96.19]:42384 "EHLO lindbergh.monkeyblade.net" rhost-flags-OK-OK-OK-OK) by vger.kernel.org with ESMTP id S1348213AbiFPCHh (ORCPT ); Wed, 15 Jun 2022 22:07:37 -0400 Received: from linux.microsoft.com (linux.microsoft.com [13.77.154.182]) by lindbergh.monkeyblade.net (Postfix) with ESMTP id 86C9BFC3; Wed, 15 Jun 2022 19:07:30 -0700 (PDT) Received: by linux.microsoft.com (Postfix, from userid 1004) id 5ABA520C327C; Wed, 15 Jun 2022 19:07:30 -0700 (PDT) DKIM-Filter: OpenDKIM Filter v2.11.0 linux.microsoft.com 5ABA520C327C DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=linuxonhyperv.com; s=default; t=1655345250; bh=6JaEm1a6tNhVzT6xGRcitQHJYk7ChgQN72DXm0YtbbQ=; h=From:To:Cc:Subject:Date:In-Reply-To:References:Reply-To:From; b=Jx5UdVfQJKpe+FxNgQDTNx/0jCDaseIf4tI0Gg3G/XDf6hUFs4nPQRKXvTkicNPjG VEaczPPSZ9q9dNm81+30eUOmdN4GfYBNxPc88I80DEIsRKNhmHNXqWRNcVWTvIH1gH Bq4iM2gVupCYnIf50F3rrS+CmUjgZXptM3rojd3A= From: longli@linuxonhyperv.com To: "K. Y. Srinivasan" , Haiyang Zhang , Stephen Hemminger , Wei Liu , Dexuan Cui , "David S. Miller" , Jakub Kicinski , Paolo Abeni , Jason Gunthorpe , Leon Romanovsky , edumazet@google.com, shiraz.saleem@intel.com, Ajay Sharma Cc: linux-hyperv@vger.kernel.org, netdev@vger.kernel.org, linux-kernel@vger.kernel.org, linux-rdma@vger.kernel.org, Long Li Subject: [Patch v4 03/12] net: mana: Handle vport sharing between devices Date: Wed, 15 Jun 2022 19:07:11 -0700 Message-Id: <1655345240-26411-4-git-send-email-longli@linuxonhyperv.com> X-Mailer: git-send-email 1.8.3.1 In-Reply-To: <1655345240-26411-1-git-send-email-longli@linuxonhyperv.com> References: <1655345240-26411-1-git-send-email-longli@linuxonhyperv.com> Reply-To: longli@microsoft.com X-Spam-Status: No, score=-11.6 required=5.0 tests=BAYES_00,DKIM_SIGNED, DKIM_VALID,DKIM_VALID_AU,HEADER_FROM_DIFFERENT_DOMAINS, RCVD_IN_DNSWL_MED,SPF_HELO_PASS,SPF_PASS,T_SCC_BODY_TEXT_LINE, USER_IN_DEF_SPF_WL autolearn=ham autolearn_force=no version=3.4.6 X-Spam-Checker-Version: SpamAssassin 3.4.6 (2021-04-09) on lindbergh.monkeyblade.net Precedence: bulk List-ID: X-Mailing-List: linux-kernel@vger.kernel.org From: Long Li For outgoing packets, the PF requires the VF to configure the vport with corresponding protection domain and doorbell ID for the kernel or user context. The vport can't be shared between different contexts. Implement the logic to exclusively take over the vport by either the Ethernet device or RDMA device. Signed-off-by: Long Li --- Change log: v2: use refcount instead of directly using atomic variables v4: change to mutex to avoid possible race with refcount drivers/net/ethernet/microsoft/mana/mana.h | 7 ++++ drivers/net/ethernet/microsoft/mana/mana_en.c | 40 ++++++++++++++++++- 2 files changed, 45 insertions(+), 2 deletions(-) diff --git a/drivers/net/ethernet/microsoft/mana/mana.h b/drivers/net/ethernet/microsoft/mana/mana.h index 51bff91b63ee..8e58abdce906 100644 --- a/drivers/net/ethernet/microsoft/mana/mana.h +++ b/drivers/net/ethernet/microsoft/mana/mana.h @@ -376,6 +376,10 @@ struct mana_port_context { mana_handle_t port_handle; + /* Mutex for sharing access to vport_use_count */ + struct mutex vport_mutex; + int vport_use_count; + u16 port_idx; bool port_is_up; @@ -567,4 +571,7 @@ struct mana_adev { struct gdma_dev *mdev; }; +int mana_cfg_vport(struct mana_port_context *apc, u32 protection_dom_id, + u32 doorbell_pg_id); +void mana_uncfg_vport(struct mana_port_context *apc); #endif /* _MANA_H */ diff --git a/drivers/net/ethernet/microsoft/mana/mana_en.c b/drivers/net/ethernet/microsoft/mana/mana_en.c index 745a9783dd70..23e7e423a544 100644 --- a/drivers/net/ethernet/microsoft/mana/mana_en.c +++ b/drivers/net/ethernet/microsoft/mana/mana_en.c @@ -530,13 +530,31 @@ static int mana_query_vport_cfg(struct mana_port_context *apc, u32 vport_index, return 0; } -static int mana_cfg_vport(struct mana_port_context *apc, u32 protection_dom_id, - u32 doorbell_pg_id) +void mana_uncfg_vport(struct mana_port_context *apc) +{ + mutex_lock(&apc->vport_mutex); + apc->vport_use_count--; + WARN_ON(apc->vport_use_count < 0); + mutex_unlock(&apc->vport_mutex); +} +EXPORT_SYMBOL_GPL(mana_uncfg_vport); + +int mana_cfg_vport(struct mana_port_context *apc, u32 protection_dom_id, + u32 doorbell_pg_id) { struct mana_config_vport_resp resp = {}; struct mana_config_vport_req req = {}; int err; + /* Ethernet driver and IB driver can't take the port at the same time */ + mutex_lock(&apc->vport_mutex); + if (apc->vport_use_count > 0) { + mutex_unlock(&apc->vport_mutex); + return -ENODEV; + } + apc->vport_use_count++; + mutex_unlock(&apc->vport_mutex); + mana_gd_init_req_hdr(&req.hdr, MANA_CONFIG_VPORT_TX, sizeof(req), sizeof(resp)); req.vport = apc->port_handle; @@ -563,9 +581,19 @@ static int mana_cfg_vport(struct mana_port_context *apc, u32 protection_dom_id, apc->tx_shortform_allowed = resp.short_form_allowed; apc->tx_vp_offset = resp.tx_vport_offset; + + netdev_info(apc->ndev, "Configured vPort %llu PD %u DB %u\n", + apc->port_handle, protection_dom_id, doorbell_pg_id); out: + if (err) { + mutex_lock(&apc->vport_mutex); + apc->vport_use_count--; + mutex_unlock(&apc->vport_mutex); + } + return err; } +EXPORT_SYMBOL_GPL(mana_cfg_vport); static int mana_cfg_vport_steering(struct mana_port_context *apc, enum TRI_STATE rx, @@ -626,6 +654,9 @@ static int mana_cfg_vport_steering(struct mana_port_context *apc, resp.hdr.status); err = -EPROTO; } + + netdev_info(ndev, "Configured steering vPort %llu entries %u\n", + apc->port_handle, num_entries); out: kfree(req); return err; @@ -1678,6 +1709,8 @@ static void mana_destroy_vport(struct mana_port_context *apc) } mana_destroy_txq(apc); + + mana_uncfg_vport(apc); } static int mana_create_vport(struct mana_port_context *apc, @@ -1929,6 +1962,9 @@ static int mana_probe_port(struct mana_context *ac, int port_idx, apc->port_handle = INVALID_MANA_HANDLE; apc->port_idx = port_idx; + mutex_init(&apc->vport_mutex); + apc->vport_use_count = 0; + ndev->netdev_ops = &mana_devops; ndev->ethtool_ops = &mana_ethtool_ops; ndev->mtu = ETH_DATA_LEN; -- 2.17.1