Received: by 2002:a05:6358:3188:b0:123:57c1:9b43 with SMTP id q8csp36462000rwd; Tue, 11 Jul 2023 01:01:36 -0700 (PDT) X-Google-Smtp-Source: APBJJlEbNNaQcnCmQpTKnE/rAWH7UHhEWzvlN/coR/i7wsw/1xaui6ZR8uSPOeySupYDvAm12vqZ X-Received: by 2002:aa7:c68d:0:b0:51d:d5a1:a7f1 with SMTP id n13-20020aa7c68d000000b0051dd5a1a7f1mr15456640edq.38.1689062495852; Tue, 11 Jul 2023 01:01:35 -0700 (PDT) ARC-Seal: i=1; a=rsa-sha256; t=1689062495; cv=none; d=google.com; s=arc-20160816; b=KBF9j80/J6tJqbxCyucgoJuU7QrNfBaW9e0vOqAd3GWzOOTrsbbrA01707baIU0fGM TNFo60W7P3EgoZQm287Q1STgXXyVNE7DBXLUkD3vuVqa7WSxN2qo1sFggGUtNuEIlKMw trQ73M5M/DUBnB29pxJKlbZjFnYHvbjUkujR/OJW4Ig3hz4mp3UvKScwwJf0mJMocfmc d85WwyiKWElbwAtgdG1QHjz9b90tUwKB89gd+PnflpGDM26cdLjvcuFWZXETVJORUvd6 GdDb901/ejlkKXJgFj2NM6Q6Gcn7mF021oI5+DqER+krQc+1ahm32uGHRNBEJX6Os1iw dCzQ== ARC-Message-Signature: i=1; a=rsa-sha256; c=relaxed/relaxed; d=google.com; s=arc-20160816; h=list-id:precedence:in-reply-to:content-disposition:mime-version :references:message-id:subject:cc:to:from:date:dkim-signature; bh=iPYdiWtKUoG2uHq4V+UNEuEf7KjilmHlVqRMY+JnH0k=; fh=Y3R4jNiwK4vURTCxe4NVPyVS+MXQGlVxOUGhi8ZpBDw=; b=ODfhiiaWBBKyhxRysq1JFdU3zNC57pvRWnIbimT1k2+x9aqx+mq/0YsxF5Occrj4ft bac66vjNqrW8CcIjyKFGJ34yJimjnYaqNX94CYui0MFgVeclHhSljVJSuUG6cjvneKXk L17JYa8Z3xMQFBvwZm5lQtHaf2Hcwwkv/qQ+fE8arPk+QocD275g9rogECIAbGDNC8J5 g8Ul8RwkLIJB+vPd3BpaOEUyJiDdbybr8MQApkgtboK1WAVG3MGlPEhQH14NBH8/hERS CPTflBuMVC8R8r0L1QhAiusFsk4nb2IQ+U+jq4n5+WNhu+CQ/UYZi7HXLmmxMNleVPth Lr5Q== ARC-Authentication-Results: i=1; mx.google.com; dkim=pass header.i=@kernel.org header.s=k20201202 header.b=bK1ndyY9; spf=pass (google.com: domain of linux-kernel-owner@vger.kernel.org designates 2620:137:e000::1:20 as permitted sender) smtp.mailfrom=linux-kernel-owner@vger.kernel.org; dmarc=pass (p=NONE sp=NONE dis=NONE) header.from=kernel.org Return-Path: Received: from out1.vger.email (out1.vger.email. [2620:137:e000::1:20]) by mx.google.com with ESMTP id w16-20020a056402071000b0051e26a33fa2si1543688edx.436.2023.07.11.01.01.12; Tue, 11 Jul 2023 01:01:35 -0700 (PDT) Received-SPF: pass (google.com: domain of linux-kernel-owner@vger.kernel.org designates 2620:137:e000::1:20 as permitted sender) client-ip=2620:137:e000::1:20; Authentication-Results: mx.google.com; dkim=pass header.i=@kernel.org header.s=k20201202 header.b=bK1ndyY9; spf=pass (google.com: domain of linux-kernel-owner@vger.kernel.org designates 2620:137:e000::1:20 as permitted sender) smtp.mailfrom=linux-kernel-owner@vger.kernel.org; dmarc=pass (p=NONE sp=NONE dis=NONE) header.from=kernel.org Received: (majordomo@vger.kernel.org) by vger.kernel.org via listexpand id S230399AbjGKHkf (ORCPT + 99 others); Tue, 11 Jul 2023 03:40:35 -0400 Received: from lindbergh.monkeyblade.net ([23.128.96.19]:53222 "EHLO lindbergh.monkeyblade.net" rhost-flags-OK-OK-OK-OK) by vger.kernel.org with ESMTP id S229939AbjGKHkc (ORCPT ); Tue, 11 Jul 2023 03:40:32 -0400 Received: from dfw.source.kernel.org (dfw.source.kernel.org [IPv6:2604:1380:4641:c500::1]) by lindbergh.monkeyblade.net (Postfix) with ESMTPS id CACD5133; Tue, 11 Jul 2023 00:40:30 -0700 (PDT) Received: from smtp.kernel.org (relay.kernel.org [52.25.139.140]) (using TLSv1.3 with cipher TLS_AES_256_GCM_SHA384 (256/256 bits) key-exchange X25519 server-signature RSA-PSS (2048 bits)) (No client certificate requested) by dfw.source.kernel.org (Postfix) with ESMTPS id 4794861359; Tue, 11 Jul 2023 07:40:30 +0000 (UTC) Received: by smtp.kernel.org (Postfix) with ESMTPSA id 254ABC433C8; Tue, 11 Jul 2023 07:40:28 +0000 (UTC) DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/simple; d=kernel.org; s=k20201202; t=1689061229; bh=mir9lMosxsKMhRVUiJnRWnuPkqQkT24ItkUJNuKUGqY=; h=Date:From:To:Cc:Subject:References:In-Reply-To:From; b=bK1ndyY9I2b4mhR0o4T3l/9VjE7JG1KzF9bFos3OF2I61Wu3daHV1EJSKmkI5emmu ip5mIrYXuvklerAmWZP5chAYRtu+nEdKIzLWcJOwCZNDcdGdmL8ZqFwlzHkMIn5SOj 3YmXqe93RMwLiHsZKV8jnl8FJDwJDD8qNNa+pcg888XbIH8G1lqKVL1Wnmt5cht1Ug R7CQRhNxRdPLBTIYTyYIsIhKm+29yvNPNxjiJxxW/2NgEWYZcGsUsVOzj4llw4y8ne PvIlP9abEb1p9PHhE6T6/9DJgsm8xJbZBGxdaeGqLPP+NEN9HXaPewYuXRwsL9sdxV m/sNgmlraD8QA== Date: Tue, 11 Jul 2023 10:40:25 +0300 From: Leon Romanovsky To: Souradeep Chakrabarti Cc: kys@microsoft.com, haiyangz@microsoft.com, wei.liu@kernel.org, decui@microsoft.com, davem@davemloft.net, edumazet@google.com, kuba@kernel.org, pabeni@redhat.com, longli@microsoft.com, sharmaajay@microsoft.com, cai.huoqing@linux.dev, ssengar@linux.microsoft.com, vkuznets@redhat.com, tglx@linutronix.de, linux-hyperv@vger.kernel.org, netdev@vger.kernel.org, linux-kernel@vger.kernel.org, linux-rdma@vger.kernel.org, schakrabarti@microsoft.com Subject: Re: [PATCH V3 net] net: mana: Configure hwc timeout from hardware Message-ID: <20230711074025.GK41919@unreal> References: <1689060957-1475-1-git-send-email-schakrabarti@linux.microsoft.com> MIME-Version: 1.0 Content-Type: text/plain; charset=us-ascii Content-Disposition: inline In-Reply-To: <1689060957-1475-1-git-send-email-schakrabarti@linux.microsoft.com> X-Spam-Status: No, score=-2.1 required=5.0 tests=BAYES_00,DKIMWL_WL_HIGH, DKIM_SIGNED,DKIM_VALID,DKIM_VALID_AU,DKIM_VALID_EF, RCVD_IN_DNSWL_BLOCKED,SPF_HELO_NONE,SPF_PASS,T_SCC_BODY_TEXT_LINE autolearn=ham autolearn_force=no version=3.4.6 X-Spam-Checker-Version: SpamAssassin 3.4.6 (2021-04-09) on lindbergh.monkeyblade.net Precedence: bulk List-ID: X-Mailing-List: linux-kernel@vger.kernel.org On Tue, Jul 11, 2023 at 12:35:57AM -0700, Souradeep Chakrabarti wrote: > At present hwc timeout value is a fixed value. > This patch sets the hwc timeout from the hardware. > It now uses a new hardware capability This is new functionality which means that patch should be sent to net-next. > GDMA_DRV_CAP_FLAG_1_HWC_TIMEOUT_RECONFIG to query > and set the value in hwc_timeout. Please try to avoid breaking your commit messages after 50 chars and use whole available line space. Thanks > > Signed-off-by: Souradeep Chakrabarti > --- > V2 -> V3: > * Removed the stable release from cc > * Formatted the variable position to follow reverse xmas tree > * Removed the log from mana_gd_query_hwc_timeout on success scenario > --- > .../net/ethernet/microsoft/mana/gdma_main.c | 30 ++++++++++++++++++- > .../net/ethernet/microsoft/mana/hw_channel.c | 25 +++++++++++++++- > include/net/mana/gdma.h | 20 ++++++++++++- > include/net/mana/hw_channel.h | 5 ++++ > 4 files changed, 77 insertions(+), 3 deletions(-) > > diff --git a/drivers/net/ethernet/microsoft/mana/gdma_main.c b/drivers/net/ethernet/microsoft/mana/gdma_main.c > index 8f3f78b68592..4537a70e30d4 100644 > --- a/drivers/net/ethernet/microsoft/mana/gdma_main.c > +++ b/drivers/net/ethernet/microsoft/mana/gdma_main.c > @@ -106,6 +106,25 @@ static int mana_gd_query_max_resources(struct pci_dev *pdev) > return 0; > } > > +static int mana_gd_query_hwc_timeout(struct pci_dev *pdev, u32 *timeout_val) > +{ > + struct gdma_context *gc = pci_get_drvdata(pdev); > + struct gdma_query_hwc_timeout_resp resp = {}; > + struct gdma_query_hwc_timeout_req req = {}; > + int err; > + > + mana_gd_init_req_hdr(&req.hdr, GDMA_QUERY_HWC_TIMEOUT, > + sizeof(req), sizeof(resp)); > + req.timeout_ms = *timeout_val; > + err = mana_gd_send_request(gc, sizeof(req), &req, sizeof(resp), &resp); > + if (err || resp.hdr.status) > + return err ? err : -EPROTO; > + > + *timeout_val = resp.timeout_ms; > + > + return 0; > +} > + > static int mana_gd_detect_devices(struct pci_dev *pdev) > { > struct gdma_context *gc = pci_get_drvdata(pdev); > @@ -879,8 +898,11 @@ int mana_gd_verify_vf_version(struct pci_dev *pdev) > struct gdma_context *gc = pci_get_drvdata(pdev); > struct gdma_verify_ver_resp resp = {}; > struct gdma_verify_ver_req req = {}; > + struct hw_channel_context *hwc; > int err; > > + hwc = gc->hwc.driver_data; > + > mana_gd_init_req_hdr(&req.hdr, GDMA_VERIFY_VF_DRIVER_VERSION, > sizeof(req), sizeof(resp)); > > @@ -907,7 +929,13 @@ int mana_gd_verify_vf_version(struct pci_dev *pdev) > err, resp.hdr.status); > return err ? err : -EPROTO; > } > - > + if (resp.pf_cap_flags1 & GDMA_DRV_CAP_FLAG_1_HWC_TIMEOUT_RECONFIG) { > + err = mana_gd_query_hwc_timeout(pdev, &hwc->hwc_timeout); > + if (err) { > + dev_err(gc->dev, "Failed to set the hwc timeout %d\n", err); > + return err; > + } > + } > return 0; > } > > diff --git a/drivers/net/ethernet/microsoft/mana/hw_channel.c b/drivers/net/ethernet/microsoft/mana/hw_channel.c > index 2bd1d74021f7..db433501e5e6 100644 > --- a/drivers/net/ethernet/microsoft/mana/hw_channel.c > +++ b/drivers/net/ethernet/microsoft/mana/hw_channel.c > @@ -174,7 +174,25 @@ static void mana_hwc_init_event_handler(void *ctx, struct gdma_queue *q_self, > complete(&hwc->hwc_init_eqe_comp); > break; > > + case GDMA_EQE_HWC_SOC_RECONFIG_DATA: > + type_data.as_uint32 = event->details[0]; > + type = type_data.type; > + val = type_data.value; > + > + switch (type) { > + case HWC_DATA_CFG_HWC_TIMEOUT: > + hwc->hwc_timeout = val; > + break; > + > + default: > + dev_warn(hwc->dev, "Received unknown reconfig type %u\n", type); > + break; > + } > + > + break; > + > default: > + dev_warn(hwc->dev, "Received unknown gdma event %u\n", event->type); > /* Ignore unknown events, which should never happen. */ > break; > } > @@ -704,6 +722,7 @@ int mana_hwc_create_channel(struct gdma_context *gc) > gd->pdid = INVALID_PDID; > gd->doorbell = INVALID_DOORBELL; > > + hwc->hwc_timeout = HW_CHANNEL_WAIT_RESOURCE_TIMEOUT_MS; > /* mana_hwc_init_queues() only creates the required data structures, > * and doesn't touch the HWC device. > */ > @@ -770,6 +789,8 @@ void mana_hwc_destroy_channel(struct gdma_context *gc) > hwc->gdma_dev->doorbell = INVALID_DOORBELL; > hwc->gdma_dev->pdid = INVALID_PDID; > > + hwc->hwc_timeout = 0; > + > kfree(hwc); > gc->hwc.driver_data = NULL; > gc->hwc.gdma_context = NULL; > @@ -818,6 +839,7 @@ int mana_hwc_send_request(struct hw_channel_context *hwc, u32 req_len, > dest_vrq = hwc->pf_dest_vrq_id; > dest_vrcq = hwc->pf_dest_vrcq_id; > } > + dev_err(hwc->dev, "HWC: timeout %u ms\n", hwc->hwc_timeout); > > err = mana_hwc_post_tx_wqe(txq, tx_wr, dest_vrq, dest_vrcq, false); > if (err) { > @@ -825,7 +847,8 @@ int mana_hwc_send_request(struct hw_channel_context *hwc, u32 req_len, > goto out; > } > > - if (!wait_for_completion_timeout(&ctx->comp_event, 30 * HZ)) { > + if (!wait_for_completion_timeout(&ctx->comp_event, > + (hwc->hwc_timeout / 1000) * HZ)) { > dev_err(hwc->dev, "HWC: Request timed out!\n"); > err = -ETIMEDOUT; > goto out; > diff --git a/include/net/mana/gdma.h b/include/net/mana/gdma.h > index 96c120160f15..88b6ef7ce1a6 100644 > --- a/include/net/mana/gdma.h > +++ b/include/net/mana/gdma.h > @@ -33,6 +33,7 @@ enum gdma_request_type { > GDMA_DESTROY_PD = 30, > GDMA_CREATE_MR = 31, > GDMA_DESTROY_MR = 32, > + GDMA_QUERY_HWC_TIMEOUT = 84, /* 0x54 */ > }; > > #define GDMA_RESOURCE_DOORBELL_PAGE 27 > @@ -57,6 +58,8 @@ enum gdma_eqe_type { > GDMA_EQE_HWC_INIT_EQ_ID_DB = 129, > GDMA_EQE_HWC_INIT_DATA = 130, > GDMA_EQE_HWC_INIT_DONE = 131, > + GDMA_EQE_HWC_SOC_RECONFIG = 132, > + GDMA_EQE_HWC_SOC_RECONFIG_DATA = 133, > }; > > enum { > @@ -531,10 +534,12 @@ enum { > * so the driver is able to reliably support features like busy_poll. > */ > #define GDMA_DRV_CAP_FLAG_1_NAPI_WKDONE_FIX BIT(2) > +#define GDMA_DRV_CAP_FLAG_1_HWC_TIMEOUT_RECONFIG BIT(3) > > #define GDMA_DRV_CAP_FLAGS1 \ > (GDMA_DRV_CAP_FLAG_1_EQ_SHARING_MULTI_VPORT | \ > - GDMA_DRV_CAP_FLAG_1_NAPI_WKDONE_FIX) > + GDMA_DRV_CAP_FLAG_1_NAPI_WKDONE_FIX | \ > + GDMA_DRV_CAP_FLAG_1_HWC_TIMEOUT_RECONFIG) > > #define GDMA_DRV_CAP_FLAGS2 0 > > @@ -664,6 +669,19 @@ struct gdma_disable_queue_req { > u32 alloc_res_id_on_creation; > }; /* HW DATA */ > > +/* GDMA_QUERY_HWC_TIMEOUT */ > +struct gdma_query_hwc_timeout_req { > + struct gdma_req_hdr hdr; > + u32 timeout_ms; > + u32 reserved; > +}; > + > +struct gdma_query_hwc_timeout_resp { > + struct gdma_resp_hdr hdr; > + u32 timeout_ms; > + u32 reserved; > +}; > + > enum atb_page_size { > ATB_PAGE_SIZE_4K, > ATB_PAGE_SIZE_8K, > diff --git a/include/net/mana/hw_channel.h b/include/net/mana/hw_channel.h > index 6a757a6e2732..3d3b5c881bc1 100644 > --- a/include/net/mana/hw_channel.h > +++ b/include/net/mana/hw_channel.h > @@ -23,6 +23,10 @@ > #define HWC_INIT_DATA_PF_DEST_RQ_ID 10 > #define HWC_INIT_DATA_PF_DEST_CQ_ID 11 > > +#define HWC_DATA_CFG_HWC_TIMEOUT 1 > + > +#define HW_CHANNEL_WAIT_RESOURCE_TIMEOUT_MS 30000 > + > /* Structures labeled with "HW DATA" are exchanged with the hardware. All of > * them are naturally aligned and hence don't need __packed. > */ > @@ -182,6 +186,7 @@ struct hw_channel_context { > > u32 pf_dest_vrq_id; > u32 pf_dest_vrcq_id; > + u32 hwc_timeout; > > struct hwc_caller_ctx *caller_ctx; > }; > -- > 2.34.1 >