Received: by 2002:a05:6a10:22f:0:0:0:0 with SMTP id 15csp474381pxk; Thu, 1 Oct 2020 07:04:49 -0700 (PDT) X-Google-Smtp-Source: ABdhPJzfpOWayOkktXGtAQrEHqBR8Ez/vt27dxWn26l8BxU/y2INeMSCMDYjLp81BesJGVB4b/Rl X-Received: by 2002:a17:906:b146:: with SMTP id bt6mr8064885ejb.287.1601561089027; Thu, 01 Oct 2020 07:04:49 -0700 (PDT) ARC-Seal: i=1; a=rsa-sha256; t=1601561089; cv=none; d=google.com; s=arc-20160816; b=xC+JwRMoqE5ZfZ3aHXN4/mm9dSu5Jks+miiGlfSmWtzPLd4AuPGbydDrs4JuVTx/QQ yIc1W4VSMrEpUMyncagDmJN2FxmPganPmNH5erbCDER9VKvl7QjJ++ywKqF1ry+Rbrc0 ADLkeYzg8cLWJasLLfqbIHd+i2irvimqD3elh2b0QgIsPn72LVK6f3hvetbwCyxD0RoI lM6PxzXP3RHI9oQDcrDIZFXfPV6ou/kYrnWOfKRpNYq3Q6iTCHb2EYfXLxUpC8GAjyQH 8CMuXpsZvC7695FLUCZvtmgxwRLOvfiF/fA/1Y1uyvPBFL03Sf50r2+WUvOeYGPjwTZI JOAg== ARC-Message-Signature: i=1; a=rsa-sha256; c=relaxed/relaxed; d=google.com; s=arc-20160816; h=list-id:precedence:references:in-reply-to:message-id:date:subject :cc:to:from; bh=bpggaz2WUCsnUGN43smeRv+gipwycigc/sn7tzmJYQc=; b=plO4pvbdx4qwNQneu3eNmnwRJ4RLpvx8jI4MbO/ZVQfBdk9Y7iW1k7tcc/9mBqz0Ju e0jD8RUxITfIHJAwi4zMAr23azUb3uC1V6y06N+0zD1ElmMRs7e5tteqWGxo5mICPJWS j8na/hueNBbgNcILp3cKSXByEia+tapXpzG9QGdyoli5xE3wiM+r9diFryXUBsfs+kt6 7qavqpqmLSItF+CBqRqVs5fxvrrpx+C/NjvhOESvKEXW+FcPyzdHtbk8ZKruptJx9kHi fNI+PyBDo5D8hX1WQeqgPH7rvNKJNiFDEJnxhFx9bVRnMyqDb4QWeeiM+iyThqCNPATT Sdeg== ARC-Authentication-Results: i=1; mx.google.com; spf=pass (google.com: domain of linux-kernel-owner@vger.kernel.org designates 23.128.96.18 as permitted sender) smtp.mailfrom=linux-kernel-owner@vger.kernel.org; dmarc=fail (p=NONE sp=NONE dis=NONE) header.from=mellanox.com Return-Path: Received: from vger.kernel.org (vger.kernel.org. [23.128.96.18]) by mx.google.com with ESMTP id b19si3458617ejd.451.2020.10.01.07.04.24; Thu, 01 Oct 2020 07:04:49 -0700 (PDT) Received-SPF: pass (google.com: domain of linux-kernel-owner@vger.kernel.org designates 23.128.96.18 as permitted sender) client-ip=23.128.96.18; Authentication-Results: mx.google.com; spf=pass (google.com: domain of linux-kernel-owner@vger.kernel.org designates 23.128.96.18 as permitted sender) smtp.mailfrom=linux-kernel-owner@vger.kernel.org; dmarc=fail (p=NONE sp=NONE dis=NONE) header.from=mellanox.com Received: (majordomo@vger.kernel.org) by vger.kernel.org via listexpand id S1732600AbgJAOCT (ORCPT + 99 others); Thu, 1 Oct 2020 10:02:19 -0400 Received: from mail-il-dmz.mellanox.com ([193.47.165.129]:36649 "EHLO mellanox.co.il" rhost-flags-OK-OK-OK-FAIL) by vger.kernel.org with ESMTP id S1732444AbgJAOAa (ORCPT ); Thu, 1 Oct 2020 10:00:30 -0400 Received: from Internal Mail-Server by MTLPINE1 (envelope-from moshe@mellanox.com) with SMTP; 1 Oct 2020 17:00:25 +0300 Received: from dev-l-vrt-136.mtl.labs.mlnx (dev-l-vrt-136.mtl.labs.mlnx [10.234.136.1]) by labmailer.mlnx (8.13.8/8.13.8) with ESMTP id 091E0PPI002061; Thu, 1 Oct 2020 17:00:25 +0300 Received: from dev-l-vrt-136.mtl.labs.mlnx (localhost [127.0.0.1]) by dev-l-vrt-136.mtl.labs.mlnx (8.14.7/8.14.7) with ESMTP id 091E0Pkb011177; Thu, 1 Oct 2020 17:00:25 +0300 Received: (from moshe@localhost) by dev-l-vrt-136.mtl.labs.mlnx (8.14.7/8.14.7/Submit) id 091E0PJi011176; Thu, 1 Oct 2020 17:00:25 +0300 From: Moshe Shemesh To: "David S. Miller" , Jakub Kicinski , Jiri Pirko Cc: netdev@vger.kernel.org, linux-kernel@vger.kernel.org, Moshe Shemesh Subject: [PATCH net-next 09/16] net/mlx5: Handle sync reset now event Date: Thu, 1 Oct 2020 16:59:12 +0300 Message-Id: <1601560759-11030-10-git-send-email-moshe@mellanox.com> X-Mailer: git-send-email 1.8.4.3 In-Reply-To: <1601560759-11030-1-git-send-email-moshe@mellanox.com> References: <1601560759-11030-1-git-send-email-moshe@mellanox.com> Precedence: bulk List-ID: X-Mailing-List: linux-kernel@vger.kernel.org On sync_reset_now event the driver does reload and PCI link toggle to activate firmware upgrade reset. When the firmware sends this event it syncs the event on all PFs, so all PFs will do PCI link toggle at once. To do PCI link toggle, the driver ensures that no other device ID under the same bridge by checking that all the PF functions under the same PCI bridge have same device ID. If no other device it uses PCI bridge link control to turn link down and up. Signed-off-by: Moshe Shemesh Reviewed-by: Saeed Mahameed --- .../ethernet/mellanox/mlx5/core/fw_reset.c | 119 ++++++++++++++++++ 1 file changed, 119 insertions(+) diff --git a/drivers/net/ethernet/mellanox/mlx5/core/fw_reset.c b/drivers/net/ethernet/mellanox/mlx5/core/fw_reset.c index cd1b4e1b56ba..26caf65e9f5f 100644 --- a/drivers/net/ethernet/mellanox/mlx5/core/fw_reset.c +++ b/drivers/net/ethernet/mellanox/mlx5/core/fw_reset.c @@ -13,6 +13,7 @@ struct mlx5_fw_reset { struct workqueue_struct *wq; struct work_struct reset_request_work; struct work_struct reset_reload_work; + struct work_struct reset_now_work; unsigned long reset_flags; struct timer_list timer; }; @@ -156,6 +157,120 @@ static void mlx5_sync_reset_request_event(struct work_struct *work) mlx5_core_warn(dev, "PCI Sync FW Update Reset Ack. Device reset is expected.\n"); } +#define MLX5_PCI_LINK_UP_TIMEOUT 2000 + +static int mlx5_pci_link_toggle(struct mlx5_core_dev *dev) +{ + struct pci_bus *bridge_bus = dev->pdev->bus; + struct pci_dev *bridge = bridge_bus->self; + u16 reg16, dev_id, sdev_id; + unsigned long timeout; + struct pci_dev *sdev; + int cap, err; + u32 reg32; + + /* Check that all functions under the pci bridge are PFs of + * this device otherwise fail this function. + */ + err = pci_read_config_word(dev->pdev, PCI_DEVICE_ID, &dev_id); + if (err) + return err; + list_for_each_entry(sdev, &bridge_bus->devices, bus_list) { + err = pci_read_config_word(sdev, PCI_DEVICE_ID, &sdev_id); + if (err) + return err; + if (sdev_id != dev_id) + return -EPERM; + } + + cap = pci_find_capability(bridge, PCI_CAP_ID_EXP); + if (!cap) + return -EOPNOTSUPP; + + list_for_each_entry(sdev, &bridge_bus->devices, bus_list) { + pci_save_state(sdev); + pci_cfg_access_lock(sdev); + } + /* PCI link toggle */ + err = pci_read_config_word(bridge, cap + PCI_EXP_LNKCTL, ®16); + if (err) + return err; + reg16 |= PCI_EXP_LNKCTL_LD; + err = pci_write_config_word(bridge, cap + PCI_EXP_LNKCTL, reg16); + if (err) + return err; + msleep(500); + reg16 &= ~PCI_EXP_LNKCTL_LD; + err = pci_write_config_word(bridge, cap + PCI_EXP_LNKCTL, reg16); + if (err) + return err; + + /* Check link */ + err = pci_read_config_dword(bridge, cap + PCI_EXP_LNKCAP, ®32); + if (err) + return err; + if (!(reg32 & PCI_EXP_LNKCAP_DLLLARC)) { + mlx5_core_warn(dev, "No PCI link reporting capability (0x%08x)\n", reg32); + msleep(1000); + goto restore; + } + + timeout = jiffies + msecs_to_jiffies(MLX5_PCI_LINK_UP_TIMEOUT); + do { + err = pci_read_config_word(bridge, cap + PCI_EXP_LNKSTA, ®16); + if (err) + return err; + if (reg16 & PCI_EXP_LNKSTA_DLLLA) + break; + msleep(20); + } while (!time_after(jiffies, timeout)); + + if (reg16 & PCI_EXP_LNKSTA_DLLLA) { + mlx5_core_info(dev, "PCI Link up\n"); + } else { + mlx5_core_err(dev, "PCI link not ready (0x%04x) after %d ms\n", + reg16, MLX5_PCI_LINK_UP_TIMEOUT); + err = -ETIMEDOUT; + } + +restore: + list_for_each_entry(sdev, &bridge_bus->devices, bus_list) { + pci_cfg_access_unlock(sdev); + pci_restore_state(sdev); + } + + return err; +} + +static void mlx5_sync_reset_now_event(struct work_struct *work) +{ + struct mlx5_fw_reset *fw_reset = container_of(work, struct mlx5_fw_reset, + reset_now_work); + struct mlx5_core_dev *dev = fw_reset->dev; + int err; + + mlx5_sync_reset_clear_reset_requested(dev, false); + + mlx5_core_warn(dev, "Sync Reset now. Device is going to reset.\n"); + + err = mlx5_cmd_fast_teardown_hca(dev); + if (err) { + mlx5_core_warn(dev, "Fast teardown failed, no reset done, err %d\n", err); + goto done; + } + + err = mlx5_pci_link_toggle(dev); + if (err) { + mlx5_core_warn(dev, "mlx5_pci_link_toggle failed, no reset done, err %d\n", err); + goto done; + } + + mlx5_enter_error_state(dev, true); + mlx5_unload_one(dev, false); +done: + mlx5_load_one(dev, false); +} + static void mlx5_sync_reset_events_handle(struct mlx5_fw_reset *fw_reset, struct mlx5_eqe *eqe) { struct mlx5_eqe_sync_fw_update *sync_fw_update_eqe; @@ -167,6 +282,9 @@ static void mlx5_sync_reset_events_handle(struct mlx5_fw_reset *fw_reset, struct case MLX5_SYNC_RST_STATE_RESET_REQUEST: queue_work(fw_reset->wq, &fw_reset->reset_request_work); break; + case MLX5_SYNC_RST_STATE_RESET_NOW: + queue_work(fw_reset->wq, &fw_reset->reset_now_work); + break; } } @@ -216,6 +334,7 @@ int mlx5_fw_reset_init(struct mlx5_core_dev *dev) INIT_WORK(&fw_reset->reset_request_work, mlx5_sync_reset_request_event); INIT_WORK(&fw_reset->reset_reload_work, mlx5_sync_reset_reload_work); + INIT_WORK(&fw_reset->reset_now_work, mlx5_sync_reset_now_event); return 0; } -- 2.18.2