Received: by 2002:a05:7412:b995:b0:f9:9502:5bb8 with SMTP id it21csp6919803rdb; Tue, 2 Jan 2024 20:09:02 -0800 (PST) X-Google-Smtp-Source: AGHT+IGZr87YljpA2hXr1jrgYAFKDrT6UhhAWx/yw0kcMV8JhgNHgpbZty4ImkX2IxxILRBtek0J X-Received: by 2002:a50:d6c2:0:b0:553:b54f:e9ae with SMTP id l2-20020a50d6c2000000b00553b54fe9aemr12162941edj.17.1704254942114; Tue, 02 Jan 2024 20:09:02 -0800 (PST) ARC-Seal: i=1; a=rsa-sha256; t=1704254942; cv=none; d=google.com; s=arc-20160816; b=p0Ok1HiGabiwf7agl9i+dNG5xTa5NYLQQIox+fs7Jry4U7pF6gse4qQdTgU1aGNT4l mY4LR1PISRe5AdEr5+NsHJnIReBpqGjMnZNVetYyADB1oHs01qhe4D+uM1vzhvl0Ffnq CDDEHZcvTMrpkljsd7UFyvG6CCEZvRX0d2j0PEOUm3tA9zQ6fGMF5wTQndbvFydRebv2 u/h9ade/k7oz+BHtGurWdH3lQODf8VJJ93DGL7juY1oS2ifRvH2fMlgq96Rc1CGFsr6y rDGC1HtjLthB7BzXZvwSKERTqY+xTaUhO8qidRED4YD38LHY7sn8NHnd5eUDb/CTthJF ry1g== ARC-Message-Signature: i=1; a=rsa-sha256; c=relaxed/relaxed; d=google.com; s=arc-20160816; h=content-transfer-encoding:mime-version:list-unsubscribe :list-subscribe:list-id:precedence:message-id:date:subject:cc:to :from:dkim-signature; bh=7zer3uRlPxIGhxPISasCvhGIoK2PpKjHVpbHTxrzLj0=; fh=IenMhONiCo+n2wl/kAENWCURaB7iiBbsL05ynZhFKmI=; b=YhWJaLUmfZxEFOg0tgCmfTG2ir2U4VfQgGmZkHNt6kSPeMgQvWgEAcaZvS8B9FL4l7 Vkg6kJ3G9lvjD6HcZyyarYGaWp18vCOgtBFrlB0APLYyLMhLbNjnwX/61/j+Yox+BZgT 5CwW+/gx8N0rmhKwUlT9Gm3s3LTtplTB6u8JmEQH4/n/YFim+w7RLYI8E4hDtIJ6mseb 8dt9XYpZEGxN8UASCE9BIb/3T8Fk+8HuJOf7kGtNKT8KBs5nNfh74Z1QtYmDVU3VaWLe fFFJGAz40iRUkb1lgwPCmEkgeVhJS1QwgyJjBVjftw99JfLi6QpSwm6kpfu+7Ky3Vo9I eK5A== ARC-Authentication-Results: i=1; mx.google.com; dkim=pass header.i=@intel.com header.s=Intel header.b=mLcMualv; spf=pass (google.com: domain of linux-crypto+bounces-1199-linux.lists.archive=gmail.com@vger.kernel.org designates 2604:1380:4601:e00::3 as permitted sender) smtp.mailfrom="linux-crypto+bounces-1199-linux.lists.archive=gmail.com@vger.kernel.org"; dmarc=pass (p=NONE sp=NONE dis=NONE) header.from=intel.com Return-Path: Received: from am.mirrors.kernel.org (am.mirrors.kernel.org. [2604:1380:4601:e00::3]) by mx.google.com with ESMTPS id p5-20020a056402500500b005555228c29csi6115539eda.65.2024.01.02.20.09.02 for (version=TLS1_3 cipher=TLS_AES_256_GCM_SHA384 bits=256/256); Tue, 02 Jan 2024 20:09:02 -0800 (PST) Received-SPF: pass (google.com: domain of linux-crypto+bounces-1199-linux.lists.archive=gmail.com@vger.kernel.org designates 2604:1380:4601:e00::3 as permitted sender) client-ip=2604:1380:4601:e00::3; Authentication-Results: mx.google.com; dkim=pass header.i=@intel.com header.s=Intel header.b=mLcMualv; spf=pass (google.com: domain of linux-crypto+bounces-1199-linux.lists.archive=gmail.com@vger.kernel.org designates 2604:1380:4601:e00::3 as permitted sender) smtp.mailfrom="linux-crypto+bounces-1199-linux.lists.archive=gmail.com@vger.kernel.org"; dmarc=pass (p=NONE sp=NONE dis=NONE) header.from=intel.com Received: from smtp.subspace.kernel.org (wormhole.subspace.kernel.org [52.25.139.140]) (using TLSv1.2 with cipher ECDHE-RSA-AES256-GCM-SHA384 (256/256 bits)) (No client certificate requested) by am.mirrors.kernel.org (Postfix) with ESMTPS id 9725D1F2387F for ; Wed, 3 Jan 2024 04:09:01 +0000 (UTC) Received: from localhost.localdomain (localhost.localdomain [127.0.0.1]) by smtp.subspace.kernel.org (Postfix) with ESMTP id 4129F168D4; Wed, 3 Jan 2024 04:08:56 +0000 (UTC) Authentication-Results: smtp.subspace.kernel.org; dkim=pass (2048-bit key) header.d=intel.com header.i=@intel.com header.b="mLcMualv" X-Original-To: linux-crypto@vger.kernel.org Received: from mgamail.intel.com (mgamail.intel.com [198.175.65.11]) (using TLSv1.2 with cipher ECDHE-RSA-AES256-GCM-SHA384 (256/256 bits)) (No client certificate requested) by smtp.subspace.kernel.org (Postfix) with ESMTPS id BFBED16411 for ; Wed, 3 Jan 2024 04:08:53 +0000 (UTC) Authentication-Results: smtp.subspace.kernel.org; dmarc=pass (p=none dis=none) header.from=intel.com Authentication-Results: smtp.subspace.kernel.org; spf=pass smtp.mailfrom=intel.com DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/simple; d=intel.com; i=@intel.com; q=dns/txt; s=Intel; t=1704254935; x=1735790935; h=from:to:cc:subject:date:message-id:mime-version: content-transfer-encoding; bh=gL8PNc7UX1Ihfr6zd2VLwkqFMWTEGULO2tEaZ4SoE8w=; b=mLcMualvh7JMY1ckHuHjNXfMy6VcvwT14/l1QU6DxBfy/0xMTQmowptw 3Fle/FS+Idch+SdRv5maiFGaD2ydV0ntKVJ/70+6/YsNVB497H4bcy7+X ftmREYPXqTv6uQ5b64BrLoFtEjJfG6fl/RfBN+8xFsyiRGZOxwaOzClmz N+N03rrBv4bCNtqQ3YkzCBloa2QWCKv2NXD6ni5gntCox2icT8BbsFuJx Fg9t1xlhuWVa8o6kEQw7qw5/x48B7Ecu+CcIdhQqhAeXO8bt8HKLN79c1 xd1XY6c6tUwPDzJHr1EvejLyStis+Gp4DrWeAsO/pcyiM4HirbsRTLVBo A==; X-IronPort-AV: E=McAfee;i="6600,9927,10941"; a="3725450" X-IronPort-AV: E=Sophos;i="6.04,326,1695711600"; d="scan'208";a="3725450" Received: from fmsmga005.fm.intel.com ([10.253.24.32]) by orvoesa103.jf.intel.com with ESMTP/TLS/ECDHE-RSA-AES256-GCM-SHA384; 02 Jan 2024 20:08:53 -0800 X-ExtLoop1: 1 X-IronPort-AV: E=McAfee;i="6600,9927,10941"; a="1111241837" X-IronPort-AV: E=Sophos;i="6.04,326,1695711600"; d="scan'208";a="1111241837" Received: from myep-mobl1.png.intel.com ([10.107.5.97]) by fmsmga005-auth.fm.intel.com with ESMTP/TLS/ECDHE-RSA-AES256-GCM-SHA384; 02 Jan 2024 20:08:50 -0800 From: Mun Chun Yep To: herbert@gondor.apana.org.au Cc: linux-crypto@vger.kernel.org, qat-linux@intel.com, Mun Chun Yep Subject: [PATCH 0/9] crypto: qat - improve recovery flows Date: Wed, 3 Jan 2024 12:07:13 +0800 Message-Id: <20240103040722.14467-1-mun.chun.yep@intel.com> X-Mailer: git-send-email 2.34.1 Precedence: bulk X-Mailing-List: linux-crypto@vger.kernel.org List-Id: List-Subscribe: List-Unsubscribe: MIME-Version: 1.0 Content-Transfer-Encoding: 8bit This set improves the error recovery flows in the QAT drivers and adds a mechanism to test it through an heartbeat simulator. When a QAT device reports either a fatal error, or an AER fatal error, or fails an heartbeat check, the PF driver sends an error notification to the VFs through PFVF comms and if `auto_reset` is enabled then the device goes through reset flows for error recovery. If SRIOV is enabled when an error is encountered, this is re-enabled after the reset cycle is done. Damian Muszynski (2): crypto: qat - add heartbeat error simulator crypto: qat - add auto reset on error Furong Zhou (3): crypto: qat - add fatal error notify method crypto: qat - disable arbitration before reset crypto: qat - limit heartbeat notifications Mun Chun Yep (4): crypto: qat - update PFVF protocol for recovery crypto: qat - re-enable sriov after pf reset crypto: qat - add fatal error notification crypto: qat - improve aer error reset handling Documentation/ABI/testing/debugfs-driver-qat | 26 +++++ Documentation/ABI/testing/sysfs-driver-qat | 20 ++++ drivers/crypto/intel/qat/Kconfig | 15 +++ drivers/crypto/intel/qat/qat_common/Makefile | 3 + .../intel/qat/qat_common/adf_accel_devices.h | 2 + drivers/crypto/intel/qat/qat_common/adf_aer.c | 107 +++++++++++++++++- .../intel/qat/qat_common/adf_cfg_strings.h | 1 + .../intel/qat/qat_common/adf_common_drv.h | 10 ++ .../intel/qat/qat_common/adf_heartbeat.c | 20 +++- .../intel/qat/qat_common/adf_heartbeat.h | 21 ++++ .../qat/qat_common/adf_heartbeat_dbgfs.c | 52 +++++++++ .../qat/qat_common/adf_heartbeat_inject.c | 76 +++++++++++++ .../intel/qat/qat_common/adf_hw_arbiter.c | 25 ++++ .../crypto/intel/qat/qat_common/adf_init.c | 12 ++ drivers/crypto/intel/qat/qat_common/adf_isr.c | 7 +- .../intel/qat/qat_common/adf_pfvf_msg.h | 7 +- .../intel/qat/qat_common/adf_pfvf_pf_msg.c | 64 ++++++++++- .../intel/qat/qat_common/adf_pfvf_pf_msg.h | 21 ++++ .../intel/qat/qat_common/adf_pfvf_pf_proto.c | 8 ++ .../intel/qat/qat_common/adf_pfvf_vf_proto.c | 6 + .../crypto/intel/qat/qat_common/adf_sriov.c | 38 ++++++- .../crypto/intel/qat/qat_common/adf_sysfs.c | 37 ++++++ 22 files changed, 565 insertions(+), 13 deletions(-) create mode 100644 drivers/crypto/intel/qat/qat_common/adf_heartbeat_inject.c -- 2.34.1