Received: by 2002:ad5:4acb:0:0:0:0:0 with SMTP id n11csp475498imw; Thu, 14 Jul 2022 05:11:43 -0700 (PDT) X-Google-Smtp-Source: AGRyM1tRz79y6/NACsvKsRptIl3/JEt5P2U5NU9vz/zTqZDwmzsnGyjxmSjkxEq4Fr/gGQn7xttP X-Received: by 2002:a05:6402:1909:b0:43a:64bb:9f27 with SMTP id e9-20020a056402190900b0043a64bb9f27mr12152685edz.24.1657800703785; Thu, 14 Jul 2022 05:11:43 -0700 (PDT) ARC-Seal: i=1; a=rsa-sha256; t=1657800703; cv=none; d=google.com; s=arc-20160816; b=XXCcnFJq52KWoVgUuOHFNPyLXkLcp2pUq81P8F3Ee0FOqMTk95VFA0jbgP2ar77HcH W3Nrjn5Ik4fDfaOLr5UZPXumU9EBLtmqEHyl2YL+TzbX83PFTwMSqB6YBjxovJkAtji9 w95MtwMPajxSHGNJBXdi03OQbTxrTz3BIeFrG+EFX/fWyHntyoEeV4mdFNxNazaTH/Xv mkFUfh6sOfsJibl0n/FcbXxXM4AxfLlW9iKI9EIXIsHszdzQA9eC2RHS0FiPkrwq+Q/B e3OuzNhty6JNQfa/YM+SwVkDaIHxop5f4+7LJ+w+g3hNLUIeRHKTovcj8JgqR/lBcYoi vrYA== ARC-Message-Signature: i=1; a=rsa-sha256; c=relaxed/relaxed; d=google.com; s=arc-20160816; h=list-id:precedence:to:content-transfer-encoding:mime-version :references:in-reply-to:message-id:date:subject:cc:from :dkim-signature; bh=+91qwRQdf1Gi2VReV8KJaZwGiwgBEWi7UjS7Pwfeeb4=; b=Cq7FhC7D8+yP8zad5a4eAx1ONF5UWnh7oYbDeYL7Qk/XYFk8pQW8OGpTPuRnRzqk4l XJKIjsH4FhRqFs+BnCxIeuTrDUl276fFQL7wHuW8ur7tTTobKba+6slZHEGG5eZFITSm nVwxDKAW7i3iEPPt3QRXR4YiWGZqAvBs1uFocaLowLWEfdKOuxBnljUL9rNS+TBlrvrn D/5AsylTc35BglLXUMzFawI5OB+DIGWU4l7mmsNys6LG5uGoFhOAny0OjfhFHxWePDo6 DXqAbMSNgKiLhevgKsY61bbBfjTbYNtih+JpzQBdDGsUVLb85FuZjYpeyczJB1eovm7v rGUA== ARC-Authentication-Results: i=1; mx.google.com; dkim=fail header.i=@kernel.org header.s=k20201202 header.b=nhY8TlS0; spf=pass (google.com: domain of linux-kernel-owner@vger.kernel.org designates 2620:137:e000::1:20 as permitted sender) smtp.mailfrom=linux-kernel-owner@vger.kernel.org; dmarc=pass (p=NONE sp=NONE dis=NONE) header.from=kernel.org Return-Path: Received: from out1.vger.email (out1.vger.email. [2620:137:e000::1:20]) by mx.google.com with ESMTP id hs8-20020a1709073e8800b0072b4f1c6680si595855ejc.75.2022.07.14.05.11.18; Thu, 14 Jul 2022 05:11:43 -0700 (PDT) Received-SPF: pass (google.com: domain of linux-kernel-owner@vger.kernel.org designates 2620:137:e000::1:20 as permitted sender) client-ip=2620:137:e000::1:20; Authentication-Results: mx.google.com; dkim=fail header.i=@kernel.org header.s=k20201202 header.b=nhY8TlS0; spf=pass (google.com: domain of linux-kernel-owner@vger.kernel.org designates 2620:137:e000::1:20 as permitted sender) smtp.mailfrom=linux-kernel-owner@vger.kernel.org; dmarc=pass (p=NONE sp=NONE dis=NONE) header.from=kernel.org Received: (majordomo@vger.kernel.org) by vger.kernel.org via listexpand id S239156AbiGNMHn (ORCPT + 99 others); Thu, 14 Jul 2022 08:07:43 -0400 Received: from lindbergh.monkeyblade.net ([23.128.96.19]:48858 "EHLO lindbergh.monkeyblade.net" rhost-flags-OK-OK-OK-OK) by vger.kernel.org with ESMTP id S238970AbiGNMGh (ORCPT ); Thu, 14 Jul 2022 08:06:37 -0400 Received: from dfw.source.kernel.org (dfw.source.kernel.org [IPv6:2604:1380:4641:c500::1]) by lindbergh.monkeyblade.net (Postfix) with ESMTPS id 50CED5C9E1 for ; Thu, 14 Jul 2022 05:06:35 -0700 (PDT) Received: from smtp.kernel.org (relay.kernel.org [52.25.139.140]) (using TLSv1.2 with cipher ECDHE-RSA-AES256-GCM-SHA384 (256/256 bits)) (No client certificate requested) by dfw.source.kernel.org (Postfix) with ESMTPS id A66EB61E68 for ; Thu, 14 Jul 2022 12:06:32 +0000 (UTC) Received: by smtp.kernel.org (Postfix) with ESMTPSA id E22ACC36AEC; Thu, 14 Jul 2022 12:06:30 +0000 (UTC) DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/simple; d=kernel.org; s=k20201202; t=1657800390; bh=WzYpG/QH8bwVkozSfvAs7xeKWEh19nsF2DzB8oRAbEA=; h=From:To:Cc:Subject:Date:In-Reply-To:References:From; b=nhY8TlS0Jl1fj9ie5IEosRZiC9uYm/AUAvDcnMMdN7bKB6yZPbU1/vKW5FQjCal69 y6LXkbWFDtVCkZQ3h7mLCvp9x/iyNI3XK5ZorK+aT79zC4yud0/QfOQTpykdjlwV43 odnVFzYiWAHO6dprbxVfz24igbMQOl1CxKrZwjaQl1+C2nDfs5PX1p2TrT+P2LmZLT 5VP28Xok+/bZR9Li2oEhudFHq0zF8GyPovBy/wKkijHUVmMsyltl7vcywTQ0bLwMuB uaoJY7gEGM/CokUGlDSVMWjZHOiHyAkJTZmu/z8767TjWJ9SG6O01Lb9YuHUOFzNhu bdcx8qAjLpSKQ== Received: from mchehab by mail.kernel.org with local (Exim 4.95) (envelope-from ) id 1oBxbw-0059tL-5M; Thu, 14 Jul 2022 13:06:28 +0100 From: Mauro Carvalho Chehab Cc: Prathap Kumar Valsan , Ashutosh Dixit , Chris Wilson , Daniel Vetter , Daniele Ceraolo Spurio , David Airlie , Jani Nikula , Joonas Lahtinen , Lucas De Marchi , Matt Atwood , Matt Roper , Mauro Carvalho Chehab , Rodrigo Vivi , Tvrtko Ursulin , dri-devel@lists.freedesktop.org, intel-gfx@lists.freedesktop.org, linux-kernel@vger.kernel.org, Niranjana Vishwanathapura , Fei Yang Subject: [PATCH v2 17/21] drm/i915: Add generic interface for tlb invalidation for XeHP Date: Thu, 14 Jul 2022 13:06:22 +0100 Message-Id: <56ae0b7a5e823736a404dcd709fbf4a1e384ef4d.1657800199.git.mchehab@kernel.org> X-Mailer: git-send-email 2.36.1 In-Reply-To: References: MIME-Version: 1.0 Content-Transfer-Encoding: 8bit X-Spam-Status: No, score=-7.7 required=5.0 tests=BAYES_00,DKIMWL_WL_HIGH, DKIM_SIGNED,DKIM_VALID,DKIM_VALID_AU,DKIM_VALID_EF,RCVD_IN_DNSWL_HI, SPF_HELO_NONE,SPF_PASS,T_SCC_BODY_TEXT_LINE autolearn=ham autolearn_force=no version=3.4.6 X-Spam-Checker-Version: SpamAssassin 3.4.6 (2021-04-09) on lindbergh.monkeyblade.net To: unlisted-recipients:; (no To-header on input) Precedence: bulk List-ID: X-Mailing-List: linux-kernel@vger.kernel.org From: Prathap Kumar Valsan Add an interface for GuC TLB actions, supporting both selective and full TLB invalidations. After this change, when GuC is enabled, tlb invalidations use GuC ct. Otherwise, use mmio interface. Signed-off-by: Prathap Kumar Valsan Cc: Niranjana Vishwanathapura Cc: Fei Yang Signed-off-by: Mauro Carvalho Chehab --- To avoid mailbombing on a large number of people, only mailing lists were C/C on the cover. See [PATCH v2 00/21] at: https://lore.kernel.org/all/cover.1657800199.git.mchehab@kernel.org/ drivers/gpu/drm/i915/gt/intel_gt_regs.h | 8 +++ drivers/gpu/drm/i915/gt/intel_tlb.c | 78 ++++++++++++++++++++++++- drivers/gpu/drm/i915/gt/intel_tlb.h | 1 + 3 files changed, 86 insertions(+), 1 deletion(-) diff --git a/drivers/gpu/drm/i915/gt/intel_gt_regs.h b/drivers/gpu/drm/i915/gt/intel_gt_regs.h index 60d6eb5f245b..52508a9c23e5 100644 --- a/drivers/gpu/drm/i915/gt/intel_gt_regs.h +++ b/drivers/gpu/drm/i915/gt/intel_gt_regs.h @@ -1054,6 +1054,14 @@ #define GEN12_GAM_DONE _MMIO(0xcf68) +#define XEHP_TLB_INV_DESC0 _MMIO(0xcf7c) +#define XEHP_TLB_INV_DESC0_ADDR_LO REG_GENMASK(31, 12) +#define XEHP_TLB_INV_DESC0_ADDR_MASK REG_GENMASK(8, 3) +#define XEHP_TLB_INV_DESC0_G REG_GENMASK(2, 1) +#define XEHP_TLB_INV_DESC0_VALID REG_BIT(0) +#define XEHP_TLB_INV_DESC1 _MMIO(0xcf80) +#define XEHP_TLB_INV_DESC0_ADDR_HI REG_GENMASK(31, 0) + #define GEN7_HALF_SLICE_CHICKEN1 _MMIO(0xe100) /* IVB GT1 + VLV */ #define GEN7_MAX_PS_THREAD_DEP (8 << 12) #define GEN7_SINGLE_SUBSCAN_DISPATCH_ENABLE (1 << 10) diff --git a/drivers/gpu/drm/i915/gt/intel_tlb.c b/drivers/gpu/drm/i915/gt/intel_tlb.c index af8cae979489..15ed83226676 100644 --- a/drivers/gpu/drm/i915/gt/intel_tlb.c +++ b/drivers/gpu/drm/i915/gt/intel_tlb.c @@ -10,6 +10,7 @@ #include "intel_gt_pm.h" #include "intel_gt_regs.h" #include "intel_tlb.h" +#include "uc/intel_guc.h" struct reg_and_bit { i915_reg_t reg; @@ -159,11 +160,16 @@ void intel_gt_invalidate_tlb_full(struct intel_gt *gt, u32 seqno) return; with_intel_gt_pm_if_awake(gt, wakeref) { + struct intel_guc *guc = >->uc.guc; + mutex_lock(>->tlb.invalidate_lock); if (tlb_seqno_passed(gt, seqno)) goto unlock; - mmio_invalidate_full(gt); + if (INTEL_GUC_SUPPORTS_TLB_INVALIDATION(guc)) + intel_guc_invalidate_tlb_full(guc, INTEL_GUC_TLB_INVAL_MODE_HEAVY); + else + mmio_invalidate_full(gt); write_seqcount_invalidate(>->tlb.seqno); unlock: @@ -171,6 +177,76 @@ void intel_gt_invalidate_tlb_full(struct intel_gt *gt, u32 seqno) } } +static bool mmio_invalidate_range(struct intel_gt *gt, u64 start, u64 length) +{ + u32 address_mask = (ilog2(length) - ilog2(I915_GTT_PAGE_SIZE_4K)); + u64 vm_total = BIT_ULL(INTEL_INFO(gt->i915)->ppgtt_size); + intel_wakeref_t wakeref; + u32 dw0, dw1; + int err; + + GEM_BUG_ON(!IS_ALIGNED(start, I915_GTT_PAGE_SIZE_4K)); + GEM_BUG_ON(!IS_ALIGNED(length, I915_GTT_PAGE_SIZE_4K)); + GEM_BUG_ON(range_overflows(start, length, vm_total)); + + dw0 = FIELD_PREP(XEHP_TLB_INV_DESC0_ADDR_LO, (lower_32_bits(start) >> 12)) | + FIELD_PREP(XEHP_TLB_INV_DESC0_ADDR_MASK, address_mask) | + FIELD_PREP(XEHP_TLB_INV_DESC0_G, 0x3) | + FIELD_PREP(XEHP_TLB_INV_DESC0_VALID, 0x1); + dw1 = upper_32_bits(start); + + err = 0; + with_intel_gt_pm_if_awake(gt, wakeref) { + struct intel_uncore *uncore = gt->uncore; + + intel_uncore_forcewake_get(uncore, FORCEWAKE_ALL); + + mutex_lock(>->tlb.invalidate_lock); + intel_uncore_write_fw(uncore, XEHP_TLB_INV_DESC1, dw1); + intel_uncore_write_fw(uncore, XEHP_TLB_INV_DESC0, dw0); + err = __intel_wait_for_register_fw(uncore, + XEHP_TLB_INV_DESC0, + XEHP_TLB_INV_DESC0_VALID, + 0, 100, 10, NULL); + mutex_unlock(>->tlb.invalidate_lock); + + intel_uncore_forcewake_put_delayed(uncore, FORCEWAKE_ALL); + } + + if (err) + drm_err_ratelimited(>->i915->drm, + "TLB invalidation response timed out\n"); + + return err == 0; +} + +bool intel_gt_invalidate_tlb_range(struct intel_gt *gt, + u64 start, u64 length) +{ + struct intel_guc *guc = >->uc.guc; + intel_wakeref_t wakeref; + + if (intel_gt_is_wedged(gt)) + return true; + + if (!INTEL_GUC_SUPPORTS_TLB_INVALIDATION_SELECTIVE(guc)) + return false; + + /*XXX: We are seeing timeouts on guc based tlb invalidations on XEHPSDV. + * Until we have a fix, use mmio + */ + if (IS_XEHPSDV(gt->i915)) + return mmio_invalidate_range(gt, start, length); + + with_intel_gt_pm_if_awake(gt, wakeref) { + intel_guc_invalidate_tlb_page_selective(guc, + INTEL_GUC_TLB_INVAL_MODE_HEAVY, + start, length); + } + + return true; +} + void intel_gt_init_tlb(struct intel_gt *gt) { mutex_init(>->tlb.invalidate_lock); diff --git a/drivers/gpu/drm/i915/gt/intel_tlb.h b/drivers/gpu/drm/i915/gt/intel_tlb.h index 46ce25bf5afe..32cc79b1d8a4 100644 --- a/drivers/gpu/drm/i915/gt/intel_tlb.h +++ b/drivers/gpu/drm/i915/gt/intel_tlb.h @@ -12,6 +12,7 @@ #include "intel_gt_types.h" void intel_gt_invalidate_tlb_full(struct intel_gt *gt, u32 seqno); +bool intel_gt_invalidate_tlb_range(struct intel_gt *gt, u64 start, u64 length); void intel_gt_init_tlb(struct intel_gt *gt); void intel_gt_fini_tlb(struct intel_gt *gt); -- 2.36.1