Received: by 2002:a05:6a10:16a7:0:0:0:0 with SMTP id gp39csp106541pxb; Tue, 10 Nov 2020 21:47:12 -0800 (PST) X-Google-Smtp-Source: ABdhPJxvy25ReGoG/BI2sKaTeip8dZVVf/9P1iXCRh8c1e5sab5/NKm1mpD4QMB6BG4kV6Qqfnbl X-Received: by 2002:aa7:d356:: with SMTP id m22mr25182491edr.270.1605073632445; Tue, 10 Nov 2020 21:47:12 -0800 (PST) ARC-Seal: i=1; a=rsa-sha256; t=1605073632; cv=none; d=google.com; s=arc-20160816; b=QOwHavdM5SUl0wBD/67eYA92IepV3ZB8muwSXkE0v/rKetMfzOS75y8vxSlvBN8YWC 7VTKR7RlNJ7eeR8owWxAOIOWgBesUz8gCqoXN7yM5FecCUGI3RypzQnB8Fq7Hf/Q3F6U SZ/KjE2W6CmAhPHlDcbNbjKmvntwsH7qBptO9OmhMnQ3BfQy4fLwpy5TUVf8KhsMEitH Gy4oF4+I8mAKXgtW5pllSZ1HJ4aMYVD/Qk0ecx15LOleZkBByTS9oKMUJA75PMYhOJ+8 t0FDe1peuIv5pd4f7BjMmimh7LkmpPsv/v/mLDQKspWoRzwAm2oubjm3PH/6oKY8vw6c QvYA== ARC-Message-Signature: i=1; a=rsa-sha256; c=relaxed/relaxed; d=google.com; s=arc-20160816; h=list-id:precedence:content-transfer-encoding:mime-version :references:in-reply-to:message-id:date:subject:cc:to:from :ironport-sdr:ironport-sdr; bh=y8APrFV6dOvBmYfvwHcm+1do9UtirqcTad6l20FaTs0=; b=xtKX2vuGRHeV/Ki8ksK+ecbUXwvqNZChIBsnMtt2sGDhCXJeGmAOtj/cdj7zJWlQya GlkvPWm0xJMvNDPp3QqXTDpJpOf+xfR4DiBJMhaexWx25G5aStIOUhhcQ4IQ0kI1koJt UmEaY9o9oziAzG6HWOQ4v3H7TTmTD/ArKzmCFEIApHwmkYVRcTeW283VbxlnyONVPyVo fwWNGL6wNYFrrHaVgUOsV3H2xo1DlX4Q/QKXLcBj8nLyQt9IB9b1B/XuyOtS6AwRBmTR 2/YaZ1uxV9uGxM0jXTIGjOdWHBddZyiJompvcuhA5gaG3n1raxTIWZdKS71/QZmGAcTe xp2w== ARC-Authentication-Results: i=1; mx.google.com; spf=pass (google.com: domain of linux-kernel-owner@vger.kernel.org designates 23.128.96.18 as permitted sender) smtp.mailfrom=linux-kernel-owner@vger.kernel.org; dmarc=fail (p=NONE sp=NONE dis=NONE) header.from=intel.com Return-Path: Received: from vger.kernel.org (vger.kernel.org. [23.128.96.18]) by mx.google.com with ESMTP id l30si705668edl.99.2020.11.10.21.46.50; Tue, 10 Nov 2020 21:47:12 -0800 (PST) Received-SPF: pass (google.com: domain of linux-kernel-owner@vger.kernel.org designates 23.128.96.18 as permitted sender) client-ip=23.128.96.18; Authentication-Results: mx.google.com; spf=pass (google.com: domain of linux-kernel-owner@vger.kernel.org designates 23.128.96.18 as permitted sender) smtp.mailfrom=linux-kernel-owner@vger.kernel.org; dmarc=fail (p=NONE sp=NONE dis=NONE) header.from=intel.com Received: (majordomo@vger.kernel.org) by vger.kernel.org via listexpand id S1726130AbgKKFop (ORCPT + 99 others); Wed, 11 Nov 2020 00:44:45 -0500 Received: from mga12.intel.com ([192.55.52.136]:54150 "EHLO mga12.intel.com" rhost-flags-OK-OK-OK-OK) by vger.kernel.org with ESMTP id S1725904AbgKKFoH (ORCPT ); Wed, 11 Nov 2020 00:44:07 -0500 IronPort-SDR: Gn52N7/v4HVmeOcNx/VTWozOXjE+nFcEBV3BMOiGgUQnu8+fYMwh9/tZBqE0Fu8xXubLS8rlAA XC6mOlIGjAiw== X-IronPort-AV: E=McAfee;i="6000,8403,9801"; a="149372962" X-IronPort-AV: E=Sophos;i="5.77,468,1596524400"; d="scan'208";a="149372962" X-Amp-Result: SKIPPED(no attachment in message) X-Amp-File-Uploaded: False Received: from fmsmga002.fm.intel.com ([10.253.24.26]) by fmsmga106.fm.intel.com with ESMTP/TLS/ECDHE-RSA-AES256-GCM-SHA384; 10 Nov 2020 21:44:05 -0800 IronPort-SDR: y+L7DoqIphxCfgbmHMu89ucgdp18Wkpq5lLB1IOm/8A8IMmV0QO3615+uwLRN+SgFXL5NsWGbF 5JoKjMFRp7Og== X-IronPort-AV: E=Sophos;i="5.77,468,1596524400"; d="scan'208";a="360414751" Received: from hccoutan-mobl1.amr.corp.intel.com (HELO bwidawsk-mobl5.local) ([10.252.131.159]) by fmsmga002-auth.fm.intel.com with ESMTP/TLS/ECDHE-RSA-AES256-GCM-SHA384; 10 Nov 2020 21:44:04 -0800 From: Ben Widawsky To: linux-cxl@vger.kernel.org Cc: linux-kernel@vger.kernel.org, linux-pci@vger.kernel.org, linux-acpi@vger.kernel.org, Dan Williams , Ira Weiny , Vishal Verma , "Kelley, Sean V" , Bjorn Helgaas , "Rafael J . Wysocki" , Ben Widawsky Subject: [RFC PATCH 4/9] cxl/mem: Map memory device registers Date: Tue, 10 Nov 2020 21:43:51 -0800 Message-Id: <20201111054356.793390-5-ben.widawsky@intel.com> X-Mailer: git-send-email 2.29.2 In-Reply-To: <20201111054356.793390-1-ben.widawsky@intel.com> References: <20201111054356.793390-1-ben.widawsky@intel.com> MIME-Version: 1.0 Content-Transfer-Encoding: 8bit Precedence: bulk List-ID: X-Mailing-List: linux-kernel@vger.kernel.org All the necessary bits are initialized in order to find and map the register space for CXL Memory Devices. This is accomplished by using the Register Locator DVSEC (CXL 2.0 - 8.1.9.1) to determine which PCI BAR to use, and how much of an offset from that BAR should be added. If the memory device registers are found and mapped a new internal data structure tracking device state is allocated. Signed-off-by: Ben Widawsky --- drivers/cxl/mem.c | 68 +++++++++++++++++++++++++++++++++++++++++++---- drivers/cxl/pci.h | 6 +++++ 2 files changed, 69 insertions(+), 5 deletions(-) diff --git a/drivers/cxl/mem.c b/drivers/cxl/mem.c index aa7d881fa47b..8d9b9ab6c5ea 100644 --- a/drivers/cxl/mem.c +++ b/drivers/cxl/mem.c @@ -7,9 +7,49 @@ #include "pci.h" struct cxl_mem { + struct pci_dev *pdev; void __iomem *regs; }; +static struct cxl_mem *cxl_mem_create(struct pci_dev *pdev, u32 reg_lo, u32 reg_hi) +{ + struct device *dev = &pdev->dev; + struct cxl_mem *cxlm; + void __iomem *regs; + u64 offset; + u8 bar; + int rc; + + offset = ((u64)reg_hi << 32) | (reg_lo & 0xffff0000); + bar = reg_lo & 0x7; + + /* Basic sanity check that BAR is big enough */ + if (pci_resource_len(pdev, bar) < offset) { + dev_err(dev, "bar%d: %pr: too small (offset: %#llx)\n", + bar, &pdev->resource[bar], (unsigned long long) offset); + return ERR_PTR(-ENXIO); + } + + rc = pcim_iomap_regions(pdev, 1 << bar, pci_name(pdev)); + if (rc != 0) { + dev_err(dev, "failed to map registers\n"); + return ERR_PTR(-ENXIO); + } + + cxlm = devm_kzalloc(&pdev->dev, sizeof(*cxlm), GFP_KERNEL); + if (!cxlm) { + dev_err(dev, "No memory available\n"); + return ERR_PTR(-ENOMEM); + } + + regs = pcim_iomap_table(pdev)[bar]; + cxlm->pdev = pdev; + cxlm->regs = regs + offset; + + dev_dbg(dev, "Mapped CXL Memory Device resource\n"); + return cxlm; +} + static int cxl_mem_dvsec(struct pci_dev *pdev, int dvsec) { int pos; @@ -34,9 +74,9 @@ static int cxl_mem_dvsec(struct pci_dev *pdev, int dvsec) static int cxl_mem_probe(struct pci_dev *pdev, const struct pci_device_id *id) { + struct cxl_mem *cxlm = ERR_PTR(-ENXIO); struct device *dev = &pdev->dev; - struct cxl_mem *cxlm; - int rc, regloc; + int rc, regloc, i; rc = cxl_bus_prepared(pdev); if (rc != 0) { @@ -44,15 +84,33 @@ static int cxl_mem_probe(struct pci_dev *pdev, const struct pci_device_id *id) return rc; } + rc = pcim_enable_device(pdev); + if (rc) + return rc; + regloc = cxl_mem_dvsec(pdev, PCI_DVSEC_ID_CXL_REGLOC); if (!regloc) { dev_err(dev, "register location dvsec not found\n"); return -ENXIO; } + regloc += 0xc; /* Skip DVSEC + reserved fields */ + + for (i = regloc; i < regloc + 0x24; i += 8) { + u32 reg_lo, reg_hi; + + pci_read_config_dword(pdev, i, ®_lo); + pci_read_config_dword(pdev, i + 4, ®_hi); + + if (CXL_REGLOG_IS_MEMDEV(reg_lo)) { + cxlm = cxl_mem_create(pdev, reg_lo, reg_hi); + break; + } + } + + if (IS_ERR(cxlm)) + return -ENXIO; - cxlm = devm_kzalloc(dev, sizeof(*cxlm), GFP_KERNEL); - if (!cxlm) - return -ENOMEM; + pci_set_drvdata(pdev, cxlm); return 0; } diff --git a/drivers/cxl/pci.h b/drivers/cxl/pci.h index beb03921e6da..be87f62e9132 100644 --- a/drivers/cxl/pci.h +++ b/drivers/cxl/pci.h @@ -12,4 +12,10 @@ #define PCI_DVSEC_ID_CXL 0x0 #define PCI_DVSEC_ID_CXL_REGLOC 0x8 +#define CXL_REGLOG_RBI_EMPTY 0 +#define CXL_REGLOG_RBI_COMPONENT 1 +#define CXL_REGLOG_RBI_VIRT 2 +#define CXL_REGLOG_RBI_MEMDEV 3 +#define CXL_REGLOG_IS_MEMDEV(x) ((((x) >> 8) & 0xff) == CXL_REGLOG_RBI_MEMDEV) + #endif /* __CXL_PCI_H__ */ -- 2.29.2