Received: by 2002:ac0:950e:0:0:0:0:0 with SMTP id f14csp570626imc; Sat, 16 Mar 2019 09:05:07 -0700 (PDT) X-Google-Smtp-Source: APXvYqxTH5VfH8ogBtsA82f984+5rKBVCGceY5819u2s5TJ95OD8luei3dFO2/d1I8ERV956k4QY X-Received: by 2002:a17:902:9688:: with SMTP id n8mr10310693plp.133.1552752307487; Sat, 16 Mar 2019 09:05:07 -0700 (PDT) ARC-Seal: i=1; a=rsa-sha256; t=1552752307; cv=none; d=google.com; s=arc-20160816; b=A86CFSky843MEZgBHHrWPP8o6Ivmi+LVRTnxwM4w9ByV+PUuTHUTsLtTLd0tlZUY59 /FxTJaFFoID+wx3CvmiHGNzR9ArjExK0L4eoNPlU29/P7cLDrtHviPsVZRcjsso4hHXo zC61Ngplh7OmZxX8YwTltgs09AfUmzxWd70nXRrbQ22EPhnCSQxWz+vsgeAA6UuVI97u RoJ//XFqQgUqjOPKaN6UqYX++kN2eg+V66ucOYlrLysUGUHD4ypZ86wxLUVxr5OF/XIy B+Y6IG3eUZ8KRFT5/vBg+M5Ke9hQJPLapm628nSl6OZ2b3JZkPhq0jEXsJmcIDOvmY1H J3cA== ARC-Message-Signature: i=1; a=rsa-sha256; c=relaxed/relaxed; d=google.com; s=arc-20160816; h=list-id:precedence:sender:content-transfer-encoding:mime-version :message-id:date:subject:cc:to:from; bh=Qg29VlbihzERM7BKduV/pRqAoupFDX0AS+YC+c188uI=; b=YuRBquccWA6AsMW5kFzDQ3uvW9U9K+QscwDIL8pxKGOp4IZCsYJGW6qWM4nh8Cxhyi xt3g2pTJMlzL3nknk6PRpQZycp8VinAAoJtUudE+entvoa/eqNUR5yR4lCew0YrQivZi iAFam9yFGLdHZwAiPzmyN3vXmuMI9kIyyNnZ4YW+bazaJ/+PXi5tG5yNnv3cs0Foqo/o wcOCANfKjnwK4k+FYMAs6q7VR2cGLaD5HmhhjRplusetCKop07WARuLpFM2u05OelXLc CGlY3ZzTxJIOMq4NWF2W519PVvctZ7PaJzs9o3/qo342nmNVREd8kmbf+DqHSj6nJG0i 10fQ== ARC-Authentication-Results: i=1; mx.google.com; spf=pass (google.com: best guess record for domain of linux-kernel-owner@vger.kernel.org designates 209.132.180.67 as permitted sender) smtp.mailfrom=linux-kernel-owner@vger.kernel.org Return-Path: Received: from vger.kernel.org (vger.kernel.org. [209.132.180.67]) by mx.google.com with ESMTP id 189si4490971pgb.412.2019.03.16.09.04.51; Sat, 16 Mar 2019 09:05:07 -0700 (PDT) Received-SPF: pass (google.com: best guess record for domain of linux-kernel-owner@vger.kernel.org designates 209.132.180.67 as permitted sender) client-ip=209.132.180.67; Authentication-Results: mx.google.com; spf=pass (google.com: best guess record for domain of linux-kernel-owner@vger.kernel.org designates 209.132.180.67 as permitted sender) smtp.mailfrom=linux-kernel-owner@vger.kernel.org Received: (majordomo@vger.kernel.org) by vger.kernel.org via listexpand id S1726954AbfCPQDk (ORCPT + 99 others); Sat, 16 Mar 2019 12:03:40 -0400 Received: from relay7-d.mail.gandi.net ([217.70.183.200]:36997 "EHLO relay7-d.mail.gandi.net" rhost-flags-OK-OK-OK-OK) by vger.kernel.org with ESMTP id S1726105AbfCPQDk (ORCPT ); Sat, 16 Mar 2019 12:03:40 -0400 X-Originating-IP: 88.190.179.123 Received: from localhost (unknown [88.190.179.123]) (Authenticated sender: repk@triplefau.lt) by relay7-d.mail.gandi.net (Postfix) with ESMTPSA id 399B720007; Sat, 16 Mar 2019 16:03:35 +0000 (UTC) From: Remi Pommarel To: Thomas Petazzoni Cc: Lorenzo Pieralisi , Bjorn Helgaas , Ellie Reeves , linux-pci@vger.kernel.org, linux-arm-kernel@lists.infradead.org, linux-kernel@vger.kernel.org, Remi Pommarel Subject: [PATCH v2] PCI: aardvark: Use LTSSM state to build link training flag Date: Sat, 16 Mar 2019 17:12:43 +0100 Message-Id: <20190316161243.29517-1-repk@triplefau.lt> X-Mailer: git-send-email 2.20.1 MIME-Version: 1.0 Content-Transfer-Encoding: 8bit Sender: linux-kernel-owner@vger.kernel.org Precedence: bulk List-ID: X-Mailing-List: linux-kernel@vger.kernel.org The PCI_EXP_LNKSTA_LT flag in the emulated root device's PCI_EXP_LNKSTA config register does not reflect the actual link training state and is always cleared. The Link Training and Status State Machine (LTSSM) flag in LMI config register could be used as a link training indicator. Indeed if the LTSSM is in L0 or upper state then link training has completed (see [1]). Unfortunately because setting the PCI_EXP_LINCTL_RL flag does not instantly imply a LTSSM state change (e.g. L0s to recovery state transition takes some time), LTSSM can be in L0 but link training has not finished yet. Thus a lower L0 LTSSM state followed by a L0 or upper state sequence has to be seen to be sure that link training has been done. Because one may not call a pcie conf register read on LNKSTA after doing a retrain link or may miss the link down state due to timing, a 20ms timeout is used. Passing this timeout link is considered retrained. This fixes boot hang or kernel panic with the following callstack due to ASPM setup doing a link re-train and polling for PCI_EXP_LNKSTA_LT flag to be cleared before using it. -------------------- 8< ------------------- [ 0.915389] dump_backtrace+0x0/0x140 [ 0.915391] show_stack+0x14/0x20 [ 0.915393] dump_stack+0x90/0xb4 [ 0.915394] panic+0x134/0x2c0 [ 0.915396] nmi_panic+0x6c/0x70 [ 0.915398] arm64_serror_panic+0x74/0x80 [ 0.915400] is_valid_bugaddr+0x0/0x8 [ 0.915402] el1_error+0x7c/0xe4 [ 0.915404] advk_pcie_rd_conf+0x4c/0x250 [ 0.915406] pci_bus_read_config_word+0x7c/0xd0 [ 0.915408] pcie_capability_read_word+0x90/0xc8 [ 0.915410] pcie_get_aspm_reg+0x68/0x118 [ 0.915412] pcie_aspm_init_link_state+0x460/0xa98 [ 0.915414] pci_scan_slot+0xe8/0x100 [ 0.915416] pci_scan_child_bus_extend+0x50/0x288 [ 0.915418] pci_scan_bridge_extend+0x348/0x4f0 [ 0.915420] pci_scan_child_bus_extend+0x1dc/0x288 [ 0.915423] pci_scan_root_bus_bridge+0xc4/0xe0 [ 0.915424] pci_host_probe+0x14/0xa8 [ 0.915426] advk_pcie_probe+0x838/0x910 [...] -------------------- 8< ------------------- [1] "PCI Express Base Specification", REV. 2.1 PCI Express, March 4 2009, Table 4-7 Signed-off-by: Remi Pommarel --- Changes since v1: - Rename retraining flag field - Fix DEVCTL register writing --- drivers/pci/controller/pci-aardvark.c | 33 ++++++++++++++++++++++++++- 1 file changed, 32 insertions(+), 1 deletion(-) diff --git a/drivers/pci/controller/pci-aardvark.c b/drivers/pci/controller/pci-aardvark.c index eb58dfdaba1b..47b707b5fc2c 100644 --- a/drivers/pci/controller/pci-aardvark.c +++ b/drivers/pci/controller/pci-aardvark.c @@ -180,6 +180,7 @@ #define LINK_WAIT_MAX_RETRIES 10 #define LINK_WAIT_USLEEP_MIN 90000 #define LINK_WAIT_USLEEP_MAX 100000 +#define LINK_RETRAIN_DELAY_MAX (20 * HZ / 1000) /* 20 ms */ #define MSI_IRQ_NUM 32 @@ -199,6 +200,8 @@ struct advk_pcie { u16 msi_msg; int root_bus_nr; struct pci_bridge_emul bridge; + unsigned long rl_deadline; /* Retrain link jiffies deadline */ + u8 rl_asked; /* Retraining has been asked and is in transition */ }; static inline void advk_writel(struct advk_pcie *pcie, u32 val, u64 reg) @@ -400,6 +403,19 @@ static int advk_pcie_wait_pio(struct advk_pcie *pcie) return -ETIMEDOUT; } +static int advk_pcie_link_retraining(struct advk_pcie *pcie) +{ + if (!advk_pcie_link_up(pcie)) { + pcie->rl_asked = 0; + return 1; + } + + if (pcie->rl_asked && time_before(jiffies, pcie->rl_deadline)) + return 1; + + pcie->rl_asked = 0; + return 0; +} static pci_bridge_emul_read_status_t advk_pci_bridge_emul_pcie_conf_read(struct pci_bridge_emul *bridge, @@ -426,11 +442,19 @@ advk_pci_bridge_emul_pcie_conf_read(struct pci_bridge_emul *bridge, return PCI_BRIDGE_EMUL_HANDLED; } + case PCI_EXP_LNKCTL: { + u32 val = advk_readl(pcie, PCIE_CORE_PCIEXP_CAP + reg) & + ~(PCI_EXP_LNKSTA_LT << 16); + if (advk_pcie_link_retraining(pcie)) + val |= (PCI_EXP_LNKSTA_LT << 16); + *value = val; + return PCI_BRIDGE_EMUL_HANDLED; + } + case PCI_CAP_LIST_ID: case PCI_EXP_DEVCAP: case PCI_EXP_DEVCTL: case PCI_EXP_LNKCAP: - case PCI_EXP_LNKCTL: *value = advk_readl(pcie, PCIE_CORE_PCIEXP_CAP + reg); return PCI_BRIDGE_EMUL_HANDLED; default: @@ -447,8 +471,15 @@ advk_pci_bridge_emul_pcie_conf_write(struct pci_bridge_emul *bridge, switch (reg) { case PCI_EXP_DEVCTL: + advk_writel(pcie, new, PCIE_CORE_PCIEXP_CAP + reg); + break; + case PCI_EXP_LNKCTL: advk_writel(pcie, new, PCIE_CORE_PCIEXP_CAP + reg); + if (new & PCI_EXP_LNKCTL_RL) { + pcie->rl_asked = 1; + pcie->rl_deadline = jiffies + LINK_RETRAIN_DELAY_MAX; + } break; case PCI_EXP_RTCTL: -- 2.20.1