Return-Path: Received: (majordomo@vger.kernel.org) by vger.kernel.org via listexpand id S1753749AbdL1OkG (ORCPT ); Thu, 28 Dec 2017 09:40:06 -0500 Received: from fllnx210.ext.ti.com ([198.47.19.17]:42990 "EHLO fllnx210.ext.ti.com" rhost-flags-OK-OK-OK-OK) by vger.kernel.org with ESMTP id S1753131AbdL1OkE (ORCPT ); Thu, 28 Dec 2017 09:40:04 -0500 Subject: Re: [PATCH v6 06/18] PCI: designware-ep: Add generic function for raising MSI irq To: Niklas Cassel References: <20171219232940.659-1-niklas.cassel@axis.com> <20171219232940.659-7-niklas.cassel@axis.com> <5fc44bf0-9d0e-e905-32fb-449d9ed1b01a@ti.com> <20171227222909.GA14106@axis.com> CC: Jingoo Han , Joao Pinto , Lorenzo Pieralisi , Bjorn Helgaas , , From: Kishon Vijay Abraham I Message-ID: <672cfdc5-9e3d-311f-6307-0aedcf5d28ad@ti.com> Date: Thu, 28 Dec 2017 20:09:50 +0530 User-Agent: Mozilla/5.0 (X11; Linux x86_64; rv:45.0) Gecko/20100101 Thunderbird/45.7.0 MIME-Version: 1.0 In-Reply-To: Content-Type: text/plain; charset="windows-1252" Content-Transfer-Encoding: 7bit X-EXCLAIMER-MD-CONFIG: e1e8a2fd-e40a-4ac6-ac9b-f7e9cc9ee180 Sender: linux-kernel-owner@vger.kernel.org List-ID: X-Mailing-List: linux-kernel@vger.kernel.org Content-Length: 5933 Lines: 147 Hi Niklas, On Thursday 28 December 2017 01:36 PM, Kishon Vijay Abraham I wrote: > Hi Niklas, > > On Thursday 28 December 2017 03:59 AM, Niklas Cassel wrote: >> On Tue, Dec 26, 2017 at 06:20:54PM +0530, Kishon Vijay Abraham I wrote: >>> Hi Niklas, >> >> Hello Kishon >> >>> >>> On Wednesday 20 December 2017 04:59 AM, Niklas Cassel wrote: >>>> Add a generic function for raising MSI irqs that can be used by all >>>> DWC based controllers. >>>> >>>> Note that certain controllers, like DRA7xx, have a special convenience >>>> register for raising MSI irqs that doesn't require you to explicitly map >>>> the MSI address. Therefore, it is likely that certain drivers will >>>> not use this generic function, even if they can. >>>> >>>> Signed-off-by: Niklas Cassel >>>> --- >>>> drivers/pci/dwc/pcie-designware-ep.c | 35 +++++++++++++++++++++++++++++++++++ >>>> drivers/pci/dwc/pcie-designware.h | 9 +++++++++ >>>> 2 files changed, 44 insertions(+) >>>> >>>> diff --git a/drivers/pci/dwc/pcie-designware-ep.c b/drivers/pci/dwc/pcie-designware-ep.c >>>> index 700ed2f4becf..c5aa1cac5041 100644 >>>> --- a/drivers/pci/dwc/pcie-designware-ep.c >>>> +++ b/drivers/pci/dwc/pcie-designware-ep.c >>>> @@ -282,6 +282,41 @@ static const struct pci_epc_ops epc_ops = { >>>> .stop = dw_pcie_ep_stop, >>>> }; >>>> >>>> +int dw_pcie_ep_raise_msi_irq(struct dw_pcie_ep *ep, >>>> + u8 interrupt_num) >>>> +{ >>>> + struct dw_pcie *pci = to_dw_pcie_from_ep(ep); >>>> + struct pci_epc *epc = ep->epc; >>>> + u16 msg_ctrl, msg_data; >>>> + u32 msg_addr_lower, msg_addr_upper; >>>> + u64 msg_addr; >>>> + bool has_upper; >>>> + int ret; >>>> + >>>> + /* Raise MSI per the PCI Local Bus Specification Revision 3.0, 6.8.1. */ >>>> + msg_ctrl = dw_pcie_readw_dbi(pci, MSI_MESSAGE_CONTROL); >>>> + has_upper = !!(msg_ctrl & PCI_MSI_FLAGS_64BIT); >>>> + msg_addr_lower = dw_pcie_readl_dbi(pci, MSI_MESSAGE_ADDR_L32); >>>> + if (has_upper) { >>>> + msg_addr_upper = dw_pcie_readl_dbi(pci, MSI_MESSAGE_ADDR_U32); >>>> + msg_data = dw_pcie_readw_dbi(pci, MSI_MESSAGE_DATA_64); >>>> + } else { >>>> + msg_addr_upper = 0; >>>> + msg_data = dw_pcie_readw_dbi(pci, MSI_MESSAGE_DATA_32); >>>> + } >>>> + msg_addr = ((u64) msg_addr_upper) << 32 | msg_addr_lower; >>>> + ret = dw_pcie_ep_map_addr(epc, ep->msi_mem_phys, msg_addr, >>>> + epc->mem->page_size); >>>> + if (ret) >>>> + return ret; >>>> + >>>> + writel(msg_data | (interrupt_num - 1), ep->msi_mem); >>> >>> Shouldn't this be msg_data + (interrupt_num - 1)? >> >> I'm not quite sure about this, but if there is a pending irq, >> not yet processed by the RC, the msg_data we read out in this >> function should have a bit set, matching the pending irq. > > IIUC, the msg_data that we read here should not depend on the pending irq on > the RC side. msg_data should have the starting MSI vector number assigned by RC > for that EP device. (msg.data = pos; in dw_msi_setup_msg() also seem to suggest > the same). >> >> If that irq is the same as the irq we are trying to raise, >> doing an addition will produce a bogus vector number, >> but a bitwise or should work. > > if msg_data has the starting MSI vector, doing an addition should get to the > correct MSI vector. >> >> For that reason, I think that doing bitwise or seems safer. >> However, other than this case, I don't see why it should >> matter if we do an addition or a bitwise or. >> >> Are you having some problem with the code? >> It seems to be working fine on ARTPEC-6: >> >> # ./pcitest -m 1 >> MSI1: OKAY >> # ./pcitest -m 2 >> MSI2: OKAY >> # ./pcitest -m 3 >> MSI3: OKAY >> # ./pcitest -m 4 >> MSI4: OKAY >> # ./pcitest -m 5 >> MSI5: OKAY >> # ./pcitest -m 6 >> MSI6: OKAY >> # ./pcitest -m 7 >> MSI7: OKAY >> # ./pcitest -m 8 >> MSI8: OKAY >> # ./pcitest -m 9 >> MSI9: OKAY >> # cat /proc/interrupts | grep -i msi >> 82: 9 0 GIC-0 180 Level artpec6-pcie-msi >> 188: 1 0 PCI-MSI 16 Edge pci-endpoint-test >> 189: 1 0 PCI-MSI 17 Edge pci-endpoint-test >> 190: 1 0 PCI-MSI 18 Edge pci-endpoint-test >> 191: 1 0 PCI-MSI 19 Edge pci-endpoint-test >> 192: 1 0 PCI-MSI 20 Edge pci-endpoint-test >> 193: 1 0 PCI-MSI 21 Edge pci-endpoint-test >> 194: 1 0 PCI-MSI 22 Edge pci-endpoint-test >> 195: 1 0 PCI-MSI 23 Edge pci-endpoint-test >> 196: 1 0 PCI-MSI 24 Edge pci-endpoint-test >> 197: 0 0 PCI-MSI 25 Edge pci-endpoint-test >> 198: 0 0 PCI-MSI 26 Edge pci-endpoint-test >> 199: 0 0 PCI-MSI 27 Edge pci-endpoint-test >> 200: 0 0 PCI-MSI 28 Edge pci-endpoint-test >> 201: 0 0 PCI-MSI 29 Edge pci-endpoint-test >> 202: 0 0 PCI-MSI 30 Edge pci-endpoint-test >> 203: 0 0 PCI-MSI 31 Edge pci-endpoint-test >> >> From EP: >> irq: 1 read msg_data: 0x10 writing: 0x10 >> irq: 2 read msg_data: 0x10 writing: 0x11 >> irq: 3 read msg_data: 0x10 writing: 0x12 >> irq: 4 read msg_data: 0x10 writing: 0x13 >> irq: 5 read msg_data: 0x10 writing: 0x14 >> irq: 6 read msg_data: 0x10 writing: 0x15 >> irq: 7 read msg_data: 0x10 writing: 0x16 >> irq: 8 read msg_data: 0x10 writing: 0x17 >> irq: 9 read msg_data: 0x10 writing: 0x18 > > since your msg_data is 0x10, you are not facing the issue. What if it's 0x1? In > my case If I have Gustavo's patch series applied, msg_data has a value of 0x1 > and I don't get certain MSI interrupts. I think Gustavo's series doesn't align the MSI vector properly. That's why I get 0x1. So your patch is fine as such. Thanks Kishon